Intelligent computing center model development method and device oriented to popularity

Abstract

The invention provides an intelligent computing center model development method and device oriented to the common computing power, and relates to the technical field of intelligent computing centers, intelligent computing centers and computing power infrastructures. The method comprises the steps that S1, a scheduling instruction is received, and the scheduling instruction comprises a model code; s2, computing power prediction is carried out based on the data size of the model code, and required computing power information is obtained; s3, acquiring a target virtual accelerator card based on the required computing power information, wherein the target virtual accelerator card is a virtual accelerator card created based on a part of GPU computing power resources of a GPU accelerator card of the first intelligent computing center; and S4, creating a first container based on the target virtual accelerator card, wherein the first container is used for executing the model code. According to the method, the computing power resource utilization rate of the intelligent computing center can be greatly improved, and the economic cost of model development is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent computing centers, smart computing centers and computing power infrastructure, and specifically to a method and device for developing an intelligent computing center model for universal computing power. Background Art

[0002] With the rapid development of artificial intelligence technology, "intelligent computing centers" and "intelligent computing centers" have come into being.

[0003] "Intelligent computing center" refers to a facility that provides the required computing power, data and algorithms for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning) by using large-scale heterogeneous computing resources, including general computing power and intelligent computing power. The intelligent computing center covers facilities, hardware, and software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0004] “Intelligent Computing Center” includes but is not limited to “Smart Computing Center”.

[0005] "Intelligent Computing Center" refers to an artificial intelligence computing center. It is a type of computing infrastructure that is based on artificial intelligence theory, adopts artificial intelligence computing architecture, and provides computing power services, data services, and algorithm services required for artificial intelligence applications.

[0006] "Computing power" is the core of "intelligent computing center" and "intelligent computing center". It is the ability of computer equipment or computing / data center to process information. It is the ability of computer hardware and software to work together to execute certain computing needs. It is the computing power to process information data and achieve target result output. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity. It mainly provides services to society through computing power infrastructure.

[0007] The current intelligent computing center provides computing resources to users through GPU (Graphics Processing Unit) accelerator cards, so that users can develop models based on the computing resources of the intelligent computing center and reduce the cost of model development. However, in the prior art, the intelligent computing center is equipped with multiple GPU accelerator cards, each of which can only handle one model development task. In the process of model development, it is often necessary not to use all the computing resources of the GPU accelerator card, resulting in low utilization of the computing resources of the intelligent computing center. At the same time, since the computing services provided by the intelligent computing center are limited by the number of GPU accelerator cards, users usually need to rent the entire GPU accelerator card when renting computing services, resulting in high model development costs and difficulty in achieving widespread application of inclusive computing power.

[0008] It can be seen that the existing technology has the problems of low utilization of computing resources and high cost of model development. Summary of the invention

[0009] The present invention provides a method and device for developing an intelligent computing center model for universal computing power, so as to solve the problems in the prior art of low utilization of computing power resources and high cost of model development.

[0010] To solve the above problems, the present invention is achieved as follows: In a first aspect, the present invention provides a method for developing an intelligent computing center model for inclusive computing power, comprising: Step S1, receiving a scheduling instruction, wherein the scheduling instruction includes a model code; Step S2: perform computing power prediction based on the data volume of the model code to obtain required computing power information; Step S3: acquiring a target virtual accelerator card based on the required computing power information, where the target virtual accelerator card is a virtual accelerator card created based on part of the GPU computing power resources of the GPU accelerator card of the first intelligent computing center; Step S4: Create a first container based on the target virtual accelerator card, where the first container is used to execute the model code.

[0011] In one embodiment, step S3 includes: Step S31, scheduling the GPU computing power resources of the target GPU accelerator card based on the required computing power information, wherein the target GPU accelerator card is an accelerator card with idle GPU computing power resources, and the idle GPU computing power resources in the target GPU accelerator card match the required computing power information; Step S32: Create the target virtual accelerator card based on the GPU computing power resources of the target GPU accelerator card.

[0012] In one embodiment, after step S4, the method further includes: Step S5: When the model code is executed in the first container, the first container is closed, the target virtual accelerator card is deregistered, and the GPU computing power resources of the target virtual accelerator card are released.

[0013] In one embodiment, step S3 includes: Step S33: acquiring a plurality of pre-created initial virtual accelerator cards and first computing power resource information corresponding to each initial virtual accelerator card, wherein the GPU computing power resources of the plurality of initial virtual accelerator cards are in an idle state; Step S34: determining a target virtual accelerator card from the multiple initial virtual accelerator cards based on the required computing power information, wherein the target virtual accelerator card is an accelerator card whose first computing power resource information meets the required computing power information.

[0014] In one embodiment, the scheduling instruction includes the priority corresponding to the model code, and the step S3 includes: Step S35: adding the scheduling instruction to the execution queue based on the priority; Step S36: Obtain a target virtual accelerator card based on the required computing power information in the order of the execution queue.

[0015] In one embodiment, after step S4, the method further includes: Step S6: monitor the idle time of the first container, where the idle time is the time when the first container stops occupying GPU computing resources without completing the execution of the model code; Step S7: When the idle time is greater than or equal to a set time threshold, the GPU computing resources occupied by the first container are set to an idle state.

[0016] In one embodiment, after step S7, the method further includes: Step S8: When the first container receives the running instruction, the idle GPU computing resources of the first intelligent computing center are scheduled to process the model code.

[0017] In one embodiment, before step S3, the method further includes: Step S9: Obtain second computing power resource information of the first intelligent computing center; The step S3 comprises: Step S36: When the second computing power resource information meets the required computing power information, obtain a target virtual accelerator card based on the required computing power information.

[0018] In one embodiment, the method further comprises: Step S10: When the second computing power resource information does not meet the required computing power information, sending the scheduling instruction to the second intelligent computing center; Step S11, receiving the processing result sent by the second intelligent computing center, where the processing result is obtained by executing the model code in the second container, and the second container is a container created by the second intelligent computing center scheduling the idle GPU computing resources of the second intelligent computing center.

[0019] In a second aspect, the present invention further provides an intelligent computing center model development device for inclusive computing power, comprising: A first receiving module, configured to receive a scheduling instruction, wherein the scheduling instruction includes a model code; A prediction module, used to perform computing power prediction based on the data volume of the model code to obtain required computing power information; A first acquisition module is used to acquire a target virtual accelerator card based on the required computing power information, where the target virtual accelerator card is a virtual accelerator card created based on part of the GPU computing power resources of the GPU accelerator card of the first intelligent computing center; A creation module is used to create a first container based on the target virtual accelerator card, where the first container is used to execute the model code.

[0020] In a third aspect, the present invention further provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps in the method for developing an intelligent computing center model for inclusive computing power as described in the first aspect above are implemented.

[0021] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method for developing an intelligent computing center model for inclusive computing power as described in the first aspect above are implemented.

[0022] In a fifth aspect, the present invention further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps in the method for developing an intelligent computing center model for inclusive computing power as described in the first aspect above.

[0023] In the present invention, a scheduling instruction is received, the scheduling instruction includes a model code; computing power prediction is performed based on the data volume of the model code to obtain required computing power information; a target virtual accelerator card is obtained based on the required computing power information, the target virtual accelerator card is a virtual accelerator card created based on part of the GPU computing power resources of the GPU accelerator card of the first intelligent computing center; a first container is created based on the target virtual accelerator card, and the first container is used to execute the model code. In this way, the target virtual accelerator card is created through part of the GPU computing power resources of the GPU accelerator card, and the remaining GPU computing power resources can be used to create other virtual accelerator cards to perform other computing power tasks, which greatly improves the utilization rate of the GPU computing power resources of the intelligent computing center. At the same time, when renting computing power services, users do not need to rent the entire GPU accelerator card, but only need to rent part of the computing power resources of the target virtual accelerator card, thereby greatly reducing the economic cost in the model development process and realizing the widespread application of inclusive computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0025] Figure 1 It is a flow chart of a method for developing an intelligent computing center model for inclusive computing power provided by the present invention; Figure 2 It is a structural diagram of an intelligent computing center model development device for universal computing power provided by the present invention; Figure 3 It is a structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The "computing power" mentioned in the present invention refers to: the ability of computer equipment or computing / data center to process information, the ability of computer hardware and software to work together to execute certain computing requirements, and the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity, and it mainly provides services to the society through computing power infrastructure.

[0028] The "computing power" (Computational Power, CP) mentioned in the present invention refers to: it is the ability of a data center server to process data and output results. It is a comprehensive indicator to measure the computing power of a data center, including general computing power, super computing power and intelligent computing power. The commonly used unit of measurement is the number of floating-point operations performed per second (FLOPS, 1EFLOPS=10^18FLOPS). The larger the value, the stronger the comprehensive computing power. According to calculations, 1 EFLOPS is approximately the computing power output of 5 Tianhe-2A or 500,000 mainstream server CPUs or 2 million mainstream notebooks. The calculation formula is: CP=CP 通用 +CP 智能 +CP 超级 .

[0029] The "carrying capacity" (Network Power, NP) mentioned in the present invention refers to: it is the performance of the data transmission capacity of the computing power facilities, including the comprehensive capabilities of network architecture, network bandwidth, transmission delay, intelligent management and scheduling, etc. It involves network transmission within and between data centers, and is a comprehensive indicator for measuring network transmission scheduling capabilities.

[0030] The "Storage Power" (SP) mentioned in the present invention refers to: the comprehensive capabilities of a data center in terms of data storage capacity, performance, safety and reliability, and green and low-carbon. It is a comprehensive indicator for measuring the data storage capacity of a data center, including external storage devices such as storage arrays and server built-in storage devices. The commonly used unit of measurement for storage capacity is exabyte (EB, 1EB=2^60bytes), and the commonly used unit of measurement for performance is the number of reads and writes per second per unit capacity (IOPS / TB, Input / Output Operations Per Second / TB). The disaster recovery ratio is an important manifestation of safety and reliability.

[0031] The "computing power infrastructure" mentioned in the present invention refers to: a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity, which can realize centralized computing, storage, transmission and application of information.

[0032] The "new information infrastructure" mentioned in the present invention refers to: mainly including network infrastructure such as 5G networks, fiber-optic broadband networks, backbone networks, international communication networks, satellite Internet, computing power infrastructure such as data centers, general computing power centers, intelligent computing centers, supercomputing centers, and new technology facilities such as artificial intelligence, blockchain, and quantum computing.

[0033] The “computing power” mentioned in the present invention includes: general computing power, intelligent computing power and super computing power.

[0034] The “general computing power” mentioned in the present invention refers to: the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.

[0035] The "intelligent computing power" mentioned in the present invention refers to: a computing platform based on GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit) and other dedicated chips for various innovative artificial intelligence applications, such as natural language processing, machine vision, etc.

[0036] The "super computing power" mentioned in the present invention refers to: the computing power mainly provided by high-performance computing clusters such as supercomputers. It uses the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for calculations in cutting-edge scientific fields, such as planetary simulation, drug molecule design, gene analysis, etc.

[0037] The "intelligent computing center" mentioned in the present invention refers to a facility that provides the required computing power, data and algorithms for artificial intelligence applications (such as artificial intelligence deep learning model development, model training and model reasoning) by using large-scale heterogeneous computing resources, including general computing power (CPU) and intelligent computing power (GPU, FPGA, ASIC, etc.). The intelligent computing center covers facilities, hardware, and software, and can provide full-stack capabilities from bottom-level computing power to top-level application enablement.

[0038] The “intelligent computing center” mentioned in the present invention includes but is not limited to the “intelligent computing center”.

[0039] The "intelligent computing center" mentioned in the present invention is an artificial intelligence computing center, which is a type of computing power infrastructure that is based on artificial intelligence theory, adopts artificial intelligence computing architecture, and provides computing power services, data services, and algorithm services required for artificial intelligence applications.

[0040] The "computing power center" mentioned in the present invention refers to: a facility that is mainly composed of infrastructure such as wind, fire, water, electricity, and IT hardware and software equipment, and has computing power, transportation capacity, and storage capacity, including general data centers, intelligent computing centers, supercomputing centers, etc.

[0041] The "supercomputing center" mentioned in the present invention refers to: a supercomputing data center, which is a data center based on a supercomputer or a large-scale computing cluster, which can provide functions such as large-scale computing, storage and network services, and is widely used in application scenarios such as aerospace, national defense, oil exploration, climate modeling and genome sequencing.

[0042] The "computing resources" mentioned in the present invention refer to: technologies and facilities with information computing, transmission, storage and application capabilities required for the development of a digital society, including but not limited to computing resources such as CPUs and GPUs, network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, and supporting and guarantee resources such as wind, fire, water and electricity.

[0043] The "inclusive computing power" mentioned in the present invention refers to providing appropriate and effective computing power services at an affordable cost to all social classes and groups that have computing power service needs based on the requirements of equal opportunity and the principle of commercial sustainability.

[0044] The “model” mentioned in the present invention includes but is not limited to a “large language model” and a “multimodal large model”.

[0045] The “large language model” mentioned in the present invention refers to a large language model (LLM), which is a language model with a large parameter scale, designed to understand and generate human language, trained with a large amount of text data, and can perform a wide range of tasks including text summarization, translation, sentiment analysis, etc.

[0046] The "Multimodal Large Models" mentioned in the present invention refer to models that combine multimodal information such as text, images, videos, audio, etc. for training, including but not limited to multimodal large language models.

[0047] See also Figure 1 , Figure 1 is a flow chart of a method for developing an intelligent computing center model for inclusive computing power provided by the present invention. Figure 1 As shown, the following steps are included: Step S1: receiving a scheduling instruction, wherein the scheduling instruction includes a model code.

[0048] The above-mentioned scheduling instructions are scheduling instructions sent by the terminal or CPU container. It should be noted that the terminal or CPU container is deployed with an editing environment for the model code, and the user can adjust and update the model code in the terminal or CPU container. After the model code in the terminal or CPU container is adjusted, the user needs to run the model code to confirm the adjustment effect. At this time, the terminal or CPU container sends a scheduling instruction, and the scheduling instruction includes the model code, so that after receiving the scheduling instruction, the intelligent computing center can run the model code included in the scheduling instruction to obtain the running result.

[0049] The CPU container is a container created based on the CPU resources of the intelligent computing center, and the terminal is a terminal that has established a communication connection with the intelligent computing center.

[0050] Step S2: perform computing power prediction based on the data volume of the model code to obtain required computing power information.

[0051] The above required computing power information is used to characterize the amount of computing power resources required to execute the model code. It should be noted that different model codes require different amounts of computing power resources. In order to increase the number of computing power services provided by the intelligent computing center, it is necessary to allocate appropriate computing power resources to each model code based on the required computing power information to improve the utilization rate of computing power resources, thereby providing computing power services to more users.

[0052] In some embodiments, computing power prediction is performed based on the data volume of the model code. A prediction model can be obtained by training based on the data volume of historical model codes, and then the data volume of different model codes is analyzed through the prediction model to obtain the required computing power information corresponding to the model code.

[0053] In some implementations, computing power prediction is performed based on the data volume of the model code. The number of tokens of the model code can be obtained, and the number of tokens and the data volume of the model code are combined for analysis to obtain the required computing power information corresponding to the model code.

[0054] Step S3: obtaining a target virtual accelerator card based on the required computing power information, where the target virtual accelerator card is a virtual accelerator card created based on part of the GPU computing power resources of the GPU accelerator card of the first intelligent computing center.

[0055] The above-mentioned virtual accelerator card is used to provide part of the GPU computing power resources of the GPU accelerator card. It should be noted that the computing power resources of a GPU accelerator card can be divided into multiple parts, and the divided part of the GPU computing power resources can create virtual accelerator cards, so as to create multiple virtual accelerator cards based on the GPU computing power resources of a GPU accelerator card, and each virtual accelerator card can be used to execute a computing power task of running a model code. In this way, a GPU accelerator card can execute multiple computing power tasks in the form of a virtual accelerator card, so as to provide computing power services to more users.

[0056] The target virtual accelerator card is a virtual accelerator card obtained based on the required computing power information, and the target virtual accelerator card meets the required computing power information. The target virtual accelerator card is obtained through the required computing power information, and the target virtual accelerator card occupies as little GPU computing power resources of the first intelligent computing center as possible while meeting the required computing power information, thereby improving the utilization rate of the GPU computing power resources of the first intelligent computing center.

[0057] In some implementations, obtaining a target virtual accelerator card based on the required computing power information may be to temporarily schedule GPU computing power resources of the first intelligent computing center to create a target virtual accelerator card that meets the required computing power information.

[0058] In some implementations, obtaining a target virtual accelerator card based on the required computing power information may be to pre-create multiple virtual accelerator cards with different computing power resource sizes, and then select a suitable virtual accelerator card from the multiple virtual accelerator cards as the target virtual accelerator card based on the required computing power information.

[0059] Step S4: Create a first container based on the target virtual accelerator card, where the first container is used to execute the model code.

[0060] In the present invention, a scheduling instruction is received, the scheduling instruction includes a model code; computing power prediction is performed based on the data volume of the model code to obtain required computing power information; a target virtual accelerator card is obtained based on the required computing power information, the target virtual accelerator card is a virtual accelerator card created based on part of the GPU computing power resources of the GPU accelerator card of the first intelligent computing center; a first container is created based on the target virtual accelerator card, and the first container is used to execute the model code. In this way, the target virtual accelerator card is created through part of the GPU computing power resources of the GPU accelerator card, and the remaining GPU computing power resources can be used to create other virtual accelerator cards to perform other computing power tasks, which greatly improves the utilization rate of the GPU computing power resources of the intelligent computing center. At the same time, when renting computing power services, users do not need to rent the entire GPU accelerator card, but only need to rent part of the computing power resources of the target virtual accelerator card, thereby greatly reducing the economic cost in the model development process and realizing the widespread application of inclusive computing power.

[0061] In one embodiment, step S3 includes: Step S31, scheduling the GPU computing power resources of the target GPU accelerator card based on the required computing power information, wherein the target GPU accelerator card is an accelerator card with idle GPU computing power resources, and the idle GPU computing power resources in the target GPU accelerator card match the required computing power information; Step S32: Create the target virtual accelerator card based on the GPU computing power resources of the target GPU accelerator card.

[0062] In the present invention, the GPU computing power resources of the target GPU accelerator card are scheduled based on the required computing power information, the target GPU accelerator card is an accelerator card with idle GPU computing power resources, and the idle GPU computing power resources in the target GPU accelerator card match the required computing power information; the target virtual accelerator card is created based on the GPU computing power resources of the target GPU accelerator card. In this way, when it is necessary to execute the model code, the idle GPU computing power resources in the GPU accelerator card are temporarily scheduled to create the target virtual accelerator card, so that the GPU computing power resources of the target virtual accelerator card can just meet the demand for executing the model code, realizing flexible scheduling of GPU computing power resources, reducing the waste of GPU computing power resources of the first intelligent computing center, and greatly improving the utilization rate of GPU computing power resources of the first intelligent computing center.

[0063] Specifically, when scheduling the GPU computing power resources of the first intelligent computing center, first obtain the GPU accelerator cards with idle GPU computing power resources in the first intelligent computing center, screen the target GPU accelerator cards, and then schedule the idle GPU computing power resources of the GPU accelerator cards to create the target virtual accelerator card.

[0064] The step of obtaining a GPU accelerator card having idle GPU computing resources in the first intelligent computing center includes: obtaining multiple GPU accelerator cards having idle GPU computing resources, and remaining computing resource information of each GPU accelerator card; The screening of the target GPU accelerator card includes: screening the target GPU accelerator card from the multiple GPU accelerator cards based on the remaining computing power resource information, the target GPU accelerator card being an accelerator card whose remaining computing power resources meet the required computing power information.

[0065] The target GPU accelerator card is screened in the above manner so that the remaining computing power resources of the target GPU accelerator card meet the required computing power information, and then the target virtual accelerator card can be created by scheduling the idle GPU computing power resources of the GPU accelerator card.

[0066] In one embodiment, after step S4, the method further includes: Step S5: When the model code is executed in the first container, the first container is closed, the target virtual accelerator card is deregistered, and the GPU computing power resources of the target virtual accelerator card are released.

[0067] It should be noted that the target virtual accelerator card is a temporarily created virtual accelerator card. If the first container is created based on the target virtual accelerator card to perform other computing tasks, insufficient or excessive GPU computing resources will occur, and it is impossible to meet the customer's model development needs while reducing the waste of computing resources. Therefore, in the present invention, when the execution of the model code is completed in the first container, the first container is closed, the target virtual accelerator card is cancelled, and the GPU computing resources of the target virtual accelerator card are released, so that when other model codes need to be executed, the idle GPU computing resources can be obtained, which greatly improves the utilization rate of the GPU computing resources of the first intelligent computing center.

[0068] In one embodiment, step S3 includes: Step S33: acquiring a plurality of pre-created initial virtual accelerator cards and first computing power resource information corresponding to each initial virtual accelerator card, wherein the GPU computing power resources of the plurality of initial virtual accelerator cards are in an idle state; Step S34: determining a target virtual accelerator card from the multiple initial virtual accelerator cards based on the required computing power information, wherein the target virtual accelerator card is an accelerator card whose first computing power resource information meets the required computing power information.

[0069] The first computing power resource information is used to characterize the computing power resource size of the initial pre-acceleration card. The multiple initial virtual accelerator cards are pre-created initial virtual accelerator cards. By pre-creating multiple initial virtual accelerator cards, there is no need to create virtual accelerator cards when the model code needs to be executed, which can greatly improve the efficiency of executing the model code.

[0070] In some embodiments, the multiple initial virtual accelerator cards are initial virtual accelerator cards with different computing power resource sizes, and the target virtual accelerator card is the virtual accelerator card with the smallest computing power resource size that meets the required computing power information. In this way, the virtual accelerator card that meets the required computing power information and has the smallest computing power resource size is selected from the multiple initial virtual accelerator cards as the target virtual accelerator card, so that the first container can occupy as little GPU computing power resources of the first intelligent computing center as possible while implementing the execution model code, thereby greatly improving the utilization rate of the GPU computing power resources of the first intelligent computing center.

[0071] Furthermore, since the multiple initial virtual accelerator cards have different computing power resource sizes, after the model code is executed in the first container, there is no need to cancel the target virtual accelerator card. It is only necessary to close the first container and set the GPU computing power resources of the target virtual accelerator card to an idle state, so that when other model codes need to be executed again, the target virtual accelerator card can be obtained for execution, thereby improving execution efficiency.

[0072] In one embodiment, the scheduling instruction includes the priority corresponding to the model code, and the step S3 includes: Step S35: adding the scheduling instruction to the execution queue based on the priority; Step S36: Obtain a target virtual accelerator card based on the required computing power information in the order of the execution queue.

[0073] In the present invention, by setting priorities to execute different model codes, computing resources are preferentially scheduled for execution for tasks with higher priorities, thereby effectively improving the user experience without changing the overall computing resources of the first intelligent computing center.

[0074] In some implementations, adding the scheduling instruction to the execution queue based on the priority includes: Obtaining the sending time of the model code; Calculating a score of the model code based on the sending time and the priority; The scheduled instruction is added to an execution queue based on the score.

[0075] It should be noted that different model codes have different priorities, but the sending time is also different. If the first intelligent computing center receives a high-priority model code after receiving a low-priority model code, the continuous scheduling of computing resources to execute the high-priority model code will cause the low-priority model code to execute too long, and the user experience is poor. Therefore, in this embodiment, the sending time and priority of the model code are combined to obtain the model code score, so that the model code with low priority but early sending time can also be executed in time, further improving the user experience.

[0076] The step of calculating the score of the model code based on the sending time and the priority may specifically be to weight the sending time and the priority to obtain the score of the model code.

[0077] In one embodiment, after step S4, the method further includes: Step S6: monitor the idle time of the first container, where the idle time is the time when the first container stops occupying GPU computing resources without completing the execution of the model code; Step S7: When the idle time is greater than or equal to a set time threshold, the GPU computing resources occupied by the first container are set to an idle state.

[0078] It should be noted that during the execution of the model code in the first container, the user needs to stop the execution of the model code for adjustment. At this time, the model code in the first container has not been completed, but the GPU computing resources of the first container are not used. If the execution of the model code is stopped for a long time, it will lead to a waste of GPU computing resources.

[0079] Therefore, in the present invention, the idle time of the first container is monitored, and the idle time is the time when the first container stops occupying GPU computing resources without completing the execution of the model code; when the idle time is greater than or equal to the set time threshold, the GPU computing resources occupied by the first container are set to an idle state. In this way, the GPU computing resources occupied by the first container are set to an idle state, so that other containers can occupy the GPU computing resources to execute the model code, realizing preemptive scheduling of this part of the GPU computing resources, and greatly improving the utilization rate of the GPU computing resources of the first intelligent computing center.

[0080] In one embodiment, after step S7, the method further includes: Step S8: When the first container receives the running instruction, the idle GPU computing resources of the first intelligent computing center are scheduled to process the model code.

[0081] It should be noted that when the idle time of the first container is greater than or equal to the set time threshold, the GPU computing resources occupied by the first container are set to idle state, and the first container does not actually occupy GPU computing resources. When receiving the run instruction, the idle GPU computing resources of the first intelligent computing center are rescheduled to process the model code, thereby ensuring the normal operation of the first container.

[0082] In one embodiment, before step S3, the method further includes: Step S9: Obtain second computing power resource information of the first intelligent computing center; The step S3 comprises: Step S36: When the second computing power resource information meets the required computing power information, obtain a target virtual accelerator card based on the required computing power information.

[0083] In one embodiment, the method further comprises: Step S10: When the second computing power resource information does not meet the required computing power information, sending the scheduling instruction to the second intelligent computing center; Step S11, receiving the processing result sent by the second intelligent computing center, where the processing result is obtained by executing the model code in the second container, and the second container is a container created by the second intelligent computing center scheduling the idle GPU computing resources of the second intelligent computing center.

[0084] It should be noted that the total GPU computing power resources of the first intelligent computing center are limited. In the process of actually providing computing power services, some intelligent computing centers may be overloaded while other intelligent computing centers may be underloaded. In order to achieve a balance between different intelligent computing centers and to enable a single intelligent computing center to provide more computing power services, the GPU computing power resources between different intelligent computing centers can be scheduled remotely in the present invention.

[0085] Specifically, when the first intelligent computing center receives the scheduling instruction, it first determines the usage of the computing power resources of the local area (i.e., the first intelligent computing center), that is, obtains the second computing power resource information of the first intelligent computing center. When the second computing power resource information meets the required computing power information, the model code can be processed directly without scheduling the computing power resources of other intelligent computing centers; when the second computing power resource information does not meet the required computing power information, the model code cannot be processed locally. At this time, the computing power resources of the second intelligent computing center are scheduled, and the computing power resources of other intelligent computing centers need to be scheduled to execute the model code. In this way, the utilization rate of computing power resources of different intelligent computing centers is improved, and the number of computing power services that can be provided by intelligent computing centers is increased.

[0086] Among them, the above-mentioned second computing power resource information is used to characterize the computing power usage of intelligent computing.

[0087] See also Figure 2 , Figure 2 is a structural diagram of an intelligent computing center model development device for universal computing power provided by the present invention, such as Figure 2 As shown, the intelligent computing center model development device 200 for inclusive computing power includes: A first receiving module 201 is used to receive a scheduling instruction, wherein the scheduling instruction includes a model code; A prediction module 202 is used to perform computing power prediction based on the data volume of the model code to obtain required computing power information; A first acquisition module 203 is used to acquire a target virtual accelerator card based on the required computing power information, where the target virtual accelerator card is a virtual accelerator card created based on part of the GPU computing power resources of the GPU accelerator card of the first intelligent computing center; The creation module 204 is used to create a first container based on the target virtual accelerator card, where the first container is used to execute the model code.

[0088] In one embodiment, the first acquisition module 203 includes: a scheduling unit, configured to schedule GPU computing power resources of a target GPU accelerator card based on the required computing power information, wherein the target GPU accelerator card is an accelerator card having GPU computing power resources in an idle state, and the GPU computing power resources in the idle state in the target GPU accelerator card match the required computing power information; A creation unit is used to create the target virtual accelerator card based on the GPU computing power resources of the target GPU accelerator card.

[0089] In one embodiment, after the creation module 204, the intelligent computing center model development device 200 for inclusive computing power further includes: The closing module is used to close the first container, cancel the target virtual accelerator card, and release the GPU computing power resources of the target virtual accelerator card when the model code is executed in the first container.

[0090] In one embodiment, the first acquisition module 203 includes: A first acquisition unit is used to acquire a plurality of pre-created initial virtual acceleration cards and first computing power resource information corresponding to each initial virtual acceleration card, wherein the GPU computing power resources of the plurality of initial virtual acceleration cards are in an idle state; A determination module is used to determine a target virtual accelerator card from the multiple initial virtual accelerator cards based on the required computing power information, where the target virtual accelerator card is an accelerator card whose first computing power resource information meets the required computing power information.

[0091] In one embodiment, the scheduling instruction includes a priority corresponding to the model code, and the first acquisition module 203 includes: an adding unit, configured to add the scheduling instruction to an execution queue based on the priority; The second acquisition unit is used to acquire a target virtual accelerator card based on the required computing power information in the order of the execution queue.

[0092] In one embodiment, after the creation module 204, the intelligent computing center model development device 200 for inclusive computing power further includes: A monitoring module, used to monitor the idle time of the first container, where the idle time is the time when the first container stops occupying GPU computing resources without completing the execution of the model code; The setting module is used to set the GPU computing power resources occupied by the first container to an idle state when the idle time is greater than or equal to a set time threshold.

[0093] In one embodiment, after the setting module, the intelligent computing center model development device 200 for inclusive computing power further includes: The scheduling module is used to schedule the idle GPU computing resources of the first intelligent computing center to process the model code when the first container receives the running instruction.

[0094] In one embodiment, before the first acquisition module 203, the intelligent computing center model development device 200 for inclusive computing power further includes: A second acquisition module is used to obtain second computing power resource information of the first intelligent computing center; The first acquisition module 203 includes: The third acquisition unit is used to acquire a target virtual accelerator card based on the required computing power information when the second computing power resource information meets the required computing power information.

[0095] In one embodiment, the intelligent computing center model development device 200 for inclusive computing power further includes: A sending module, configured to send the scheduling instruction to a second intelligent computing center when the second computing power resource information does not meet the required computing power information; The second receiving module is used to receive the processing result sent by the second intelligent computing center, where the processing result is obtained by executing the model code in the second container, and the second container is a container created by the second intelligent computing center to schedule the GPU computing power resources in the idle state of the second intelligent computing center.

[0096] The intelligent computing center model development device for inclusive computing power provided by the present invention can realize the various processes of the various embodiments of the above-mentioned intelligent computing center model development method for inclusive computing power. The technical features correspond one to one and can achieve the same technical effects. To avoid repetition, they will not be described here.

[0097] It should be noted that the intelligent computing center model development device for inclusive computing power in the present invention can be a device, or a component, integrated circuit, or chip in an electronic device.

[0098] The present invention also provides an electronic device, see Figure 3 , Figure 3 is a schematic diagram of the structure of an electronic device provided by the present invention, the electronic device includes a memory 301, a processor 302 and a program or instruction stored in the memory 301 and running thereon, and when the program or instruction is executed by the processor 302, it can be realized Figure 1 Any steps in the corresponding embodiment of the intelligent computing center model development method for inclusive computing power and the same beneficial effects achieved will not be repeated here.

[0099] The processor 302 may be a CPU, an ASIC, an FPGA or a GPU.

[0100] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above-mentioned embodiment of the intelligent computing center model development method for inclusive computing power can be completed through hardware related to program instructions, and the program can be stored in a readable medium.

[0101] The present invention also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above Figure 1 Any steps in the corresponding embodiments of the method for developing a model of an intelligent computing center for inclusive computing power can achieve the same technical effect, and will not be repeated here to avoid repetition. The storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0102] The present invention also provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the above Figure 1The corresponding processes of the implementation method of the intelligent computing center model development method for inclusive computing power can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0103] The terms "first", "second" etc. in the present invention are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of the connected objects, such as A and / or B and / or C, which means to include 7 situations including single A, single B, single C, and A and B all exist, B and C all exist, A and C all exist, and A, B and C all exist.

[0104] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a second terminal device, etc.) to execute the methods of each embodiment of the present application.

[0106] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for developing an intelligent computing center model for inclusive computing power, characterized in that: include: Step S1, receiving a scheduling instruction, wherein the scheduling instruction includes a model code; Step S2: perform computing power prediction based on the data volume of the model code to obtain required computing power information; Step S3: acquiring a target virtual accelerator card based on the required computing power information, where the target virtual accelerator card is a virtual accelerator card created based on part of the GPU computing power resources of the GPU accelerator card of the first intelligent computing center; Step S4: Create a first container based on the target virtual accelerator card, where the first container is used to execute the model code.

2. The method according to claim 1, characterized in that The step S3 comprises: Step S31, scheduling the GPU computing power resources of the target GPU accelerator card based on the required computing power information, wherein the target GPU accelerator card is an accelerator card with idle GPU computing power resources, and the idle GPU computing power resources in the target GPU accelerator card match the required computing power information; Step S32: Create the target virtual accelerator card based on the GPU computing power resources of the target GPU accelerator card.

3. The method according to claim 2, characterized in that After step S4, the method further includes: Step S5: When the model code is executed in the first container, the first container is closed, the target virtual accelerator card is deregistered, and the GPU computing power resources of the target virtual accelerator card are released.

4. The method according to claim 1, characterized in that The step S3 comprises: Step S33: acquiring a plurality of pre-created initial virtual accelerator cards and first computing power resource information corresponding to each initial virtual accelerator card, wherein the GPU computing power resources of the plurality of initial virtual accelerator cards are in an idle state; Step S34: determining a target virtual accelerator card from the multiple initial virtual accelerator cards based on the required computing power information, wherein the target virtual accelerator card is an accelerator card whose first computing power resource information meets the required computing power information.

5. The method according to claim 1, characterized in that The scheduling instruction includes the priority corresponding to the model code, and the step S3 includes: Step S35: adding the scheduling instruction to the execution queue based on the priority; Step S36: Obtain a target virtual accelerator card based on the required computing power information in the order of the execution queue.

6. The method according to any one of claims 1 to 5, characterized in that After step S4, the method further includes: Step S6: monitor the idle time of the first container, where the idle time is the time when the first container stops occupying GPU computing resources without completing the execution of the model code; Step S7: When the idle time is greater than or equal to a set time threshold, the GPU computing resources occupied by the first container are set to an idle state.

7. The method according to claim 6, characterized in that After step S7, the method further includes: Step S8: When the first container receives the running instruction, the idle GPU computing resources of the first intelligent computing center are scheduled to process the model code.

8. The method according to any one of claims 1 to 5, characterized in that Before step S3, the method further includes: Step S9: Obtain second computing power resource information of the first intelligent computing center; The step S3 comprises: Step S36: When the second computing power resource information meets the required computing power information, obtain a target virtual accelerator card based on the required computing power information.

9. The method according to claim 8, characterized in that The method further comprises: Step S10: When the second computing power resource information does not meet the required computing power information, sending the scheduling instruction to the second intelligent computing center; Step S11, receiving the processing result sent by the second intelligent computing center, where the processing result is obtained by executing the model code in the second container, and the second container is a container created by the second intelligent computing center scheduling the idle GPU computing resources of the second intelligent computing center.

10. An intelligent computing center model development device for universal computing power, characterized in that: include: A first receiving module, configured to receive a scheduling instruction, wherein the scheduling instruction includes a model code; A prediction module, used to perform computing power prediction based on the data volume of the model code to obtain required computing power information; A first acquisition module is used to acquire a target virtual accelerator card based on the required computing power information, where the target virtual accelerator card is a virtual accelerator card created based on part of the GPU computing power resources of the GPU accelerator card of the first intelligent computing center; A creation module is used to create a first container based on the target virtual accelerator card, where the first container is used to execute the model code.

11. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method for developing an intelligent computing center model for inclusive computing power as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for developing an intelligent computing center model for inclusive computing power as described in any one of claims 1 to 9.

13. A computer program product, characterized in that It includes computer instructions, which, when executed by a processor, implement the steps of the intelligent computing center model development method for universal computing power as described in any one of claims 1 to 9.

Images