Computing power scheduling method and device of intelligent computing center

Abstract

The invention provides a computing power scheduling method and device of an intelligent computing center, and relates to the technical field of computing power infrastructure, and the method comprises the steps: receiving a first reasoning request of a first reasoning task; searching a first candidate container in an idle container pool of the intelligent computing center based on the identifier of the first model and the identifier of the first LoRA; and under the condition that the first candidate container is found, routing the first reasoning request to the first candidate container, and scheduling computing power resources of the first candidate container to execute the first reasoning task. Therefore, the intelligent computing center can accurately match the loaded container in the idle container pool with the identifier of the model and the identifier of the LoRA corresponding to the reasoning task, if the matched container exists in the container pool, the model and the LoRA do not need to be reloaded, computing power resources of the existing container can be directly reused to execute the reasoning task, and the efficiency of the reasoning task is improved. Therefore, the waste of computing power resources is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent computing centers, smart computing centers and computing power infrastructure, and specifically to a computing power scheduling method and device for an intelligent computing center. Background Art

[0002] With the rapid development of artificial intelligence technology, "intelligent computing centers" and "intelligent computing centers" have emerged.

[0003] An "Intelligent Computing Center" is a facility that uses large-scale heterogeneous computing resources, including general-purpose and intelligent computing power, to provide the computing power, data, and algorithms required for AI applications (such as AI deep learning model development, model training, and model inference). The Intelligent Computing Center encompasses facilities, hardware, and software, and provides a full stack of capabilities, from bottom-level computing power to top-level application enablement.

[0004] “Intelligent Computing Center” includes but is not limited to “Smart Computing Center”.

[0005] "Intelligent Computing Center" refers to an artificial intelligence computing center. It is a type of computing power infrastructure that is based on artificial intelligence theory, adopts artificial intelligence computing architecture, and provides computing power services, data services, and algorithm services required for artificial intelligence applications.

[0006] "Computing power" is the core of "intelligent computing center" and "intelligent computing center". It is the ability of computer equipment or computing / data center to process information. It is the ability of computer hardware and software to work together to perform certain computing needs. It is the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity. It mainly provides services to society through computing power infrastructure.

[0007] As the core hub for processing massive amounts of data and performing complex computing tasks, the Intelligent Computing Center (ICC) is tasked with providing powerful computing power to numerous enterprises, research institutions, and various intelligent applications. In the ICC, ComfyUI is often used for text-to-image tasks, generating images based on input text descriptions. As a backend service, it handles the computations and logic associated with image generation. ComfyUI has an internal task queue mechanism, meaning that only one inference task can be in progress at any given moment. Each time a different inference task is executed, the corresponding inference model must be reloaded from scratch, which consumes time and wastes computing power. Therefore, since the emergence of the ICC, reducing the waste of computing power when executing inference tasks has become a pressing technical issue. Summary of the Invention

[0008] The present invention provides a computing power scheduling method and device for an intelligent computing center to solve the problem of how to achieve network virtualization of computing power resources.

[0009] To solve the above problems, the present invention is achieved as follows:

[0010] In a first aspect, the present invention provides a computing power scheduling method for an intelligent computing center, comprising:

[0011] Step S1: receiving a first reasoning request for a first reasoning task, where the first reasoning request includes an identifier of a first model corresponding to the first reasoning task and an identifier of a first low-rank adaptive LoRA;

[0012] Step S2: searching for a first candidate container in an idle container pool of an intelligent computing center based on the identifier of the first model and the identifier of the first LoRA;

[0013] Step S3: When the first candidate container is found, the first inference request is routed to the first candidate container, and computing resources of the first candidate container are scheduled to execute the first inference task.

[0014] In one embodiment, the first candidate container is a container whose identifier of a loaded model matches the identifier of the first model, and whose identifier of a loaded LoRA matches the identifier of the first LoRA.

[0015] In one embodiment, after step S3, the method further includes:

[0016] Step S4: If the first candidate container is not found, search the idle container pool for a second candidate container, where the second candidate container is a container whose identifier of the loaded model matches the identifier of the first model and for which LoRA is not loaded.

[0017] Step S5: When the second candidate container is found, the inference request of the first inference task is routed to the second candidate container, and the computing resources of the second candidate container are scheduled to execute the first inference task.

[0018] In one embodiment, after step S5, the method further includes:

[0019] Step S6: If the second candidate container is not found, construct a target container in the intelligent computing center;

[0020] Step S7: After loading the first model and the first LoRA in the target container, route the inference request of the first inference task to the target container, and schedule the computing resources of the target container to execute the first inference task.

[0021] In one embodiment, the first reasoning request further includes a first workflow identifier of the first reasoning task;

[0022] The method further comprises:

[0023] Step S8: receiving a second reasoning request for a second reasoning task, where the second reasoning request includes a second workflow identifier for the second reasoning task;

[0024] Step S9: When the first workflow identifier and the second workflow identifier match, the second inference request is routed to the candidate container for executing the first inference task, and the computing resources of the candidate container for executing the first inference task are scheduled to execute the second inference task.

[0025] In one embodiment, the method further comprises:

[0026] Step S10: using a serverless mechanism to regularly detect the idle time of the containers in the idle container pool;

[0027] Step S11: clearing the containers whose idle time exceeds a preset time.

[0028] In a second aspect, the present invention further provides a computing power scheduling device for an intelligent computing center, comprising:

[0029] A first receiving module is configured to receive a first reasoning request for a first reasoning task, where the first reasoning request includes an identifier of a first model corresponding to the first reasoning task and an identifier of a first low-rank adaptive LoRA;

[0030] A first search module is configured to search for a first candidate container in an idle container pool of an intelligent computing center based on an identifier of the first model and an identifier of the first LoRA;

[0031] A first scheduling module is configured to, when the first candidate container is found, route the first inference request to the first candidate container and schedule computing resources of the first candidate container to execute the first inference task.

[0032] In a third aspect, the present invention also provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps in the computing power scheduling method of the intelligent computing center as described in the first aspect above are implemented.

[0033] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the computing power scheduling method of the intelligent computing center as described in the first aspect above are implemented.

[0034] In a fifth aspect, the present invention further provides a computer program product comprising computer instructions, which, when executed by a processor, implement the steps in the computing power scheduling method of the intelligent computing center as described in the first aspect above.

[0035] In the present invention, a first inference request for a first inference task is received, the first inference request including an identifier of a first model corresponding to the first inference task and an identifier of a first low-rank adaptive LoRA; based on the identifier of the first model and the identifier of the first LoRA, a first candidate container is searched for in the idle container pool of the intelligent computing center; when the first candidate container is found, the first inference request is routed to the first candidate container, and the computing power resources of the first candidate container are scheduled to execute the first inference task. In this way, the intelligent computing center can accurately match the loaded container in the idle container pool based on the identifier of the model corresponding to the inference task and the identifier of the LoRA. If a matching container already exists in the container pool, there is no need to reload the model and LoRA, and the computing power resources of the existing container can be directly reused to execute the inference task, avoiding the repeated use of computing power resources to load the model, thereby reducing the waste of computing power resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for the description of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 This is a flow chart of a computing power scheduling method for an intelligent computing center provided by the present invention;

[0038] Figure 2 It is a schematic diagram of the execution of the reasoning task provided by the relevant technology;

[0039] Figure 3 is a schematic diagram of executing the reasoning task provided by the present invention;

[0040] Figure 4 It is a schematic diagram of a computing power scheduling method for an intelligent computing center provided by the present invention;

[0041] Figure 5 This is a structural diagram of a computing power scheduling device for an intelligent computing center provided by the present invention;

[0042] Figure 6 This is a structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0044] The "computing power" mentioned in the present invention refers to: the ability of computer equipment or computing / data centers to process information, the ability of computer hardware and software to work together to execute certain computing requirements, and the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity, and mainly provides services to society through computing power infrastructure.

[0045] The "computing power" (Computational Power, CP) mentioned in the present invention refers to: the ability of a data center server to process data and output results. It is a comprehensive indicator to measure the computing power of a data center, including general computing power, super computing power and intelligent computing power. The commonly used unit of measurement is the number of floating-point operations performed per second (FLOPS, 1EFLOPS=10^18FLOPS). The larger the value, the stronger the comprehensive computing power. According to calculations, 1EFLOPS is approximately the computing power output of 5 Tianhe-2A or 500,000 mainstream server CPUs or 2 million mainstream notebooks. The calculation formula is: CP=CP 通用 +CP 智能 +CP 超级 .

[0046] The "carrying capacity" (Network Power, NP) mentioned in the present invention refers to: it is the performance of the data transmission capability of the computing power facility, which includes comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, etc. It involves network transmission within and between data centers, and is a comprehensive indicator for measuring network transmission scheduling capabilities.

[0047] The "Storage Power" (SP) described in this invention refers to the comprehensive capabilities of a data center in terms of data storage capacity, performance, security and reliability, and environmental friendliness. It is a comprehensive indicator for measuring a data center's data storage capacity, encompassing both external storage devices such as storage arrays and internal server storage. Storage capacity is commonly measured in exabytes (EB, 1EB = 2^60 bytes), while performance is commonly measured in IOPS / TB (Input / Output Operations Per Second / TB). Disaster recovery ratio is a key indicator of security and reliability.

[0048] The "computing power infrastructure" mentioned in the present invention refers to a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity, and can realize the centralized calculation, storage, transmission and application of information.

[0049] The "new information infrastructure" mentioned in the present invention refers to: mainly including network infrastructure such as 5G networks, fiber-optic broadband networks, backbone networks, international communication networks, satellite Internet, computing power infrastructure such as data centers, general computing power centers, intelligent computing centers, supercomputing centers, and new technology facilities such as artificial intelligence, blockchain, and quantum computing.

[0050] The "computing power" mentioned in the present invention includes: general computing power, intelligent computing power and super computing power.

[0051] The "general computing power" mentioned in the present invention refers to the computing power provided by servers based on central processing unit (CPU) chips, which is used to support basic general computing such as cloud computing and edge computing.

[0052] The "intelligent computing power" mentioned in the present invention refers to: a computing platform based on large-scale deployment of special chips such as graphics processing units (GPUs), field programmable gate arrays (FPGAs), and application-specific integrated circuits (ASICs) for various innovative artificial intelligence applications, such as natural language processing and machine vision.

[0053] The "supercomputing power" mentioned in the present invention refers to the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for calculations in cutting-edge scientific fields, such as planetary simulation, drug molecule design, genetic analysis, etc.

[0054] The "intelligent computing center" described in this article refers to a facility that provides the computing power, data, and algorithms required for artificial intelligence applications (such as AI deep learning model development, model training, and model inference) by utilizing large-scale heterogeneous computing resources, including general-purpose computing power (CPU) and intelligent computing power (GPU, FPGA, ASIC, etc.). The intelligent computing center encompasses facilities, hardware, and software, and can provide a full stack of capabilities, from bottom-level computing power to top-level application enablement.

[0055] The "intelligent computing center" mentioned in the present invention includes but is not limited to the "intelligent computing center".

[0056] The "intelligent computing center" mentioned in the present invention is an artificial intelligence computing center, which is a type of computing power infrastructure based on artificial intelligence theory, adopts artificial intelligence computing architecture, and provides computing power services, data services and algorithm services required for artificial intelligence applications.

[0057] The "computing power center" mentioned in the present invention refers to: a facility that is mainly composed of infrastructure such as wind, fire, water, electricity, and IT hardware and software equipment, and has computing power, transportation capacity, and storage capacity, including general data centers, intelligent computing centers, supercomputing centers, etc.

[0058] The "supercomputing center" mentioned in the present invention refers to: a supercomputing data center, which is a data center based on a supercomputer or a large-scale computing cluster, which can provide large-scale computing, storage and network services and other functions, and is widely used in application scenarios such as aerospace, national defense, oil exploration, climate modeling and genome sequencing.

[0059] The "computing resources" mentioned in the present invention refer to: technologies and facilities with information computing, transmission, storage and application capabilities required for the development of a digital society, including but not limited to computing resources such as CPUs and GPUs, network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, and supporting and guarantee resources such as wind, fire, water and electricity.

[0060] The "serverless mechanism" described in the present invention refers to severless, that is, tasks are completed by running in stateless computing containers triggered by events. These containers may only exist in one call, or the number of instances may be automatically adjusted according to the load.

[0061] The "LoRA" described in this invention refers to an efficient model parameter fine-tuning technology. Its core idea is not to directly update all the weights of the pre-trained model, but to indirectly achieve model adaptation by adding a small amount of trainable low-rank matrices.

[0062] "ComfyUI" as described in the present invention refers to a node-based stable diffusion graphical user interface that allows users to create and execute complex image generation workflows in a highly customizable and flexible manner.

[0063] In the prior art, ComfyUI is often used in intelligent computing centers for text-based image tasks, where images are generated based on input text descriptions. It serves as a backend service, handling the computation and logic associated with image generation. ComfyUI has an internal task queue mechanism, meaning that only one inference task can be active at any given moment. Each execution of an inference task requires reloading the corresponding inference model from scratch, which consumes time and wastes computing resources. Therefore, since the emergence of intelligent computing centers, reducing the waste of computing resources when executing inference tasks has become a pressing technical issue. To reduce the waste of computing resources when executing inference tasks, the present invention receives a first inference request for a first inference task, the first inference request including an identifier of a first model corresponding to the first inference task and an identifier of a first low-rank adaptive LoRA. Based on the identifier of the first model and the identifier of the first LoRA, a first candidate container is searched for in the idle container pool of the intelligent computing center. If the first candidate container is found, the first inference request is routed to the first candidate container, and the computing resources of the first candidate container are scheduled to execute the first inference task. In this way, the intelligent computing center can accurately match the loaded containers in the idle container pool based on the identifier of the model corresponding to the inference task and the identifier of LoRA. If there is already a matching container in the container pool, there is no need to reload the model and LoRA. The computing power resources of the existing container can be directly reused to perform the inference task, avoiding the reuse of computing power resources to load the model, thereby reducing the waste of computing power resources.

[0064] For details, see Figure 1 , Figure 1 This is a flow chart of a computing power scheduling method for an intelligent computing center provided by the present invention. Figure 1 As shown, the following steps are included:

[0065] Step S1: receiving a first reasoning request for a first reasoning task, where the first reasoning request includes an identifier of a first model corresponding to the first reasoning task and an identifier of a first low-rank adaptive LoRA;

[0066] In this step, the first inference task refers to a specific computing task initiated by the user or upper-level system and needs to be executed by the intelligent computing center. Its core goal is to generate a corresponding image based on the input text description, such as the text-to-image task. This task requires the use of a specific deep learning model and low-rank adaptation (LoRA) to complete the inference calculation. LoRA is an efficient model parameter fine-tuning technology. Its core idea is not to directly update all the weights of the pre-trained model, but to indirectly achieve model adaptation by adding a small number of trainable low-rank matrices, thereby significantly reducing the number of parameters and computation.

[0067] The first inference request is a task request sent by a user or upper-level system to the intelligent computing center to trigger the execution of the first inference task. The first inference request includes the identifier of the first model corresponding to the first inference task and the identifier of the first LoRA. The identifier of the first model is used to determine the deep learning model that needs to be called to execute the first inference task, and the identifier of the first LoRA is used to determine the LoRA that matches the deep learning model.

[0068] Step S2: searching for a first candidate container in an idle container pool of an intelligent computing center based on the identifier of the first model and the identifier of the first LoRA;

[0069] In this step, the "idle container pool" is a resource pool used to manage reusable computing resources in the intelligent computing center. The containers in this pool are idle but pre-loaded with specific models and LoRAs, waiting to be scheduled to execute inference tasks. Based on the identifier of the first model and the identifier of the first LoRA required for the first inference task obtained in step S1, the intelligent computing center can search the idle resource pool for a container loaded with the same model and configured with the same LoRA as the first candidate container.

[0070] Step S3: When the first candidate container is found, the first inference request is routed to the first candidate container, and computing resources of the first candidate container are scheduled to execute the first inference task.

[0071] In this step, when the intelligent computing center finds a first candidate container that meets the requirements, it sends the first reasoning request to the first candidate container, and then uses the computing power resources allocated to the first candidate container to execute the first reasoning task.

[0072] In the above embodiment, a first reasoning request for a first reasoning task is received, and the first reasoning request includes an identifier of a first model corresponding to the first reasoning task and an identifier of a first low-rank adaptive LoRA; based on the identifier of the first model and the identifier of the first LoRA, a first candidate container is searched in the idle container pool of the intelligent computing center; when the first candidate container is found, the first reasoning request is routed to the first candidate container, and the computing power resources of the first candidate container are scheduled to execute the first reasoning task. In this way, the intelligent computing center can accurately match the loaded container in the idle container pool based on the identifier of the model corresponding to the reasoning task and the identifier of the LoRA. If there is already a matching container in the container pool, there is no need to reload the model and LoRA, and the computing power resources of the existing container can be directly reused to execute the reasoning task, thereby reducing the waste of computing power resources.

[0073] In one embodiment, the first candidate container is a container whose identifier of a loaded model matches the identifier of the first model, and whose identifier of a loaded LoRA matches the identifier of the first LoRA.

[0074] In the above embodiment, the "first candidate container" refers to a container in the idle container pool of the intelligent computing center that has been loaded with a model and LoRA that fully matches the current task requirements. For example, the first model is identified as FLUX-D and the first LoRA is identified as LoRA1. A container in the idle container pool that has been loaded with a model identified as FLUX-D and a LoRA identified as LoRA1 can be determined as the first candidate container. If a container has been loaded with a model identified as FLUX-S and a LoRA identified as LoRA2, that container cannot be selected as the first candidate container.

[0075] In this embodiment, the first candidate container is determined by a strategy that fully matches the model identifier and the LoRA identifier. This allows the system to prioritize the reuse of existing containers, reducing computing resource waste. Furthermore, fully matching containers can execute tasks directly without additional loading or configuration, shortening task response time.

[0076] In one embodiment, after step S3, the method further includes:

[0077] Step S4: If the first candidate container is not found, search the idle container pool for a second candidate container, where the second candidate container is a container whose identifier of the loaded model matches the identifier of the first model and for which LoRA is not loaded.

[0078] Step S5: When the second candidate container is found, the inference request of the first inference task is routed to the second candidate container, and the computing resources of the second candidate container are scheduled to execute the first inference task.

[0079] In the above embodiment, if the intelligent computing center fails to find a first candidate container in the idle container pool that fully matches the task requirements (i.e., a container that matches both the model and LoRA), it will further search for a second candidate container. This second candidate container is a container whose loaded model identifier matches the identifier of the first model and whose LoRA is not loaded. Since the model is pre-loaded, the intelligent computing center only needs to dynamically load the LoRA required for the task into the container.

[0080] It should be noted that a container whose loaded model's identifier matches the identifier of the first model but whose loaded LoRA identifier does not match the identifier of the first LoRA cannot be selected as the second candidate container. Because LoRA is a fine-tuning parameter of the model, used to adjust the model output to meet specific requirements, if the LoRA loaded in the container is inconsistent with the LoRA requested by the task, the model output may not meet expectations and may even cause the task to fail. In this case, if the LoRA loaded in the container does not match the LoRA requested by the task, the system may need to first uninstall the old LoRA and then load the new LoRA, adding additional overhead.

[0081] For example, if the first model is identified as FLUX-D and the first LoRA is identified as LoRA1, then a container in the idle container pool that has a loaded model identified as FLUX-D and no LoRA loaded can be determined as the second candidate container. If a container has a loaded model identified as FLUX-D and a loaded LoRA identified as LoRA2, then that container cannot be selected as the second candidate container.

[0082] In this embodiment, if a fully matched first candidate container is not found, the second candidate container is selected to perform the first reasoning task. By reusing the loaded model and only loading LoRA, computing resources can be significantly saved.

[0083] In one embodiment, after step S5, the method further includes:

[0084] Step S6: If the second candidate container is not found, construct a target container in the intelligent computing center;

[0085] Step S7: After loading the first model and the first LoRA in the target container, route the inference request of the first inference task to the target container, and schedule the computing resources of the target container to execute the first inference task.

[0086] In the above embodiment, when the second candidate container is not found, the intelligent computing center will dynamically build a target container, that is, start a new container instance. Subsequently, the first model and the first LoRA are loaded in the target container, and the reasoning task request is routed to the target container. After receiving the reasoning request, the target container needs to use its own computing resources to perform this task. Computing resources include the processing power of computing devices such as the CPU and GPU of the server where the container is located. Scheduling refers to the reasonable allocation and management of these computing resources to ensure that the first reasoning task can be efficiently executed in the target container. For example, based on the complexity of the task and the current load of the container, decide how many CPU time slices or GPU cores to allocate to process this reasoning task, so that the task can be completed quickly and accurately, and the reasoning results can be returned.

[0087] In this embodiment, even if no existing containers are available, they can be dynamically created to meet task requirements, avoiding idle resources or task blocking. Starting new containers only when necessary can reduce the overhead of repeated loading of models and LoRA, improving overall efficiency.

[0088] In one embodiment, the first reasoning request further includes a first workflow identifier of the first reasoning task;

[0089] The method further comprises:

[0090] Step S8: receiving a second reasoning request for a second reasoning task, where the second reasoning request includes a second workflow identifier for the second reasoning task;

[0091] Step S9: When the first workflow identifier and the second workflow identifier match, the second inference request is routed to the candidate container for executing the first inference task, and the computing resources of the candidate container for executing the first inference task are scheduled to execute the second inference task.

[0092] In the above embodiment, the workflow identifier uniquely identifies the workflow to which this inference task belongs. A workflow can be understood as a series of ordered tasks. This identifier can be used to distinguish inference tasks in different workflows. For example, different business scenarios may have different workflows, each with its own unique identifier.

[0093] By determining whether the first and second workflow identifiers match, we can determine whether the first and second inference tasks belong to the same workflow. If so, we send the second inference request to the candidate container that executed the first inference task. The computing resources of the candidate container that executed the first inference task are then dispatched to execute the second inference task.

[0094] In the above embodiment, when two inference tasks belong to the same workflow, the computing resources of the already running candidate container are reused, avoiding the allocation of new computing resources for each inference task separately, improving the utilization of computing resources, and reducing the computing resource overhead of the system.

[0095] In one embodiment, the method further comprises:

[0096] Step S10: using a serverless mechanism to regularly detect the idle time of the containers in the idle container pool;

[0097] Step S11: clearing the containers whose idle time exceeds a preset time.

[0098] In the above embodiment, the serverless mechanism does not mean that there are no servers at all. Rather, it means that developers do not need to pay too much attention to underlying operations such as server management and configuration, as the platform handles these tasks. Regular monitoring means that the intelligent computing center counts the idle time of each container in the idle container pool at regular intervals (such as every 10 minutes). The idle time can be understood as the length of time that the container is inactive. The preset time can be understood as a pre-set time threshold, such as 30 minutes.

[0099] If a container has been idle for longer than the preset time, it will be cleared. This means removing the container from the idle container pool, freeing up system resources such as memory and CPU.

[0100] In this embodiment, containers that have been idle for a long time are promptly cleared to free up the system resources they occupy, allowing these resources to be used by other tasks or containers in need. This improves overall resource utilization efficiency and avoids resource waste. For example, this prevents excessive memory usage by containers from causing other running tasks to suffer performance degradation due to insufficient memory.

[0101] See also Figure 2 In related technologies, ComfyUI is often used for text-to-image tasks, generating images based on input text descriptions. As a backend service, it handles the computation and logic associated with image generation. ComfyUI has an internal task queue mechanism, meaning that only one inference task can be in progress at any given moment. Each inference task requires reloading the corresponding inference model from scratch, which consumes time and wastes computing resources.

[0102] See also Figure 3 and Figure 4In an embodiment of the present application, the identifiers of the models corresponding to each container group (such as Flux-d and Flux-s) and the identifiers of LoRA (such as LoRA1 and LoRA2) are first registered in the controller. The scheduler receives the user's inference request, which includes the identifier of the model corresponding to the user's inference task and the identifier of LoRA. The scheduler searches for candidate containers in the idle container pool based on the identifier of the model corresponding to the user's inference task and the identifier of LoRA, and according to the information of each pre-registered container group: first, search for idle containers that fully match both the model and Lora; if there is no complete match, search for containers that match the model and have no Lora requirements; if none are found, start a new container. In addition, it is necessary to ensure that requests for the same workflow are routed to the same container as much as possible. Moreover, all containers will be cleared regularly by serverless.

[0103] In this way, the intelligent computing center can accurately match the loaded container in the idle container pool based on the identifier of the model corresponding to the inference task and the identifier of LoRA. If a matching container already exists in the container pool, there is no need to reload the model and LoRA. The computing power resources of the existing container can be directly reused to perform the inference task, thus reducing the waste of computing power resources. In addition, model loading and LoRA initialization are time-consuming operations. By reusing the loaded container, the overhead of repeated loading is avoided, significantly shortening the response time of the inference task.

[0104] See Figure 5 , Figure 5 This is a structural diagram of a computing power scheduling device for an intelligent computing center provided by the present invention. Figure 5 As shown, the computing power scheduling device 500 of the intelligent computing center includes:

[0105] A first receiving module 501 is configured to receive a first reasoning request for a first reasoning task, where the first reasoning request includes an identifier of a first model corresponding to the first reasoning task and an identifier of a first low-rank adaptive LoRA;

[0106] A first search module 502 is configured to search for a first candidate container in an idle container pool of an intelligent computing center based on the identifier of the first model and the identifier of the first LoRA;

[0107] The first scheduling module 503 is configured to, when the first candidate container is found, route the first inference request to the first candidate container and schedule computing resources of the first candidate container to execute the first inference task.

[0108] In one embodiment, the first candidate container is a container whose identifier of a loaded model matches the identifier of the first model, and whose identifier of a loaded LoRA matches the identifier of the first LoRA.

[0109] In one embodiment, the apparatus further comprises:

[0110] a second search module, configured to search for a second candidate container in the idle container pool if the first candidate container is not found, where the second candidate container is a container whose identifier of the loaded model matches the identifier of the first model and for which LoRA is not loaded;

[0111] The second scheduling module is configured to, when the second candidate container is found, route the inference request of the first inference task to the second candidate container and schedule computing resources of the second candidate container to execute the first inference task.

[0112] In one embodiment, the apparatus further comprises:

[0113] A first construction module is configured to construct a target container in the intelligent computing center if the second candidate container is not found;

[0114] The third scheduling module is used to route the inference request of the first inference task to the target container after loading the first model and the first LoRA in the target container, and schedule the computing power resources of the target container to execute the first inference task.

[0115] In one embodiment, the first reasoning request further includes a first workflow identifier of the first reasoning task;

[0116] The device further comprises:

[0117] A first receiving module, configured to receive a second reasoning request for a second reasoning task, where the second reasoning request includes a second workflow identifier for the second reasoning task;

[0118] A fourth scheduling module is used to route the second inference request to a candidate container for executing the first inference task when the first workflow identifier and the second workflow identifier match, and to schedule the computing resources of the candidate container for executing the first inference task to execute the second inference task.

[0119] In one embodiment, the apparatus further comprises:

[0120] A first detection module is configured to periodically detect the idle time of containers in the idle container pool by using a serverless mechanism;

[0121] The first clearing module is configured to clear the container whose idle time exceeds a preset time.

[0122] The computing power scheduling device for the intelligent computing center provided by the present invention is capable of implementing the various processes of each embodiment of the computing power scheduling method for the above-mentioned intelligent computing center. The technical features correspond one to one and can achieve the same technical effects. To avoid repetition, they will not be described here.

[0123] It should be noted that the computing power scheduling device of the intelligent computing center in the present invention can be a device, or a component, integrated circuit, or chip in an electronic device.

[0124] The present invention also provides an electronic device, see Figure 6 , Figure 6 The electronic device includes a memory 601, a processor 602, and a program or instruction stored in the memory 601 and executed by the processor 602. Figure 1 Any steps in the corresponding embodiment of the computing power scheduling method for the intelligent computing center and the same beneficial effects are achieved will not be repeated here.

[0125] The processor 602 may be a CPU, an ASIC, an FPGA, or a GPU.

[0126] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned embodiment of the computing power scheduling method for the intelligent computing center can be completed through hardware related to program instructions, and the program can be stored in a readable medium.

[0127] The present invention also provides a readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above Figure 1 The corresponding steps in the embodiment of the computing power scheduling method of the intelligent computing center can achieve the same technical effect. To avoid repetition, they are not repeated here. The storage medium is such as a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] The present invention also provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the above Figure 1 The various processes of the implementation method of the computing power scheduling method of the corresponding intelligent computing center can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0129] The terms "first", "second" and the like in the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. In addition, the terms "comprise" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or devices. In addition, "and / or" is used in this application to represent at least one of the connected objects, for example A and / or B and / or C, which means comprising seven situations including single A, single B, single C, and both A and B exist, both B and C exist, both A and C exist, and both A, B and C exist.

[0130] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or second terminal device, etc.) to execute the methods of each embodiment of the present application.

[0132] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A computing power scheduling method for an intelligent computing center, characterized in that: include: Step S1: receiving a first reasoning request for a first reasoning task, where the first reasoning request includes an identifier of a first model corresponding to the first reasoning task and an identifier of a first low-rank adaptive LoRA; Step S2: searching for a first candidate container in an idle container pool of an intelligent computing center based on the identifier of the first model and the identifier of the first LoRA; Step S3: When the first candidate container is found, the first inference request is routed to the first candidate container, and computing resources of the first candidate container are scheduled to execute the first inference task.

2. The computing power scheduling method of the intelligent computing center according to claim 1, characterized in that: The first candidate container is a container whose identifier of the loaded model matches the identifier of the first model, and whose identifier of the loaded LoRA matches the identifier of the first LoRA.

3. The computing power scheduling method of the intelligent computing center according to claim 2, characterized in that: After step S3, the method further includes: Step S4: If the first candidate container is not found, search the idle container pool for a second candidate container, where the second candidate container is a container whose identifier of the loaded model matches the identifier of the first model and for which LoRA is not loaded. Step S5: When the second candidate container is found, the inference request of the first inference task is routed to the second candidate container, and the computing resources of the second candidate container are scheduled to execute the first inference task.

4. The computing power scheduling method of the intelligent computing center according to claim 3, characterized in that: After step S5, the method further includes: Step S6: If the second candidate container is not found, construct a target container in the intelligent computing center; Step S7: After loading the first model and the first LoRA in the target container, route the inference request of the first inference task to the target container, and schedule the computing resources of the target container to execute the first inference task.

5. The computing power scheduling method of an intelligent computing center according to any one of claims 1 to 4, characterized in that: The first reasoning request further includes a first workflow identifier of the first reasoning task; The method further comprises: Step S8: receiving a second reasoning request for a second reasoning task, where the second reasoning request includes a second workflow identifier for the second reasoning task; Step S9: When the first workflow identifier and the second workflow identifier match, the second inference request is routed to the candidate container for executing the first inference task, and the computing resources of the candidate container for executing the first inference task are scheduled to execute the second inference task.

6. The computing power scheduling method of an intelligent computing center according to any one of claims 1 to 4, characterized in that: The method further comprises: Step S10: using a serverless mechanism to regularly detect the idle time of the containers in the idle container pool; Step S11: clearing the containers whose idle time exceeds a preset time.

7. A computing power scheduling device for an intelligent computing center, characterized in that: include: A first receiving module is configured to receive a first reasoning request for a first reasoning task, where the first reasoning request includes an identifier of a first model corresponding to the first reasoning task and an identifier of a first low-rank adaptive LoRA; A first search module is configured to search for a first candidate container in an idle container pool of an intelligent computing center based on an identifier of the first model and an identifier of the first LoRA; A first scheduling module is configured to, when the first candidate container is found, route the first inference request to the first candidate container and schedule computing resources of the first candidate container to execute the first inference task.

8. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the computing power scheduling method for an intelligent computing center as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the computing power scheduling method for an intelligent computing center according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the computing power scheduling method of the intelligent computing center as described in any one of claims 1 to 6.

Images