An emulation method and device considering load balancing and related equipment
By generating task queues of matching types in autonomous driving simulation tasks and allocating computing resources based on the availability of machine pools, the simulation task failure problem caused by OOM is solved, and load balancing and stability are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU WERIDE TECH LTD CO
- Filing Date
- 2022-08-18
- Publication Date
- 2026-08-04
AI Technical Summary
In the simulation of autonomous driving algorithms, existing technologies are prone to OutOfMemoryError (OOM), which can cause simulation tasks to fail.
By generating simulation tasks and inserting them into task queues that match the type, the target number of worker nodes is determined based on the number of task queues and the availability of the machine pool, and computing resources are allocated to run the simulation tasks, thus avoiding OutOfMemoryError (OOM).
Load balancing of simulation tasks was achieved, avoiding OutOfMemoryError (OOM) and improving the stability and efficiency of simulation operations.
Smart Images
Figure CN115345011B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of autonomous driving technology, and more specifically, to a simulation method, apparatus, and related equipment that takes into account load balancing. Background Technology
[0002] The simulation of autonomous driving algorithms requires a large amount of GPU computing resources. In the context of cloud computing, a cluster of machines with GPU computing resources is usually formed and the simulation is executed in a unified scheduling manner.
[0003] However, if the simulation task is not scheduled properly, an Out Of Memory (OOM) error may occur, causing the simulation task to fail. Summary of the Invention
[0004] In view of this, this application provides a simulation method, apparatus and related equipment that take into account load balancing, so as to solve one of the above-mentioned technical defects, especially the technical defect in the prior art that is prone to OOM and causes simulation task execution failure.
[0005] To achieve the above objectives, the first aspect of this application provides a simulation method that balances load balancing, comprising:
[0006] Based on the received simulation request, at least one simulation task is generated, and each simulation task is inserted into a task queue that matches the type of the simulation task.
[0007] Based on the number of simulation tasks in each task queue, determine the target number of working nodes required for the task queue.
[0008] Obtain the availability of the machine pool corresponding to each task queue, the availability including the number of worker nodes that each GPU card in each machine in the machine pool can start, and the number of worker nodes that have already started;
[0009] Based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to the task queue, the computing resources allocated to the task queue are determined, and the computing resources are used to run each simulation task in the task queue.
[0010] Preferably, the process of generating at least one simulation task based on the received simulation request includes:
[0011] Based on the received simulation request, obtain the set of simulation scenarios;
[0012] A simulation task is generated based on the simulation scenarios in the set of simulation scenarios that can run in the same simulation model.
[0013] Preferably, the process of obtaining the availability of the machine pool corresponding to each task queue includes:
[0014] Based on the type of each task queue, determine the machine pool corresponding to the task queue;
[0015] Based on the preset configuration information, determine the number of worker nodes that each GPU card of each machine in the machine pool can start;
[0016] Based on a dynamic two-dimensional array, determine the number of worker nodes that have been started for each GPU card of each machine in the machine pool.
[0017] Preferably, the process of determining the computing resources allocated to the task queue based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to the task queue includes:
[0018] Based on the availability of the machine pool corresponding to the task queue, the idle rate of each GPU card in the machine pool is determined. The idle rate of the GPU card is equal to the number of worker nodes that have been started in the GPU card divided by the number of worker nodes that can be started.
[0019] Based on the rule of sorting the idle rates from high to low, the GPU cards in the machine pool are sorted to obtain the GPU card queue;
[0020] Select the N target GPU cards that are at the front of the GPU card queue, wherein the number of remaining startable worker nodes of the N target GPU cards is greater than or equal to the number of target worker nodes M, and the number of remaining startable worker nodes of the (N-1) target GPU cards that are at the front is less than the number of target worker nodes M;
[0021] The M remaining bootable worker nodes from the N target GPU cards are determined as the computing resources of the task queue.
[0022] Preferably, the process of determining the M remaining bootable worker nodes from the N target GPU cards as the computing resources of the task queue includes:
[0023] Determine if N is 1. If so, determine the M remaining startable worker nodes in the target GPU card as the computing resources of the task queue.
[0024] If not, the remaining bootable worker nodes of the first (N-1) target GPU cards out of the N GPU cards are determined as the first resource, and the remaining bootable worker nodes of the Nth target GPU card are determined as the second resource of computing resources. The N GPU cards are arranged in descending order of idle rate, and the total number of remaining bootable worker nodes of the first resource and the second resource is M. The first resource and the second resource constitute the computing resources of the task queue.
[0025] Preferably, the process of running each simulation task in the task queue using the computing resources includes:
[0026] The target worker nodes are created using the computing resources through the container cluster management system Kubernetes (K8S).
[0027] Each simulation task in the task queue is run using the target working nodes.
[0028] Preferably, the process of determining the target number of working nodes required for each task queue based on the number of simulation tasks in each task queue includes:
[0029] The number of simulation tasks in each task queue is determined as the target number of working nodes required by the task queue.
[0030] A second aspect of this application provides a simulation device that balances load balancing, comprising:
[0031] A task queue generation unit is used to generate at least one simulation task based on the received simulation request, and insert each simulation task into a task queue that matches the type of the simulation task.
[0032] The resource requirement calculation unit is used to determine the number of target working nodes required by the task queue based on the number of simulation tasks in each task queue.
[0033] The available resource computing unit is used to obtain the availability of the machine pool corresponding to each task queue. The availability includes the number of worker nodes that can be started for each machine in the machine pool and each GPU card in each machine, and the number of worker nodes that have been started.
[0034] The computing resource allocation unit is used to determine the computing resources allocated to the task queue based on the target number of working nodes for each task queue and the availability of the machine pool corresponding to the task queue, and to use the computing resources to run each simulation task in the task queue.
[0035] A third aspect of this application provides a simulation device that balances load balancing, comprising: a memory and a processor;
[0036] The memory is used to store programs;
[0037] The processor is used to execute the program to implement the various steps of the above-described simulation method that takes into account load balancing.
[0038] A fourth aspect of this application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the various steps of the simulation method for balancing load as described above.
[0039] As described in the above technical solution, this application first generates at least one simulation task based on the received simulation request, and inserts each simulation task into a task queue that matches the type of the simulation task. It is understood that different types of simulation tasks have different GPU memory requirements. By arranging simulation tasks of the same type in the same task queue, the simulation tasks in the same task queue can have similar GPU memory requirements. Next, based on the number of simulation tasks in each task queue, the target number of worker nodes required by the task queue is determined, and the availability of the machine pool corresponding to each task queue is obtained. The availability includes the number of worker nodes that can be started by each machine in the machine pool and each GPU card in each machine, and the number of worker nodes that have already been started. It is understood that since different task queues have different GPU memory requirements, by setting up machine pools that correspond one-to-one with each task queue, the resource allocation in the machine pool can be made as close as possible to the requirements of the task queue while meeting the requirements of the task queue. Finally, based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to the task queue, the computing resources allocated to the task queue are determined, and the simulation tasks in the task queue are run using the computing resources. Since the availability is refined to the availability of each machine in the machine pool and each GPU card within each machine, the allocation of computing resources also considers the granularity of the machine and the GPU cards within it. This allows for the reasonable allocation of simulation tasks to the appropriate GPUs on the machines, avoiding simulation task failures due to OutOfMemoryError (OOM). Attached Figure Description
[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0041] Figure 1This is a schematic diagram of the simulation method for load balancing disclosed in the embodiments of this application;
[0042] Figure 2 This is a schematic diagram of the machine pool, machine, GPU, and worker node disclosed in the embodiments of this application;
[0043] Figure 3 This is a schematic diagram of a simulation device that balances load according to an embodiment of this application;
[0044] Figure 4 This is a schematic diagram of a simulation device that balances load balancing, as disclosed in an embodiment of this application. Detailed Implementation
[0045] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0046] The following describes a simulation method that balances load balancing, as provided in the embodiments of this application. Please also refer to... Figure 1 and Figure 2 The simulation method that takes load balancing into account provided in this application embodiment may include the following steps:
[0047] Step S101: Based on the received simulation request, generate at least one simulation task and insert each simulation task into a task queue that matches the type of the simulation task.
[0048] The simulation requests submitted by users can include scene configurations, datasets, etc. Based on the specific content of the simulation request, one or more simulation tasks can be generated. The type of simulation task determines its simulation computation requirements, such as the required simulation model and video memory (GPU memory) parameters. By grouping simulation tasks of the same type into a task queue, all simulation tasks within that queue have the same simulation computation requirements. This facilitates configuring a machine pool for each task queue, ensuring that the computing power of each machine in the pool can meet the simulation computation needs of the task queue.
[0049] Step S102: Based on the number of simulation tasks in each task queue, determine the number of target working nodes required for that task queue.
[0050] Here, a worker node refers to a container launched on the machine, and the target number of worker nodes is the number of containers that need to be launched. Typically, a single container can be used to run a simulation task.
[0051] Step S103: Obtain the availability of the machine pool corresponding to each task queue.
[0052] As mentioned above, each task queue is configured with a machine pool, and the computing power of each machine in the pool is sufficient to meet the simulation computing requirements of the task queue. For autonomous driving algorithm simulation tasks, the computation process involves a large amount of GPU computing resources. When executing simulation tasks, the simulation software loads multiple simulation models into the GPU's video memory. Then, the simulation software uses some machine learning frameworks to call these simulation models to derive the algorithm.
[0053] On the one hand, since each simulation model occupies a certain amount of video memory, there is an upper limit to the number of simulation models that each GPU can support. Exceeding this limit will cause an OutOfMemoryError (OOM), leading to simulation failure. On the other hand, each machine typically has a fixed number of GPU cards, such as 8 or 4. Therefore, when calculating availability, the availability of GPU cards on each machine in the machine pool should be fully considered.
[0054] Based on this, the availability information can include the number of worker nodes that each GPU card in each machine in the machine pool can start, and the number of worker nodes that have already started. This availability information allows us to know the availability of each machine in the machine pool and the availability of each GPU in each machine, which is beneficial for ensuring that OOM (Out of Memory) errors do not occur during subsequent simulation task scheduling and for achieving balanced scheduling among the machines.
[0055] Step S104: Based on the number of target worker nodes in each task queue and the availability of the machine pool corresponding to the task queue, determine the computing resources allocated to the task queue, and use the computing resources to run each simulation task in the task queue.
[0056] Since each simulation task in each task queue uses the same simulation model, the amount of GPU memory occupied when loading the same simulation model is also similar. Therefore, by using a machine pool corresponding to a task queue to execute each simulation task, the concurrency of a single GPU card can be improved.
[0057] This application first generates at least one simulation task based on the received simulation request, and inserts each simulation task into a task queue that matches the type of the simulation task. It is understood that different types of simulation tasks have different GPU memory requirements; by arranging simulation tasks of the same type in the same task queue, the simulation tasks in the same task queue can have similar GPU memory requirements. Next, based on the number of simulation tasks in each task queue, the target number of worker nodes required by the task queue is determined, and the availability of the machine pool corresponding to each task queue is obtained. The availability includes the number of worker nodes that can be started by each machine and each GPU card in the machine pool, and the number of worker nodes that have already been started. It is understood that since different task queues have different GPU memory requirements, by setting up a machine pool that corresponds one-to-one with each task queue, the resource allocation in the machine pool can be as close as possible to the requirements of the task queue while meeting the requirements of the task queue. Finally, based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to the task queue, the computing resources allocated to the task queue are determined, and the simulation tasks in the task queue are run using the computing resources. Since the availability is refined to the availability of each machine in the machine pool and each GPU card in each machine, the allocation of computing resources also takes into account the granularity of the machine and the GPU card in the machine. This allows simulation tasks to be reasonably allocated to the GPUs of the corresponding machines, avoiding simulation task failures due to OutOfMemoryError (OOM).
[0058] In some embodiments of this application, the process of generating at least one simulation task based on the received simulation request in step S101 may include:
[0059] S1, based on the received simulation request, obtain the set of simulation scenarios.
[0060] S2 generates a simulation task based on simulation scenarios in the set of simulation scenarios that can run in the same simulation model.
[0061] Different simulation models are responsible for simulation calculations of different simulation scenarios. After obtaining the set of simulation scenarios, it is also necessary to integrate similar simulation scenarios that can be supported by the same simulation model into a single simulation task based on the characteristics of each simulation scenario in the set. The calculations of these similar simulation scenarios are then executed simultaneously in this simulation task to avoid separate executions and to save computing resources and time.
[0062] It is understandable that if there are different types of simulation scenarios in the set of simulation scenarios, it means that it is necessary to split into multiple simulation tasks to perform simulation calculations on different types of simulation scenarios. That is, based on the received simulation request, multiple simulation tasks can be generated.
[0063] In some embodiments of this application, the process of determining the target number of working nodes required for each task queue based on the number of simulation tasks in each task queue may include:
[0064] The number of simulation tasks in each task queue is determined as the target number of working nodes required for that task queue.
[0065] This can be understood as a one-to-one correspondence between simulation tasks and target working nodes, with each target working node independently executing a simulation task.
[0066] In some embodiments of this application, the process of obtaining the availability of the machine pool corresponding to each task queue in step S103 above may include:
[0067] S1, based on the type of each task queue, determine the machine pool corresponding to that task queue.
[0068] Understandably, for each type of task queue, a machine pool is pre-configured, containing multiple machines, and each machine is equipped with multiple GPU cards.
[0069] S2, based on the preset configuration information, determines the number of worker nodes that each GPU card of each machine in the machine pool can start.
[0070] For example, based on the number of GPU cards in each machine and the parameters of the GPU cards themselves, the number of worker nodes that a GPU card can start can be calculated. By combining the number of GPU cards in each machine, the number of worker nodes that each machine can start can be obtained.
[0071] S3, based on the dynamic two-dimensional array, determines the number of worker nodes that have been started for each GPU card of each machine in the machine pool.
[0072] Understandably, this dynamic two-dimensional array is updated in real time by monitoring the startup status of each GPU card's worker nodes on each machine in the machine pool. It records the number of worker nodes that have been started on each GPU card on each machine in the machine pool. It should be noted that startup status broadly includes both startup and shutdown.
[0073] In some embodiments of this application, the process of determining the computing resources allocated to a task queue based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to that task queue may include:
[0074] S1. Based on the availability of the machine pool corresponding to the task queue, determine the idle rate of each GPU card in the machine pool.
[0075] The idle rate δ of the GPU card is equal to the number of already started worker nodes n1 divided by the number of worker nodes that can be started n2:
[0076] δ=n1 / n2
[0077] S2, based on the rule of sorting the GPU cards in the machine pool from high to low idle rate, to obtain the GPU card queue.
[0078] S3, select the N target GPU cards that are at the front of the GPU card queue.
[0079] Among them, the number of remaining bootable worker nodes k1 of these N target GPU cards is greater than or equal to the number of target worker nodes M, and the number of remaining bootable worker nodes k2 of the first (N-1) target GPU cards is less than the number of target worker nodes M:
[0080] k1≥M and k2 <M
[0081] N and M are both natural numbers. The remaining bootable worker nodes k of a certain GPU card can be calculated using the following equation:
[0082] k = n2 - n1
[0083] Where n2 is the number of worker nodes that the GPU card can start, and n1 is the number of worker nodes that the GPU card has already started.
[0084] As can be seen from the descriptions of S2 and S3, their main purpose is to find the N most idle target GPU cards from the machine pool, and the number of remaining startable worker nodes of these target GPU cards can cover the number of target worker nodes.
[0085] S4. Determine the M remaining bootable worker nodes from these N target GPU cards as the computing resources for this task queue.
[0086] In the above allocation of computing resources, the load of each GPU card was fully considered to avoid overloading the GPU card resources.
[0087] Since the number of remaining bootable worker nodes in these N target GPU cards can cover the number of target worker nodes M, we can either distribute the target worker nodes evenly among these N target GPU cards, or start with the target GPU with the highest idle rate among these N target GPU cards and fill each target GPU one by one.
[0088] Based on this, in some embodiments of this application, the process of S4 determining the M remaining bootable worker nodes from the N target GPU cards as computing resources for the task queue may include:
[0089] S41, determine if N is 1; if yes, execute S42; otherwise, execute S43.
[0090] Where N=1 means that the number of remaining startable worker nodes in the first GPU card in the GPU card queue, i.e. the GPU card with the highest idle rate in the machine pool, is sufficient to meet the target number of worker nodes.
[0091] S42, determine the M remaining bootable worker nodes in the target GPU card as the computing resources for this task queue.
[0092] That is, reserve M remaining bootable worker nodes from the remaining bootable worker nodes in the target GPU card as computing resources for the task queue.
[0093] S43, determine the remaining bootable working nodes of the first (N-1) target GPU cards among these N GPU cards as the first resource, and determine the remaining bootable working nodes of the Nth target GPU card as the second resource of computing resources.
[0094] Among them, the N GPU cards are arranged in descending order of idle rate, and the total number of remaining startable worker nodes of the first and second resources is M. The first and second resources constitute the computing resources of this task queue.
[0095] In practice, for S2 to S4 above, based on the availability of the machine pool, the GPU card with the highest idle rate is identified, and it is determined whether the GPU card has M remaining bootable worker nodes. If so, the M remaining bootable worker nodes of the GPU card are directly designated as computing resources for the task queue; otherwise, all remaining bootable worker nodes of the GPU card are reserved for the task queue. Then, the GPU card with the second highest idle rate is identified from the machine pool, and this process is repeated until enough remaining bootable worker nodes are reserved. Finally, all reserved remaining bootable worker nodes are designated as computing resources for the task queue.
[0096] Through the above process, the number of working nodes on a single GPU card can be precisely controlled, which is equivalent to controlling the number of simulation software / simulation models running on a single GPU card, avoiding OOM (Out of Memory) errors and improving the stability of simulation operations.
[0097] In some embodiments of this application, the process of using the computing resources to run each simulation task in the task queue in step S104 may include:
[0098] S1 uses the computing resources of the container cluster management system Kubernetes to create target worker nodes.
[0099] S2, using each target working node to run each simulation task in the task queue.
[0100] For example, by calling the relevant APIs of K8S, a corresponding container (target worker node) is created in the GPU card corresponding to the computing resource, and then each simulation task in the task queue is run in these target worker nodes. Specifically, each target worker node is responsible for running one simulation task.
[0101] The simulation device that balances load balancing provided in the embodiments of this application will be described below. The simulation device that balances load balancing described below can be referred to in correspondence with the simulation method that balances load balancing described above.
[0102] Please see Figure 3 The simulation device that balances load balancing provided in this application embodiment may include:
[0103] The task queue generation unit 21 is used to generate at least one simulation task based on the received simulation request, and insert each simulation task into a task queue that matches the type of the simulation task.
[0104] Resource requirement calculation unit 22 is used to determine the target number of working nodes required by the task queue based on the number of simulation tasks in each task queue.
[0105] Available resource computing unit 23 is used to obtain the availability of the machine pool corresponding to each task queue. The availability includes the number of worker nodes that can be started for each machine in the machine pool and each GPU card in each machine, and the number of worker nodes that have been started.
[0106] The computing resource allocation unit 24 is used to determine the computing resources allocated to the task queue based on the target number of working nodes in each task queue and the availability of the machine pool corresponding to the task queue, and to use the computing resources to run each simulation task in the task queue.
[0107] In some embodiments of this application, the process by which the task queue generation unit 21 generates at least one simulation task based on a received simulation request may include:
[0108] Based on the received simulation request, obtain the set of simulation scenarios;
[0109] A simulation task is generated based on the simulation scenarios in the set of simulation scenarios that can run in the same simulation model.
[0110] In some embodiments of this application, the process by which the resource requirement calculation unit 22 determines the target number of working nodes required by each task queue based on the number of simulation tasks in each task queue may include:
[0111] The number of simulation tasks in each task queue is determined as the target number of working nodes required by the task queue.
[0112] In some embodiments of this application, the process by which the resource computing unit 23 obtains the availability of the machine pool corresponding to each task queue may include:
[0113] Based on the type of each task queue, determine the machine pool corresponding to the task queue;
[0114] Based on the preset configuration information, determine the number of worker nodes that each GPU card of each machine in the machine pool can start;
[0115] Based on a dynamic two-dimensional array, determine the number of worker nodes that have been started for each GPU card of each machine in the machine pool.
[0116] In some embodiments of this application, the process by which the computing resource allocation unit 24 determines the computing resources allocated to the task queue based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to the task queue may include:
[0117] Based on the availability of the machine pool corresponding to the task queue, the idle rate of each GPU card in the machine pool is determined. The idle rate of the GPU card is equal to the number of worker nodes that have been started in the GPU card divided by the number of worker nodes that can be started.
[0118] Based on the rule of sorting the idle rates from high to low, the GPU cards in the machine pool are sorted to obtain the GPU card queue;
[0119] Select the N target GPU cards that are at the front of the GPU card queue, wherein the number of remaining startable worker nodes of the N target GPU cards is greater than or equal to the number of target worker nodes M, and the number of remaining startable worker nodes of the (N-1) target GPU cards that are at the front is less than the number of target worker nodes M;
[0120] The M remaining bootable worker nodes from the N target GPU cards are determined as the computing resources of the task queue.
[0121] In some embodiments of this application, the process by which the computing resource allocation unit 24 determines the M remaining startable worker nodes from the N target GPU cards as computing resources for the task queue may include:
[0122] Determine if N is 1. If so, determine the M remaining startable worker nodes in the target GPU card as the computing resources of the task queue.
[0123] If not, the remaining bootable worker nodes of the first (N-1) target GPU cards out of the N GPU cards are determined as the first resource, and the remaining bootable worker nodes of the Nth target GPU card are determined as the second resource of computing resources. The N GPU cards are arranged in descending order of idle rate, and the total number of remaining bootable worker nodes of the first resource and the second resource is M. The first resource and the second resource constitute the computing resources of the task queue.
[0124] In some embodiments of this application, the process by which the computing resource allocation unit 24 uses the computing resources to run each simulation task in the task queue may include:
[0125] The target worker nodes are created using the computing resources through the container cluster management system Kubernetes (K8S).
[0126] Each simulation task in the task queue is run using the target working nodes.
[0127] The load-balancing simulation device provided in this application embodiment can be applied to load-balancing simulation devices, such as computers. Optionally, Figure 4 The hardware structure block diagram of the simulation device that balances load balancing is shown. (Refer to...) Figure 4 The hardware structure of a simulation device that takes into account load balancing may include: at least one processor 31, at least one communication interface 32, at least one memory 33 and at least one communication bus 34.
[0128] In this embodiment, the number of processor 31, communication interface 32, memory 33 and communication bus 34 is at least one, and processor 31, communication interface 32 and memory 33 communicate with each other through communication bus 34;
[0129] The processor 31 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.
[0130] The memory 33 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;
[0131] The memory 33 stores a program, and the processor 31 can call the program stored in the memory 33. The program is used for:
[0132] Based on the received simulation request, at least one simulation task is generated, and each simulation task is inserted into a task queue that matches the type of the simulation task.
[0133] Based on the number of simulation tasks in each task queue, determine the target number of working nodes required for the task queue.
[0134] Obtain the availability of the machine pool corresponding to each task queue, the availability including the number of worker nodes that each GPU card in each machine in the machine pool can start, and the number of worker nodes that have already started;
[0135] Based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to the task queue, the computing resources allocated to the task queue are determined, and the computing resources are used to run each simulation task in the task queue.
[0136] Optionally, the refined and extended functions of the program can be found in the description above.
[0137] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor, the program being used for:
[0138] Based on the received simulation request, at least one simulation task is generated, and each simulation task is inserted into a task queue that matches the type of the simulation task.
[0139] Based on the number of simulation tasks in each task queue, determine the target number of working nodes required for the task queue.
[0140] Obtain the availability of the machine pool corresponding to each task queue, the availability including the number of worker nodes that each GPU card in each machine in the machine pool can start, and the number of worker nodes that have already started;
[0141] Based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to the task queue, the computing resources allocated to the task queue are determined, and the computing resources are used to run each simulation task in the task queue.
[0142] Optionally, the refined and extended functions of the program can be found in the description above.
[0143] In summary:
[0144] This application first generates at least one simulation task based on the received simulation request, and inserts each simulation task into a task queue that matches the type of the simulation task. It is understood that different types of simulation tasks have different GPU memory requirements; by arranging simulation tasks of the same type in the same task queue, the simulation tasks in the same task queue can have similar GPU memory requirements. Next, based on the number of simulation tasks in each task queue, the target number of worker nodes required by the task queue is determined, and the availability of the machine pool corresponding to each task queue is obtained. The availability includes the number of worker nodes that can be started by each machine and each GPU card in the machine pool, and the number of worker nodes that have already been started. It is understood that since different task queues have different GPU memory requirements, by setting up a machine pool that corresponds one-to-one with each task queue, the resource allocation in the machine pool can be as close as possible to the requirements of the task queue while meeting the requirements of the task queue. Finally, based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to the task queue, the computing resources allocated to the task queue are determined, and the simulation tasks in the task queue are run using the computing resources. Since the availability is refined to the availability of each machine in the machine pool and each GPU card in each machine, the allocation of computing resources also takes into account the granularity of the machine and the GPU card in the machine. This allows simulation tasks to be reasonably allocated to the GPUs of the corresponding machines, avoiding simulation task failures due to OutOfMemoryError (OOM).
[0145] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0146] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.
[0147] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A simulation method that takes into account load balancing, characterized in that, include: Based on the received simulation request, at least one simulation task is generated, and each simulation task is inserted into a task queue that matches the type of the simulation task. Based on the number of simulation tasks in each task queue, determine the target number of working nodes required for the task queue. Obtain the availability of the machine pool corresponding to each task queue, the availability including the number of worker nodes that each GPU card in each machine in the machine pool can start, and the number of worker nodes that have already started; Based on the target number of worker nodes for each task queue and the availability of the machine pool corresponding to the task queue, the computing resources allocated to the task queue are determined, and the computing resources are used to run each simulation task in the task queue. The process of determining the computing resources allocated to each task queue based on the target number of worker nodes and the availability of the machine pool corresponding to the task queue includes: Based on the availability of the machine pool corresponding to the task queue, the idle rate of each GPU card in the machine pool is determined. The idle rate of the GPU card is equal to the number of worker nodes that have been started in the GPU card divided by the number of worker nodes that can be started. Based on the rule of sorting the idle rates from high to low, the GPU cards in the machine pool are sorted to obtain the GPU card queue; Select the first N target GPU cards from the GPU card queue, wherein the number of remaining startable worker nodes of the N target GPU cards is greater than or equal to the number of target worker nodes M, and the number of remaining startable worker nodes of the first N-1 target GPU cards is less than the number of target worker nodes M. The M remaining bootable worker nodes from the N target GPU cards are identified as the computing resources of the task queue; The process of determining the M remaining bootable worker nodes from the N target GPU cards as the computing resources for the task queue includes: Determine if N is 1. If so, determine the M remaining startable worker nodes in the target GPU card as the computing resources of the task queue. If not, the remaining bootable worker nodes of the first N-1 target GPU cards out of the N target GPU cards are determined as the first resource, and the remaining bootable worker nodes of the Nth target GPU card are determined as the second resource of computing resources. The N target GPU cards are arranged in descending order of idle rate, and the total number of remaining bootable worker nodes of the first resource and the second resource is M. The first resource and the second resource constitute the computing resources of the task queue.
2. The method according to claim 1, characterized in that, The process of generating at least one simulation task based on a received simulation request includes: Based on the received simulation request, obtain the set of simulation scenarios; A simulation task is generated based on the simulation scenarios in the set of simulation scenarios that can run in the same simulation model.
3. The method according to claim 1, characterized in that, The process of obtaining the availability of the machine pool corresponding to each task queue includes: Based on the type of each task queue, determine the machine pool corresponding to the task queue; Based on the preset configuration information, determine the number of worker nodes that each GPU card of each machine in the machine pool can start; Based on a dynamic two-dimensional array, determine the number of worker nodes that have been started for each GPU card of each machine in the machine pool.
4. The method according to claim 1, characterized in that, The process of using the computing resources to run each simulation task in the task queue includes: The target worker nodes are created using the computing resources through the container cluster management system Kubernetes (K8S). Each simulation task in the task queue is run using the target working nodes.
5. The method according to claim 1, characterized in that, The process of determining the target number of worker nodes required for each task queue based on the number of simulation tasks in each task queue includes: The number of simulation tasks in each task queue is determined as the target number of working nodes required by the task queue.
6. A simulation device that also considers load balancing, characterized in that, include: A task queue generation unit is used to generate at least one simulation task based on the received simulation request, and insert each simulation task into a task queue that matches the type of the simulation task. The resource requirement calculation unit is used to determine the number of target working nodes required by the task queue based on the number of simulation tasks in each task queue. The available resource computing unit is used to obtain the availability of the machine pool corresponding to each task queue. The availability includes the number of worker nodes that can be started for each machine in the machine pool and each GPU card in each machine, and the number of worker nodes that have been started. The computing resource allocation unit is used to determine the computing resources allocated to the task queue based on the target number of working nodes in each task queue and the availability of the machine pool corresponding to the task queue, and to use the computing resources to run each simulation task in the task queue. The process of determining the computing resources allocated to each task queue based on the target number of worker nodes and the availability of the machine pool corresponding to the task queue includes: Based on the availability of the machine pool corresponding to the task queue, the idle rate of each GPU card in the machine pool is determined. The idle rate of the GPU card is equal to the number of worker nodes that have been started in the GPU card divided by the number of worker nodes that can be started. Based on the rule of sorting the idle rates from high to low, the GPU cards in the machine pool are sorted to obtain the GPU card queue; Select the first N target GPU cards from the GPU card queue, wherein the number of remaining startable worker nodes of the N target GPU cards is greater than or equal to the number of target worker nodes M, and the number of remaining startable worker nodes of the first N-1 target GPU cards is less than the number of target worker nodes M. The M remaining bootable worker nodes from the N target GPU cards are identified as the computing resources of the task queue; The process of determining the M remaining bootable worker nodes from the N target GPU cards as the computing resources for the task queue includes: Determine if N is 1. If so, determine the M remaining startable worker nodes in the target GPU card as the computing resources of the task queue. If not, the remaining bootable worker nodes of the first N-1 target GPU cards out of the N target GPU cards are determined as the first resource, and the remaining bootable worker nodes of the Nth target GPU card are determined as the second resource of computing resources. The N target GPU cards are arranged in descending order of idle rate, and the total number of remaining bootable worker nodes of the first resource and the second resource is M. The first resource and the second resource constitute the computing resources of the task queue.
7. A simulation device that also considers load balancing, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the simulation method that takes load balancing into account as described in any one of claims 1 to 5.
8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the simulation method that takes into account load balancing as described in any one of claims 1 to 5.