A dynamic sharding method for a three-dimensional matrix task, an electronic device, and a medium

Abstract

The application discloses a dynamic fragmentation method of a three-dimensional matrix task, an electronic device and a medium. First, the number of online nodes, the free GPU display memory of the online nodes and the CPU NUMA topology are acquired, and the expected running time and resource demand weight of each task in the three-dimensional matrix task are determined. Then, the three-dimensional matrix task is divided into multiple parts through an improved 0-1 knapsack algorithm based on the number of online nodes, the free GPU display memory of the online nodes, the CPU NUMA topology, the expected running time and the resource demand weight of each task. The dynamic fragmentation method has adaptability, can eliminate long tail delay caused by uneven node load, makes task distribution closer to the theoretical optimum, and further reduces the whole test time consumption by at least 80%, so that the actual utilization rate of heterogeneous computing power resources is close to the theoretical upper limit.

Description

Technical Field

[0001] This invention relates to the field of model compilation and verification technology, and in particular to a dynamic fragmentation method, electronic device and medium for a three-dimensional matrix task. Background Technology

[0002] With the continuous evolution of software development technologies, the field of artificial intelligence is placing increasingly higher demands on the efficiency and reliability of the entire software development, testing, and delivery process. The CI / CD system, based on the DevOps philosophy, integrates continuous integration (CI), continuous delivery (CD), and continuous deployment (CD) into an automated pipeline. This system can rapidly respond to changing requirements, simplify and accelerate software construction and deployment, improve business flexibility while reducing operating costs, and has been widely adopted.

[0003] However, the number of Cartesian product combinations for target delivery architectures (x86 / ARM / NPU, etc.), compiler toolchains (GCC / Clang / LLVM-NPU, etc.), and model specifications (1 B→175 B) in AI projects is enormous, reaching hundreds. Consequently, if traditional static matrix tasks are used, they need to be executed sequentially on fixed nodes, resulting in long processing times, low resource utilization, and failure at any single point of failure causing the entire round to be restarted. In addition, the compiler caches of different chips cannot be shared, and bandwidth limitations in remote data centers further increase iteration time.

[0004] To address the aforementioned issues, some technical solutions mention task sharding methods. However, existing solutions are mostly limited to "single-dimensional" or "two-dimensional" test matrices (such as container × test cases, board × firmware), lacking support for three-dimensional and above explosion scenarios such as "compiler × architecture × model." When the number of tasks exceeds 100, the scheduling granularity is coarse, and resource matching efficiency drops sharply. Furthermore, existing solutions generally use static sharding, with task sharding typically hard-coded in the pipeline code. This makes it impossible to dynamically adjust sharding boundaries based on real-time GPU / CPU availability, historical failure probabilities, and network topology, resulting in both congestion on high-load nodes and idle nodes. Embedded solutions only cache firmware, and container solutions only cache images, lacking a unified binary caching layer with the "compiler-architecture-model" triple as the key. Repeated compilation and data transfer across regional data centers account for more than 40% of the time, and version consistency is difficult to guarantee. Furthermore, existing technologies do not handle failures by rerunning the entire round or retrying at a single point. They do not construct a task dependency graph to identify "sibling tasks that are related to the failure dimension" and cannot perform minimum dependency set rearrangement, resulting in a large amount of ineffective computing power consumption in microservice and heterogeneous hardware scenarios. Summary of the Invention

[0005] To address some or all of the problems in existing technologies and to improve the processing efficiency of "compiler × architecture × model" three-dimensional matrix tasks, the first aspect of this invention provides a dynamic partitioning method for three-dimensional matrix tasks, comprising: Obtain the number of online nodes (Agents), the idle GPU memory of the online nodes, and the CPU NUMA topology; Determine the estimated runtime and resource requirement weights for each task in the 3D matrix task; The improved 0-1 knapsack algorithm divides the three-dimensional matrix task into multiple parts based on the number of online nodes, the idle GPU memory of the online nodes, the CPU NUMA topology, the estimated running time of each task, and the resource requirement weight.

[0006] Furthermore, the task The resource demand weights are based on their historical average execution time. GPU memory requirements Compiler affinity weight Sure: , in, These are weighting coefficients, automatically optimized and determined through grid search.

[0007] Furthermore, the historical average execution time The GPU memory requirement was determined through a gene database query. Based on task The corresponding model size and compiler type are estimated.

[0008] Furthermore, the optimization of the weighting coefficients includes: Based on historical scheduling logs, determine the execution effect under different combinations of coefficients; With the goal of "shortest end-to-end execution time", the system traverses and searches within a preset parameter grid to determine the optimal combination of coefficients.

[0009] Furthermore, the improved 0-1 knapsack algorithm includes: minimizing the total load variance of each task under the constraint that the GPU memory usage ratio of online nodes is less than a preset value. , in, For the first Task Total load, ; This is the theoretical average load. ,in This represents the number of online nodes.

[0010] Furthermore, the preset value is 80%.

[0011] Furthermore, the improved 0-1 knapsack algorithm also includes ensuring that the difference in the number of tasks contained in each task does not exceed a specified value.

[0012] Furthermore, the specified value is 20%.

[0013] Furthermore, when a node becomes disconnected or the memory malfunctions, tasks that have not yet started are re-sharded according to the dynamic sharding method described above.

[0014] Based on the dynamic sharding method described above, a second aspect of the present invention provides an electronic device for dynamically sharding a three-dimensional matrix task, comprising a memory and a processor, wherein the memory is configured to store a computer program that executes the dynamic sharding method described above when the processor is running.

[0015] The present invention also provides a computer-readable storage medium for dynamically sharding a three-dimensional matrix task, which stores a computer program that executes the dynamic sharding method as described above when running on a processor.

[0016] This invention provides a dynamic sharding method for three-dimensional matrix tasks. It acquires real-time information on the current status of online nodes, node GPU memory, and CPU NUMA topology to partition the task and obtain the optimal sharding boundary. Simultaneously, some feature sub-genes in the partitioning algorithm are dynamically statistically analyzed and continuously updated with the task execution history, exhibiting adaptability. This dynamic sharding algorithm eliminates long-tail latency caused by uneven node load, making task distribution closer to the theoretical optimality. This reduces the overall test round time from the industry norm of tens of minutes to minutes, a reduction of over 80%, bringing the actual utilization rate of heterogeneous computing resources close to the theoretical upper limit. The dynamic sharding method collaboratively designs real-time load awareness and task affinity aggregation. On the one hand, it ensures accurate matching between resource-intensive tasks and sufficient nodes; on the other hand, by aggregating and scheduling tasks with the same compiler or similar characteristics, it maximizes the reuse probability of cached hot data. Attached Figure Description

[0017] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the various embodiments of the present invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope. In the drawings, identical or corresponding parts will be indicated by identical or similar reference numerals for clarity.

[0018] Figure 1The diagram shows a flowchart of a dynamic partitioning method for a three-dimensional matrix task according to an embodiment of the present invention. Detailed Implementation

[0019] In the following description, the invention is described with reference to various embodiments. However, those skilled in the art will recognize that the embodiments may be practiced without one or more specific details or in conjunction with other alternatives and / or additional methods or components. In other instances, well-known structures or operations are not shown or described in detail so as not to obscure the inventive points of the invention. Similarly, for illustrative purposes, specific numbers and configurations are set forth to provide a comprehensive understanding of embodiments of the invention. However, the invention is not limited to these specific details.

[0020] In this specification, references to "an embodiment" or "this embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. The phrase "in one embodiment" appearing throughout this specification does not necessarily refer to the same embodiment in all instances.

[0021] It should be noted that the embodiments of the present invention describe the method steps in a specific order; however, this is only for illustrating the specific embodiment and not for limiting the order of the steps. On the contrary, in different embodiments of the present invention, the order of the steps can be adjusted according to actual needs.

[0022] In the field of AI / ML model compilation and verification, testing tasks exhibit typical three-dimensional combined characteristics: compiler version... Target chip architecture AI model versions, including compiler versions such as GCC, Clang, ICC, XLA, TVM, etc., target chip architectures such as ARM64, AMD64, RISC-V, NPU, DSA, etc., and AI model versions such as ResNet-50, BERT-Large, YOLOv8, etc. As mentioned above, the number of three-dimensional combinations is enormous. For example, when including 10 compilers, 4 architectures, and 20 models, the number of combinations will exceed 100. This makes the compilation and verification using existing technologies face a serious problem of coarse scheduling granularity. Specifically, existing technologies typically hard-code parallelism in the pipeline code, which cannot dynamically adjust concurrency based on real-time GPU memory availability, CPU NUMA topology, and network bandwidth levels. This results in task queuing during peak periods and resource idleness during off-peak periods, leading to inefficient resource utilization. Furthermore, existing technologies lack intelligent scheduling based on historical failure probabilities; high-failure-rate tasks can block the entire task flow, slowing down critical paths and exhibiting high failure sensitivity. Simultaneously, while existing solutions like Kubernetes HPA support Pod-level scaling, their fixed test task sharding strategies cannot re-divide task boundaries according to load changes, negating the benefits of scaling and resulting in delayed elastic response. To address these issues, this invention uses Jenkins dynamic sharding to divide high-dimensional combinations into elastically schedulable subtasks on demand. It further combines distributed caching and a minimum failure reordering strategy, reducing the overall test duration from hours to minutes, significantly improving the efficiency and reliability of AI software delivery.

[0023] The technical solution of the present invention will be further described below with reference to the accompanying drawings of the embodiments.

[0024] Figure 1 This diagram illustrates a flowchart of a dynamic partitioning method for a three-dimensional matrix task according to an embodiment of the present invention. Figure 1 As shown, a dynamic partitioning method for a three-dimensional matrix task includes: First, in step 101, the resource status is determined. Before sharding, the number of online nodes (Agents), the idle GPU memory of the online nodes, and the CPU NUMA topology are obtained in real time. Next, in step 102, resource requirements are determined. The estimated runtime and resource requirement weights for each task in the three-dimensional matrix task to be sliced ​​are determined. The estimated runtime can be estimated based on the historical execution time of the corresponding task. In one embodiment of the invention, the historical execution time is obtained from gene pool statistics. The gene pool data is dynamically updated, meaning it continuously updates with the task execution history, thus enabling the dynamic slicing method to have task-characteristic adaptability. In one embodiment of the invention, the resource requirement weight is a comprehensive resource requirement weight, which normalizes and encodes multi-dimensional heterogeneous resource requirements, such as time, GPU memory, and affinity, into a single scalar weight value, facilitating unified processing by subsequent combinatorial optimization algorithms. In one embodiment of the invention, the resource requirement weight is determined based on its historical average execution time, GPU memory requirements, and compiler affinity weight. The historical average execution time is also obtained based on gene pool statistics, while the GPU memory requirements are based on the task... The corresponding model size and compiler type are estimated. The role of the compiler affinity weight is to drive tasks with the same compiler version to aggregate to the same node, thereby improving the cache hit rate.

[0025] In one embodiment of the present invention, the task The resource demand weights are based on their historical average execution time. GPU memory requirements Compiler affinity weight Sure: , in, The weighting coefficients are automatically optimized and determined through grid search. Specifically, in one embodiment of the invention, historical scheduling logs are collected to determine the actual execution effect under different coefficient combinations. Then, with "shortest end-to-end execution time" as the optimization objective, a traversal search is performed within a preset parameter grid range to determine the optimal coefficient combination. In another embodiment of the invention, the weighting coefficients can also be periodically recalibrated as business characteristics evolve. Finally, in step 103, task sharding is performed. Using an improved 0-1 knapsack algorithm, based on the number of online nodes, the available GPU memory of the online nodes, the CPU NUMA topology, the estimated runtime of each task, and the resource requirement weights, the three-dimensional matrix task is divided into multiple parts, generating a Map shard-Id→List. <task>In one embodiment of the present invention, the optimization objective of the improved 0-1 knapsack algorithm is to minimize the total load variance of each task: , in, For the first Task Total load, ; This is the theoretical average load. ,in This represents the number of online nodes.

[0026] In one embodiment of the present invention, the improved 0-1 knapsack algorithm includes a two-layer mechanism of hard constraints and soft constraints. Hard constraints are inviolable conditions that the algorithm must satisfy, while soft constraints are part of the optimization objective, allowing for appropriate relaxation when necessary to ensure the hard constraints are met. In one embodiment of the present invention, the hard constraint is a memory limit constraint, specifically, ensuring that the GPU memory usage ratio of each node is less than a preset value, preferably 80%. This hard constraint allows each node's GPU memory to retain a portion of its buffer to cope with memory fluctuations. In one embodiment of the present invention, the soft constraint is a task quantity constraint, specifically, ensuring that the difference in the number of tasks in each task does not exceed a specified value, preferably 20%.

[0027] The improved 0-1 knapsack algorithm described above can ensure that the load on each online node is as balanced as possible. Tests show that, using the dynamic sharding method described above, under typical test loads containing hundreds of dimensions, the end-to-end execution time can be reduced by more than 80% compared to before optimization, and the actual utilization rate of heterogeneous computing resources is close to the theoretical upper limit.

[0028] In one embodiment of the present invention, when a node becomes disconnected or its memory malfunctions, the unstarted tasks can be further reshuffled using the method described above and migrated to the remaining online nodes.

[0029] In one embodiment of the present invention, before implementing the dynamic sharding method as described above, high-risk tasks can be identified and sharded separately to reduce the number of retries due to resource exclusivity strategies.

[0030] In another embodiment of the present invention, when scaling up or down a Pod, the task sharding strategy is recalculated synchronously to ensure that the scaling resources take effect immediately. This significantly reduces task queuing and waiting time in scenarios with fluctuating cluster load, and reduces elastic response latency from minutes to seconds.

[0031] The dynamic fragmentation method described above can also be extended to technical fields such as chip EDA verification, automotive functional safety, and cloud-native security. In the cloud-native security field, container image vulnerability scanning faces challenges related to OS distributions. Language version The CVE database undergoes "three-dimensional expansion." The dynamic sharding method described earlier is employed to flexibly allocate tasks based on the load of scanning nodes, reducing the time required for full scans from hours to minutes.

[0032] It should be understood that in other embodiments of the present invention, other algorithms may be used to shard the task, as long as the constraints described above are met and load balancing of each node is achieved.

[0033] Based on the dynamic sharding method described above, the present invention also provides an electronic device for dynamically sharding a three-dimensional matrix task, comprising a memory and a processor, wherein the memory is configured to store a computer program that executes the dynamic sharding method described above when the processor is running.

[0034] The present invention also provides a computer-readable storage medium for dynamically sharding a three-dimensional matrix task, which stores a computer program that executes the dynamic sharding method as described above when running on a processor.

[0035] The dynamic sharding method described above can be encapsulated into a declarative syntax natively recognizable by Jenkins, ensuring compatibility with existing CI / CD infrastructure without requiring modifications to the underlying infrastructure. The Pipeline script implements sharding execution through parallel blocks and aggregates results using `collect` to avoid serialization exceptions caused by Groovy closure captures. This dynamic sharding method executes within methods annotated with `@NonCPS`, preventing the Jenkins Pipeline DSL's CPS transformer from incorrectly transforming complex object streams. Persistence via Redis or workspace files supports pipeline recovery after a Master restart. Simultaneously, a `try-finally` block ensures that GPU memory is released and temporary build directories are cleaned up when an Agent goes offline, preventing cluster resource exhaustion.

[0036] By using the dynamic sharding method described above, combined with distributed caching and the least-failure-reordering strategy, the entire test round can be compressed from hours to minutes, significantly improving the efficiency and reliability of AI software delivery.

[0037] Although various embodiments of the invention have been described above, it should be understood that they are presented by way of example only and not as limitations. It will be apparent to those skilled in the art that various combinations, modifications, and alterations can be made without departing from the spirit and scope of the invention. Therefore, the breadth and scope of the invention disclosed herein should not be limited by the exemplary embodiments disclosed above, but should be defined solely by the appended claims and their equivalents.< / task>

Claims

1. A dynamic partitioning method for a three-dimensional matrix task, characterized in that, include: Obtain the number of online nodes, the available GPU memory of the online nodes, and the CPU NUMA topology; Determine the estimated runtime and resource requirement weights for each task in the 3D matrix task; The improved 0-1 knapsack algorithm divides the three-dimensional matrix task into multiple parts based on the number of online nodes, the idle GPU memory of the online nodes, the CPU NUMA topology, the estimated running time of each task, and the resource requirement weight.

2. The dynamic fragmentation method as described in claim 1, characterized in that, Task The resource demand weight Based on its historical average execution time GPU memory requirements Compiler affinity weight Sure: ， in, These are the weighting coefficients.

3. The dynamic fragmentation method as described in claim 2, characterized in that, The historical average execution time The GPU memory requirement was determined through a gene database query. Based on task The corresponding model size and compiler type are estimated.

4. The dynamic fragmentation method as described in claim 2, characterized in that, The weighting coefficients are automatically optimized and determined through grid search, including: Based on historical scheduling logs, determine the execution effect under different combinations of coefficients; With the goal of "shortest end-to-end execution time", the system traverses and searches within a preset parameter grid to determine the optimal combination of coefficients.

5. The dynamic fragmentation method as described in claim 1, characterized in that, The improved 0-1 knapsack algorithm includes minimizing the total load variance of each task under the constraint that the GPU memory usage ratio of online nodes is less than a preset value. ， in, For the first Task Total load, ; This is the theoretical average load. ,in This represents the number of online nodes.

6. The dynamic fragmentation method as described in claim 5, characterized in that, The improved 0-1 knapsack algorithm further includes ensuring that the difference in the number of tasks contained in each task does not exceed a specified value.

7. The dynamic fragmentation method as described in claim 6, characterized in that, The preset value is 80%, and the specified value is 20%.

8. The dynamic fragmentation method as described in claim 1, characterized in that, When a node becomes disconnected or there is a memory error, unstarted tasks are re-sharded.

9. An electronic device for dynamically segmenting a three-dimensional matrix task, characterized in that, The system includes a memory and a processor, wherein the memory is configured to store a computer program that executes the dynamic sharding method as described in any one of claims 1 to 8 when the processor is running.

10. A computer-readable storage medium for dynamically fragmenting a three-dimensional matrix task, characterized in that, The system contains a computer program that, when run on a processor, executes the dynamic sharding method as described in any one of claims 1 to 8.

Images