Performance prediction method and system for artificial intelligence calculation task, and electronic equipment

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a performance prediction method for an artificial intelligence calculation task, which comprises the following steps: disassembling the artificial intelligence calculation task into a plurality of execution units, and determining basic performance data of each execution unit on target hardware; constructing an asynchronous execution dependency graph based on the asynchronous dependency relationship between the execution units; wherein graph vertexes of the asynchronous execution dependency graph represent execution units and are marked with hardware resource labels required by execution, and graph directed edges represent asynchronous dependency relationships among the execution units and are marked with dependency types; and taking the asynchronous execution dependency graph, the basic performance data and the artificial intelligence hardware parameters as input, performing simulation through a hardware perceived performance simulation algorithm, and outputting a performance prediction result. And through synchronous modeling, asynchronous dependence and hardware resource constraint, the precision of performance prediction of the artificial intelligence calculation task is greatly improved. The invention further discloses a performance prediction system for the artificial intelligence calculation task and electronic equipment.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, such as a performance prediction method and system for artificial intelligence computing tasks, and electronic devices. Background Technology

[0002] With the rapid development of deep learning, the demand for computing power in AI training and inference tasks such as large language model training and real-time computer vision inference is exploding. To address this challenge, heterogeneous computing architectures composed of CPUs and GPUs (Graphics Processing Units), NPUs (Neural Processing Units), and other parallel strategies such as data parallelism, tensor parallelism, and pipelined parallelism are commonly adopted to improve system efficiency. In the actual deployment of tasks or hardware system design phases, the ability to quickly and accurately predict the performance indicators (such as training throughput and inference latency) of a specific task under a given hardware configuration and parallel strategy has become a key aspect of optimizing resource utilization, reducing development trial-and-error costs, and guiding hardware selection.

[0003] Currently, performance prediction methods based on simplified analysis models are commonly used in related technical fields. The core idea of ​​this method is to approximate the overall execution time of an AI computing task as a linear sum of the computation and communication times of each independent stage. This technical approach implicitly assumes an idealized premise: that all execution stages are executed strictly sequentially, and that the task's use of hardware resources (such as computing cores, memory bandwidth, and interconnect links) is exclusive, with no resource contention or waiting.

[0004] In the process of implementing the embodiments of this disclosure, at least the following problems were found in the related art: In modern AI computing tasks, the overlap between computation and communication, the complex asynchronous dependencies introduced by different parallel strategies, and the resulting dynamic competition for hardware resources lead to a highly asynchronous and uncertain execution process. Performance prediction methods based on simplified analysis models often result in significant discrepancies between predicted and actual performance, exhibiting low accuracy.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0007] This disclosure provides a method, system, and electronic device for performance prediction of artificial intelligence computing tasks, in order to improve the accuracy of performance prediction for artificial intelligence computing tasks.

[0008] In some embodiments, a performance prediction method for artificial intelligence computing tasks includes: decomposing the artificial intelligence computing task into multiple execution units and determining the basic performance data of each execution unit on the target hardware; constructing an asynchronous execution dependency graph based on the asynchronous dependencies between the execution units; wherein the vertices of the asynchronous execution dependency graph represent execution units and are labeled with the hardware resource tags required for execution, and the directed edges of the graph represent the asynchronous dependencies between execution units and are labeled with the dependency types; taking the asynchronous execution dependency graph, the basic performance data, and the artificial intelligence hardware parameters as input, simulating the performance using a hardware-aware performance simulation algorithm, and outputting the performance prediction result.

[0009] Optionally, based on the asynchronous dependencies between execution units, an asynchronous execution dependency graph is constructed, including: selecting a target dependency template from a predefined dependency pattern library according to the target parallel strategy and pipeline strategy; and generating an asynchronous execution dependency graph based on the target dependency template, according to the execution units and the hardware resource tags marked by the execution units.

[0010] Optionally, based on the target dependency template, an asynchronous execution dependency graph is generated according to the execution unit and the hardware resource tags labeled on the execution unit. This includes: using the execution unit as the graph vertex of the asynchronous execution dependency graph and associating the graph vertex with a hardware resource tag; establishing directed graph edges between the graph vertices according to the dependency pattern in the target dependency template and labeling the graph directed edges with dependency types; organizing the instantiated graph vertex set and graph directed edge set to construct a directed graph data structure and obtain the asynchronous execution dependency graph.

[0011] Optionally, the asynchronous execution dependency graph, basic performance data, and AI hardware parameters are taken as input, and a hardware-aware performance simulation algorithm is used to simulate the process and output performance prediction results, including: initializing the hardware resource state table, the queue to be executed, the queue to be executed, and the cumulative time; adding graph vertices without pre-dependencies in the asynchronous execution dependency graph to the queue to be executed, and updating the hardware resource state table to the occupied state; iteratively simulating the queue to be executed based on the basic performance data and AI hardware parameters; and outputting performance prediction results based on the cumulative time of the iterative simulation when both the queue to be executed and the queue to be executed are empty.

[0012] Optionally, based on basic performance data and AI hardware parameters, iterative simulation is performed on the queue to be executed, including: selecting graph vertices from the queue to be executed that meet the dependency conditions and whose hardware resources corresponding to the labeled hardware resource tags are all in an idle state at the current simulation time, as the vertices of the graph to be executed; calculating the estimated execution time when the vertices of the graph to be executed occupy hardware resources according to the basic performance data and AI hardware parameters; allocating execution time periods to the vertices of the graph to be executed according to the estimated execution time, and updating the hardware resource status table to the occupied state; releasing the hardware resources after the simulated execution of the vertices to be executed is completed, and adding the successor vertices of the vertices of the graph to be executed in the asynchronous execution dependency graph to the queue to be executed.

[0013] Optionally, when the vertex of the graph to be executed is a vertex of a computational class graph, the estimated execution time when the vertex of the graph to be executed occupies hardware resources is calculated based on the basic performance data and the artificial intelligence hardware parameters. This includes: obtaining the number of floating-point operations and the amount of data access corresponding to the vertex of the computational class graph in the basic performance data; determining the first duration of computational limitation based on the number of floating-point operations based on the peak computing power of the computing unit identified by the hardware resource tag in the artificial intelligence hardware parameters; determining the second duration of memory access limitation based on the amount of data access based on the effective bandwidth of the memory system associated with the hardware resource tag in the artificial intelligence hardware parameters; and determining the larger value between the first duration and the second duration as the estimated execution time of the vertex of the computational class graph.

[0014] Optionally, the performance prediction method for artificial intelligence computing tasks further includes: adjusting the configuration parameters of the target parallel strategy and pipeline strategy based on the performance bottleneck location information in the performance prediction results to generate a new strategy configuration; performing local reconstruction of the subgraph structure in the asynchronous execution dependency graph that is directly affected by the parameter adjustment according to the new strategy configuration to obtain an updated asynchronous execution dependency graph; and performing incremental local re-simulation based on the updated asynchronous execution dependency graph to obtain optimized performance prediction results.

[0015] Optionally, based on the new strategy configuration, the subgraph structure directly affected by parameter adjustments in the asynchronous execution dependency graph is partially reconstructed to obtain an updated asynchronous execution dependency graph. This includes: generating a target subgraph that conforms to the dependency pattern in the target dependency template based on the new strategy configuration; removing the original subgraph corresponding to the target subgraph in the asynchronous execution dependency graph; and connecting the target subgraph to the asynchronous execution dependency graph to obtain an updated asynchronous execution dependency graph.

[0016] In some embodiments, a performance prediction device for artificial intelligence computing tasks includes: a decomposition module configured to decompose the artificial intelligence computing task into multiple execution units and determine the basic performance data of each execution unit on target hardware; a construction module communicatively connected to the decomposition module and configured to construct an asynchronous execution dependency graph based on the asynchronous dependencies between execution units; wherein the vertices of the asynchronous execution dependency graph represent execution units and are labeled with hardware resource tags required for execution, and the directed edges of the graph represent asynchronous dependencies between execution units and are labeled with dependency types; and a simulation module communicatively connected to the construction module and configured to take the asynchronous execution dependency graph, basic performance data, and artificial intelligence hardware parameters as input, simulate them using a hardware-aware performance simulation algorithm, and output performance prediction results.

[0017] In some embodiments, the electronic device includes a processor, a memory, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the aforementioned performance prediction method for artificial intelligence computing tasks.

[0018] The performance prediction method, system, and electronic device for artificial intelligence computing tasks provided in this disclosure can achieve the following technical effects: This disclosed technical solution decomposes artificial intelligence computing tasks into multiple execution units. By constructing vertices labeled with hardware resources and directed edges labeled with dependency types, an asynchronous execution dependency graph is formed. This accurately depicts the complex computation, communication, and synchronization relationships between execution units under parallel modes such as data, tensors, and pipelines. Furthermore, a hardware-aware performance simulation algorithm combines the asynchronous execution dependency graph, execution unit performance, and hardware parameters to dynamically simulate the scheduling, competition, and execution processes of each execution unit under limited hardware resource constraints, achieving fine-grained extrapolation of the end-to-end execution behavior of the task. By synchronously modeling asynchronous dependencies and hardware resource constraints, the shortcomings of traditional linear superposition methods in ignoring parallel overlap and resource competition are effectively overcome, significantly improving the accuracy of performance prediction for artificial intelligence computing tasks. Simultaneously, the graph traversal-based performance simulation algorithm maintains high efficiency while ensuring prediction accuracy and can accurately identify and locate performance bottlenecks caused by specific resource saturation.

[0019] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0020] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein: Figure 1 This is a schematic flowchart of a performance prediction method for artificial intelligence computing tasks provided in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating another performance prediction method for artificial intelligence computing tasks provided in this disclosure embodiment; Figure 3 This is a flowchart illustrating another performance prediction method for artificial intelligence computing tasks provided in this disclosure embodiment; Figure 4 This is a flowchart illustrating another performance prediction method for artificial intelligence computing tasks provided in this disclosure embodiment; Figure 5A This is a schematic diagram of an asynchronous execution dependency graph provided in an embodiment of this disclosure; Figure 5B This is a schematic diagram simulating an asynchronous execution dependency graph provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of a performance prediction system for artificial intelligence computing tasks provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0021] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0022] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0023] Unless otherwise stated, the term "multiple" means two or more. In embodiments of this disclosure, the character " / " indicates that the preceding and following objects are in an "OR" relationship. For example, A / B means: A or B. The term "and / or" describes an association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B. The term "correspondence" can refer to an association or binding relationship; A corresponding to B means that there is an association or binding relationship between A and B.

[0024] The following explains some key technical terms used in the embodiments of this disclosure: Artificial intelligence computing tasks are computational workloads that run on heterogeneous computing systems (such as CPU+GPU / NPU) with the core objective of training or inferring machine learning models. AI computing tasks include Large Language Model (LLM) training tasks and real-time computer vision inference tasks. Large Language Model training involves iteratively optimizing model parameters on large amounts of text data to enable the model to understand and generate natural language. Real-time computer vision inference tasks apply a trained visual model to input data to generate prediction results in real time.

[0025] An asynchronous execution dependency graph is a directed graph data structure used to characterize the execution process of artificial intelligence computing tasks. An asynchronous execution dependency graph is defined as a directed graph G = (V, E), where V is the set of vertices and E is the set of directed edges. Each vertex v ∈ V represents an execution unit of the decomposed artificial intelligence computing task. Each vertex v is associated with a hardware resource label, explicitly identifying the specific hardware resources required for its execution. Each directed edge e = (u, v) ∈ E indicates an asynchronous dependency relationship from vertex u to vertex v. Each edge e is associated with a dependency type to distinguish different dependency properties. Figure 5A As shown, graph vertex a (CPU) is the starting module, graph vertex b (GPU) and graph vertex c (network) depend on graph vertex a, graph vertex d (CPU) depends on graph vertex b, and graph vertex e (GPU) depends on graph c. Each directed edge is labeled with its corresponding dependency type.

[0026] The hardware-aware performance simulation algorithm is an event-driven discrete event simulator. It takes an asynchronous execution dependency graph, basic performance data of execution units, and detailed hardware system parameters as input. By dynamically tracking and simulating the occupancy and contention of heterogeneous hardware resources, it extrapolates the execution timeline of the entire artificial intelligence computing task.

[0027] Parallelism and pipelined strategies are core configuration schemes preset by users or systems to guide the decomposition and scheduling of AI computing tasks on heterogeneous hardware for efficient execution. Parallelism is a method to decompose a single computing task in the spatial dimension by utilizing multiple computing devices to work simultaneously; pipelined strategies are a method to achieve task-level parallelism by utilizing multiple computing devices to decompose AI computing tasks in an interleaved manner in the temporal and spatial dimensions. A typical pipelined strategy is a one-forward-one-backward (1F1B) scheduling.

[0028] A dependency template is a predefined, parameterized, structured blueprint or pattern library entry that describes standard execution patterns and their dependencies under a specific parallel or pipeline strategy. The core defining elements of a dependency template include pattern descriptions, parameterized interfaces, and structural definitions.

[0029] The hardware resource status table is a dynamic core data structure used to track the occupancy of all critical physical resources in a heterogeneous computing system in real time during performance simulation.

[0030] Combination Figure 1 As shown in the embodiments of this disclosure, a performance prediction method for artificial intelligence computing tasks is provided, including the following steps: S101 breaks down artificial intelligence computing tasks into multiple execution units and determines the basic performance data of each execution unit on the target hardware.

[0031] Here, the target hardware refers to the specific physical hardware system that actually runs the artificial intelligence computing tasks. Basic performance data includes one or more of the following: execution time of the computing kernel, number of floating-point operations (FLOPs), memory throughput, and latency, effective bandwidth, and data transfer volume of communication operations.

[0032] In practical applications, AI computing tasks are decomposed into multiple execution units. This includes: parsing the computation graph of the AI ​​computing task to obtain the complete operator sequence; and decomposing the complete operator sequence into multiple independent execution units based on parallel strategy configuration files (e.g., data parallelism, number of pipeline stages, tensor partitioning dimension). For example, a standard matrix multiplication operator sequence, under a tensor parallel strategy, can be decomposed into two independent execution units: local block computation and a single All-Gather communication. The basic performance data of each execution unit on the target hardware can be determined through micro-benchmarking.

[0033] By standardizing and structuring the complex and holistic AI computing tasks, the fuzzy task description is transformed into a series of well-defined, measurable execution units directly related to the hardware. Each unit is assigned baseline performance characteristics on the target hardware (such as computation latency and data throughput), thus providing a precise and quantifiable input basis for subsequent modeling and simulation. The abstract task performance problem is transformed into a simulation problem of how to schedule and execute a series of execution units with known performance characteristics under specific constraints.

[0034] S102, construct an asynchronous execution dependency graph based on the asynchronous dependency relationships between execution units; wherein, the vertices of the asynchronous execution dependency graph represent execution units and are labeled with the hardware resource labels required for execution, and the directed edges of the graph represent the asynchronous dependency relationships between execution units and are labeled with the dependency types.

[0035] Optionally, based on the asynchronous dependencies between execution units, an asynchronous execution dependency graph is constructed, including: selecting a target dependency template from a predefined dependency pattern library according to the target parallel strategy and pipeline strategy; and generating an asynchronous execution dependency graph based on the target dependency template, according to the execution units and the hardware resource tags marked by the execution units.

[0036] The typical dependency patterns of various parallel strategies (such as data-parallel AllReduce synchronization, tensor-parallel All-Gather / Reduce-Scatter communication pairs, and pipelined parallel 1F1B spatiotemporal grid) are pre-summarized and solidified into a set of standardized dependency templates, forming a dependency pattern library. When a target parallel strategy and pipelined strategy are specified, the corresponding target dependency template is selected from the predefined dependency pattern library through rule matching. Through parameter binding (such as parallelism and device mapping) and hardware resource tag injection, the abstract nodes in the target dependency template are instantiated into concrete execution units, automatically generating an asynchronous execution dependency graph that conforms to both the strategy semantics and the actual hardware topology.

[0037] In some possible implementations, based on the target dependency template, an asynchronous execution dependency graph is generated according to the execution unit and the hardware resource labels annotated by the execution unit. This includes: using the execution unit as the graph vertex of the asynchronous execution dependency graph and associating the graph vertex with hardware resource labels; establishing directed graph edges between the graph vertices according to the dependency pattern in the target dependency template and labeling the graph directed edges with dependency types; organizing the instantiated graph vertex set and graph directed edge set to construct a directed graph data structure to obtain the asynchronous execution dependency graph.

[0038] In practical applications, based on the operator types and quantities defined in the target dependency template, each execution unit is instantiated as a graph vertex and its hardware resource label is bound. Following the dependency patterns (such as data flow and control flow) encoded within the target dependency template, directed edges with explicit dependency types are established between the corresponding graph vertices. The instantiated graph vertices and directed edges are organized into a complete directed graph data structure, thereby generating an asynchronous execution dependency graph that both carries the parallel strategy logic and is precisely bound to physical hardware resources.

[0039] The following is a specific example illustrating how to construct an asynchronous execution dependency graph: The AI ​​computing task is broken down into multiple execution units: computing unit V1 (GEMM_TP_QKV_GPU0), computing unit V2 (GEMM_TP_QKV_GPU1), and communication unit V3 (ALL_GATHER_ACTIVATION). Three graph vertices are created, corresponding to the three execution units respectively, and hardware resource labels are associated with the graph vertices: V1 (GEMM_TP_QKV_GPU0), Hardware: GPU0; V2 (GEMM_TP_QKV_GPU1), Hardware: GPU1; V3 (ALL_GATHER_ACTIVATION), Hardware: Link_GPU0_GPU1.

[0040] Based on the target parallel strategy and pipeline strategy of "tensor parallelism-column partitioning-forward computation", the target dependency template TP_FWD_COL_SPLIT is selected from the predefined dependency pattern library. The dependency patterns of this target dependency template include: parallel computation → global collection → subsequent computation.

[0041] According to the dependency pattern rules of the TP_FWD_COL_SPLIT template, V3 depends on the output data of V1 and V2 to begin. Based on this, two directed edges are created: E1: V1→V3, labeled as data dependency; E2: V2→V3, labeled as data dependency. {V1, V2, V3} and {E1, E2} are then organized into a directed graph data structure, thus generating the asynchronous execution dependency graph.

[0042] The method of constructing the asynchronous execution dependency graph achieves a precise integration of policy logic and hardware mapping. This allows the generated asynchronous execution dependency graph to not only accurately reflect the computation order of AI computing tasks, but also clearly indicate the physical resources required for each operation, laying the foundation for subsequent high-fidelity hardware-aware simulation.

[0043] S103 takes the asynchronous execution dependency graph, basic performance data, and artificial intelligence hardware parameters as input, performs simulation through a hardware-aware performance simulation algorithm, and outputs performance prediction results.

[0044] Using an asynchronous execution dependency graph as the scheduling blueprint, basic performance data as the time base, and artificial intelligence hardware parameters as the constraint rules, the entire simulation process is driven by an event queue (task start, completion, resource release). Finally, by collecting the completion time points of all execution units, the complete performance prediction results of the AI ​​computing task are deduced.

[0045] Performance prediction results are a comprehensive quantitative evaluation report on the performance of a target AI computing task under specified hardware and parallel strategies, generated by extrapolating the asynchronous execution dependency graph using hardware-aware simulation algorithms. In practical applications, performance prediction results include one or more of the following: overall performance metrics (e.g., total task execution time, training throughput, inference latency), resource utilization metrics (e.g., average utilization of each computing unit, memory bandwidth utilization, communication link load), performance bottleneck location information (e.g., clearly identifying key resources or operations that limit performance, the severity of the bottleneck, and its location in the dependency graph), path analysis (e.g., identifying the task sequence that determines the shortest completion time of the task and its duration), and timeline decomposition (e.g., decomposing the total time into components such as computation time, communication time, and idle waiting time).

[0046] When the target AI computing task is a large language model training task, the overall performance indicators in the performance prediction results can be throughput and resource efficiency; when the target AI computing task is a real-time computer vision inference task, the overall performance indicator in the performance prediction results can be latency.

[0047] The performance prediction method for artificial intelligence computing tasks provided in this disclosure decomposes the AI ​​computing task into multiple execution units. An asynchronous execution dependency graph is formed by constructing vertices labeled with hardware resources and directed edges labeled with dependency types. This accurately depicts the complex computation, communication, and synchronization relationships between execution units under parallel modes such as data, tensors, and pipelines. Furthermore, a hardware-aware performance simulation algorithm combines the asynchronous execution dependency graph, execution unit performance, and hardware parameters to dynamically simulate the scheduling, competition, and execution processes of each execution unit under limited hardware resource constraints, achieving fine-grained extrapolation of the end-to-end execution behavior of the task. By synchronously modeling asynchronous dependencies and hardware resource constraints, the method effectively overcomes the shortcomings of traditional linear superposition methods that ignore parallel overlap and resource competition, significantly improving the accuracy of performance prediction for AI computing tasks. Simultaneously, the graph traversal-based performance simulation algorithm maintains high efficiency while ensuring prediction accuracy and can accurately identify and locate performance bottlenecks caused by specific resource saturation.

[0048] In some embodiments, the asynchronous execution dependency graph, basic performance data, and AI hardware parameters are taken as input, and a hardware-aware performance simulation algorithm is used to simulate the process and output a performance prediction result. This includes: initializing a hardware resource state table, a queue to be executed, a queue to be executed, and a cumulative time; adding graph vertices without prerequisite dependencies in the asynchronous execution dependency graph to the queue to be executed, and updating the hardware resource state table to an occupied state; iteratively simulating the queue to be executed based on the basic performance data and AI hardware parameters; and outputting a performance prediction result based on the cumulative time of the iterative simulation when both the queue to be executed and the queue to be executed are empty.

[0049] This method simulates the execution of AI computing tasks on heterogeneous hardware through event-driven discrete event simulation. An asynchronous execution dependency graph serves as the logical blueprint for task execution, while a hardware resource status table tracks the real-time occupancy of physical resources as the core constraint. Task states are managed by maintaining queues for tasks to be executed and queues for tasks in progress. In each iteration, the task's dependency readiness and hardware resource availability are rigorously checked. Tasks are only scheduled for execution from the queues when both conditions are met. The precise execution time is calculated based on fundamental performance data and AI hardware parameters, while the hardware resource status and cumulative time are dynamically updated. This allows for a high-fidelity reproduction of the various latency effects of AI computing tasks in a real hardware environment caused by dependency waiting and resource contention on a virtual timeline.

[0050] It enables refined and dynamic simulation of the execution process of AI computing tasks in heterogeneous parallel environments, significantly improving the accuracy and reliability of AI computing task performance prediction. Furthermore, based on queue scheduling and resource status tracking mechanisms, it not only operates efficiently but also possesses powerful bottleneck diagnosis capabilities, clearly identifying whether performance limitations are due to idle GPUs, saturated communication links, or insufficient memory bandwidth, providing direct and accurate evidence for system optimization.

[0051] Optionally, based on basic performance data and AI hardware parameters, iterative simulation is performed on the queue to be executed, including: selecting graph vertices from the queue to be executed that meet the dependency conditions and whose hardware resources corresponding to the labeled hardware resource tags are all in an idle state at the current simulation time, as the vertices of the graph to be executed; calculating the estimated execution time when the vertices of the graph to be executed occupy hardware resources according to the basic performance data and AI hardware parameters; allocating execution time periods to the vertices of the graph to be executed according to the estimated execution time, and updating the hardware resource status table to the occupied state; releasing the hardware resources after the simulated execution of the vertices to be executed is completed, and adding the successor vertices of the vertices of the graph to be executed in the asynchronous execution dependency graph to the queue to be executed.

[0052] When selecting graph vertices to be executed from the queue at each simulation time step, two rigid conditions must be met simultaneously: first, all logical predecessors of the graph vertex have been completed (dependencies satisfied); second, all physical resources corresponding to its labeled hardware resource tags are idle at the current time (resources available). For graph vertices that meet the conditions, the execution time under competitive conditions is estimated by combining their basic performance data with current hardware parameters (such as actual available bandwidth and computing unit power), and a continuous execution period is allocated to them on the simulation timeline. Simultaneously, the hardware resource status table is updated to lock hardware resources, thus accurately simulating the entire process of task scheduling, execution, and resource occupancy. When the task simulation is completed, the released resources immediately trigger a new round of scheduling checks, forming a closed-loop event-driven progression chain. Figure 5B As shown, first, vertex a is selected and added to the execution queue; after vertex a is executed, the execution queue W = {b, c}; after vertex c is executed, the execution queue W = {e}; after vertex b is executed, the execution queue W = {e, d}; after vertex e is executed, the execution queue W = {b, c}. .

[0053] By explicitly modeling resource contention and queuing, performance prediction results can accurately reflect latency caused by communication link congestion, insufficient memory bandwidth, etc., thus improving the accuracy of performance prediction. Simultaneously, each resource wait or dependency blockage recorded during the simulation can be directly associated with a specific graph vertex and hardware resource, thereby precisely revealing the root cause of system bottlenecks and providing powerful real-time diagnostic and bottleneck location capabilities.

[0054] In some possible implementations, when the vertex of the graph to be executed is a vertex of a computational class graph, the estimated execution time of the vertex occupying hardware resources is calculated based on basic performance data and artificial intelligence hardware parameters. This includes: obtaining the number of floating-point operations and data access volume corresponding to the vertex of the computational class graph in the basic performance data; determining the first duration of computational limitation based on the number of floating-point operations based on the peak computing power of the computing unit identified by the hardware resource tag in the artificial intelligence hardware parameters; determining the second duration of memory access limitation based on the data access volume based on the effective bandwidth of the memory system associated with the hardware resource tag in the artificial intelligence hardware parameters; and determining the larger value between the first duration and the second duration as the estimated execution time of the vertex of the computational class graph.

[0055] In some practical applications, the first duration is calculated using the following formula: in, For the first duration, To calculate the total number of floating-point operations required to complete the vertices of the class graph, The peak computing power of the computing unit. To calculate the unit utilization factor ( ), reflecting instruction-level parallelism, This is a multiprocessor utilization adjustment factor that reflects the impact of thread block size on parallelism. To calculate startup and synchronization overhead, including kernel startup latency and stream synchronization wait time.

[0056] In some practical applications, the second duration is calculated using the following formula: in, For the second duration, To calculate the total amount of memory data that the vertices of the class graph need to access, For the effective bandwidth of the memory system, This is a memory access parallelism factor, reflecting the performance of concurrent access by multiple memory controllers. To calculate the memory overlap factor, which reflects the proportion of overlap between computation and memory access, There is a fixed overhead for memory operations, including address translation and cache line alignment penalty.

[0057] By using the calculation formulas for the first and second durations mentioned above, and by taking into account micro-architectural characteristics such as computing unit utilization, memory access parallelism, and caching effects, the execution behavior of actual hardware can be more accurately reflected.

[0058] By simultaneously considering computation and memory access, the system avoids overly optimistic estimates based solely on peak computing power or blind estimations based solely on empirical coefficients, significantly improving the prediction accuracy of single-node computation time. Furthermore, if the first duration is longer than the second duration, it indicates that the computation unit is "computation-bound," and if the second duration is longer than the first duration, it indicates that the computation unit is "memory-bound." This allows for the allocation of appropriate hardware resources to different types of computation units, assigning "computation-bound" tasks to hardware devices with higher peak computing power and "memory-bound" tasks to hardware devices with greater memory bandwidth.

[0059] Combination Figure 2 As shown in the embodiments of this disclosure, a performance prediction method for artificial intelligence computing tasks is provided, including the following steps: S201 breaks down artificial intelligence computing tasks into multiple execution units and determines the basic performance data of each execution unit on the target hardware.

[0060] S202, construct an asynchronous execution dependency graph based on the asynchronous dependency relationships between execution units; wherein, the vertices of the asynchronous execution dependency graph represent execution units and are labeled with the hardware resource labels required for execution, and the directed edges of the graph represent the asynchronous dependency relationships between execution units and are labeled with the dependency types.

[0061] S203 initializes the hardware resource status table, pending queue, currently executing queue, and cumulative time.

[0062] S204 adds the vertices in the asynchronous execution dependency graph that have no preceding dependencies to the execution queue and updates the hardware resource status table to the occupied state.

[0063] S205. From the queue of vertices to be executed, select the graph vertices that meet the dependency conditions and whose hardware resources corresponding to the labeled hardware resource tags are all in an idle state at the current simulation time, and use them as the graph vertices to be executed.

[0064] S206, based on basic performance data and artificial intelligence hardware parameters, calculates the estimated execution time when the vertices of the graph to be executed occupy hardware resources.

[0065] S207. Based on the estimated execution time, allocate execution time periods to the vertices of the graph to be executed, and update the hardware resource status table to the occupied status.

[0066] S208: After the simulation of the vertex to be executed is completed, hardware resources are released, and the successor vertex of the vertex to be executed in the asynchronous execution dependency graph is added to the execution queue.

[0067] S209: When both the executing queue and the pending queue are empty, output the performance prediction result based on the cumulative time of the iterative simulation.

[0068] In this embodiment, the AI ​​computing task is deconstructed and modeled as an asynchronous execution dependency graph. Vertices and dependency edges accurately describe the execution logic and timing constraints under different parallel strategies. The asynchronous execution dependency graph serves as the scheduling basis, and the hardware resource status table serves as the constraint condition. A simulation clock is dynamically advanced through a maintenance queue system. Dependency satisfaction and resource availability are rigorously checked at each scheduling point, and the precise time consumption under resource constraints is calculated for the task. This simulation reproduces the complete process of parallel task execution, resource contention, and resulting blocking and waiting in a real hardware environment, achieving event-driven hardware resource contention simulation. By combining dependency graph modeling and resource contention simulation, the shortcomings of traditional methods in ignoring execution overlap and resource contention are fundamentally overcome, enabling high-precision performance prediction of artificial intelligence computing tasks.

[0069] In some embodiments, the performance prediction method for artificial intelligence computing tasks further includes: adjusting the configuration parameters of the target parallel strategy and pipeline strategy based on the performance bottleneck location information in the performance prediction results to generate a new strategy configuration; performing local reconstruction of the subgraph structure in the asynchronous execution dependency graph that is directly affected by the parameter adjustment according to the new strategy configuration to obtain an updated asynchronous execution dependency graph; and performing incremental local re-simulation based on the updated asynchronous execution dependency graph to obtain an optimized performance prediction result.

[0070] Here, performance bottleneck location information is identified and extracted after the hardware-aware performance simulation is completed. This is achieved through in-depth analysis of the complete execution timeline and historical resource status data generated during the simulation, revealing specific constraints and their quantitative descriptions that cause the overall task performance to fall short of expectations. For example, performance bottleneck location information may include saturation of specific communication links or low device utilization. The subgraph structure directly affected by parameter adjustments in the asynchronous execution dependency graph is the graph structure formed by reversely searching and identifying all affected graph vertices that depend on the changed parameters; or, it can be the graph structure formed by expanding outwards from the affected graph vertices as the core.

[0071] In practical applications, the configuration parameters of the target parallelism strategy and the pipeline strategy are adjusted to generate a new strategy configuration. For example, the original strategy configuration of the target parallelism strategy and the pipeline strategy is {data parallelism: 4}, and the new strategy configuration is {data parallelism: 2, model parallelism: 2}.

[0072] By leveraging the bottleneck location information revealed by the initial performance prediction results, the source strategy parameters that cause the bottleneck are derived and adjusted in reverse (such as reducing data parallelism and changing the pipeline split point). Then, through differential analysis, the local subgraph structure affected by the strategy change in the asynchronous execution dependency graph is accurately located, and only this part is reconstructed and updated. Finally, incremental local resimulation is performed starting from the change point, thereby quickly obtaining the performance prediction results under the new strategy, realizing the targeted exploration and efficient evaluation of the strategy space.

[0073] Incremental local resimulation avoids the enormous overhead of full-scale resimulation, making it possible to evaluate multiple optimization schemes within minutes or even seconds, thus improving the efficiency and agility of system optimization. Simultaneously, each strategy adjustment directly addresses specific bottleneck diagnoses, allowing for clear attribution of performance gains or losses after optimization, providing explicit and reliable decision-making basis, and ensuring the accuracy and interpretability of the optimization.

[0074] Optionally, based on the new strategy configuration, the subgraph structure directly affected by parameter adjustments in the asynchronous execution dependency graph is partially reconstructed to obtain an updated asynchronous execution dependency graph. This includes: generating a target subgraph that conforms to the dependency pattern in the target dependency template based on the new strategy configuration; removing the original subgraph corresponding to the target subgraph in the asynchronous execution dependency graph; and connecting the target subgraph to the asynchronous execution dependency graph to obtain an updated asynchronous execution dependency graph.

[0075] In practical applications, based on the new strategy configuration, the corresponding target dependency template is invoked to re-instantiate and generate a new target subgraph. From the original asynchronous execution dependency graph, the affected original subgraphs are removed, and the newly generated target subgraph is precisely aligned and connected with the interface vertices of the original subgraphs, with its input / output vertices precisely aligned with and connected to the interface vertices of the original subgraphs.

[0076] The asynchronous execution dependency graph is viewed as a combination of multiple subgraph modules generated according to a policy template. When the policy configuration changes, the affected atomic graph modules are located through difference analysis, and a new target subgraph with compatible interfaces is re-instantiated based on the new configuration parameters. Subsequently, the atomic graph is removed, and the target subgraph is seamlessly integrated into the original graph with the same input / output interfaces. This ensures that only local updates are performed while maintaining the semantic consistency of global data flow and control flow. This greatly improves the efficiency of optimization iteration, avoids the huge overhead of full graph reconstruction and full simulation, and makes policy evaluation at the second / minute level possible. In addition, it ensures the reliability and accuracy of the optimization process. Since the old and new subgraphs follow the same template specification and have consistent interfaces, local replacement will not introduce inconsistencies or break global dependencies, making the results of incremental simulation equivalent to those of full reconstruction simulation.

[0077] Combination Figure 3 As shown in the embodiments of this disclosure, a performance prediction method for artificial intelligence computing tasks is provided, including the following steps: S301 breaks down artificial intelligence computing tasks into multiple execution units and determines the basic performance data of each execution unit on the target hardware.

[0078] S302, based on the asynchronous dependency relationship between execution units, construct an asynchronous execution dependency graph; wherein, the vertices of the asynchronous execution dependency graph represent execution units and are labeled with the hardware resource labels required for execution, and the directed edges of the graph represent the asynchronous dependency relationship between execution units and are labeled with the dependency type.

[0079] S303 takes asynchronous execution dependency graph, basic performance data, and artificial intelligence hardware parameters as input, performs simulation through hardware-aware performance simulation algorithm, and outputs performance prediction results.

[0080] S304, based on the performance bottleneck location information in the performance prediction results, adjusts the configuration parameters of the target parallel strategy and pipeline strategy to generate a new strategy configuration.

[0081] S305, based on the new strategy configuration, locally reconstructs the subgraph structure in the asynchronous execution dependency graph that is directly affected by parameter adjustments, and obtains the updated asynchronous execution dependency graph.

[0082] S306 performs incremental local resimulation based on the updated asynchronous execution dependency graph to obtain optimized performance prediction results.

[0083] In this embodiment, an asynchronous execution dependency graph is used to perform topological abstraction of task logic and asynchronous dependencies. Then, hardware-aware simulation is used to dynamically combine the logic graph with physical hardware constraints to perform high-fidelity simulation. Bottleneck information from the simulation output drives strategy adjustment, and an innovative incremental graph update and local re-simulation technique is adopted. Only the affected task subgraphs are reconstructed and simulated, thereby achieving rapid evaluation and iteration of optimization strategies. The incremental optimization closed loop avoids the huge overhead of full re-simulation in traditional methods, making it possible to rapidly explore massive strategy combinations. Simultaneously, a complete automated workflow is formed from performance prediction and bottleneck diagnosis to strategy recommendation and effect verification, effectively supporting the rapid design and precise tuning of AI computing systems.

[0084] Combination Figure 4 As shown in the embodiments of this disclosure, a performance prediction method for artificial intelligence computing tasks is provided, including the following steps: S401 breaks down artificial intelligence computing tasks into multiple execution units and determines the basic performance data of each execution unit on the target hardware.

[0085] S402, based on the asynchronous dependency relationship between execution units, construct an asynchronous execution dependency graph; wherein, the vertices of the asynchronous execution dependency graph represent execution units and are labeled with the hardware resource labels required for execution, and the directed edges of the graph represent the asynchronous dependency relationship between execution units and are labeled with the dependency type.

[0086] S403 initializes the hardware resource status table, pending queue, currently executing queue, and cumulative time.

[0087] S404 adds the vertices in the asynchronous execution dependency graph that have no preceding dependencies to the execution queue and updates the hardware resource status table to the occupied state.

[0088] S405: Select from the queue of vertices to be executed the graph vertices that meet the dependency conditions and whose hardware resources corresponding to the labeled hardware resource tags are all in an idle state at the current simulation time.

[0089] S406 calculates the estimated execution time when the vertices of the graph to be executed occupy hardware resources, based on basic performance data and artificial intelligence hardware parameters.

[0090] S407: Based on the estimated execution time, allocate execution time periods to the vertices of the graph to be executed, and update the hardware resource status table to the occupied status.

[0091] S408 releases hardware resources after the simulated execution of the vertex to be executed is completed, and adds the successor vertex of the vertex to be executed in the asynchronous execution dependency graph to the queue to be executed.

[0092] S409, when both the executing queue and the pending queue are empty, outputs the performance prediction result based on the cumulative time of the iterative simulation.

[0093] S410, based on the performance bottleneck location information in the performance prediction results, adjusts the configuration parameters of the target parallel strategy and pipeline strategy to generate a new strategy configuration.

[0094] S411, based on the new strategy configuration, locally reconstruct the subgraph structure in the asynchronous execution dependency graph that is directly affected by parameter adjustments, and obtain the updated asynchronous execution dependency graph.

[0095] S412 performs incremental local resimulation based on the updated asynchronous execution dependency graph to obtain optimized performance prediction results.

[0096] In this embodiment, firstly, the AI ​​computing task is deconstructed into graph vertices carrying hardware labels and directed graph edges labeled with dependency types, constructing an asynchronous execution dependency graph to uniformly represent the computing and communication relationships under different parallel strategies. Secondly, using the asynchronous execution dependency graph as scheduling logic and a resource status table as competition constraints, a queue mechanism is used to dynamically advance the process, simulating the execution and blocking of each unit in a resource-competitive environment. Then, based on the bottleneck location identified in the simulation output, the strategy parameters are adjusted, and only the affected subgraphs in the asynchronous execution dependency graph are partially reconstructed and resimulated, enabling rapid iterative verification of the strategy and forming a closed-loop incremental optimization mechanism. Because it simultaneously depicts asynchronous dependencies and resource competition, it realistically reflects the effects of parallel overlap and blocking, significantly improving prediction accuracy. Furthermore, the incremental local resimulation avoids full graph resimulation, allowing multi-round strategy tuning to be completed in a very short time, greatly improving optimization efficiency.

[0097] Combination Figure 6 As shown in the figure, this disclosure provides a performance prediction device 600 for artificial intelligence computing tasks, including a decomposition module 610, a construction module 620, and a simulation module 630. The decomposition module 610 is configured to decompose the artificial intelligence computing task into multiple execution units and determine the basic performance data of each execution unit on the target hardware. The construction module 620 is communicatively connected to the decomposition module 610 and is configured to construct an asynchronous execution dependency graph based on the asynchronous dependencies between execution units. The vertices of the asynchronous execution dependency graph represent execution units and are labeled with the hardware resources required for execution; the directed edges of the graph represent the asynchronous dependencies between execution units and are labeled with the dependency type. The simulation module 630 is communicatively connected to the construction module 620 and is configured to take the asynchronous execution dependency graph, the basic performance data, and the artificial intelligence hardware parameters as input, perform simulation using a hardware-aware performance simulation algorithm, and output the performance prediction result.

[0098] The performance prediction device for artificial intelligence computing tasks provided in this disclosure decomposes the AI ​​computing task into multiple execution units. By constructing vertices labeled with hardware resources and directed edges labeled with dependency types, an asynchronous execution dependency graph is formed. This accurately depicts the complex computation, communication, and synchronization relationships between execution units under parallel modes such as data, tensors, and pipelines. Furthermore, a hardware-aware performance simulation algorithm combines the asynchronous execution dependency graph, execution unit performance, and hardware parameters to dynamically simulate the scheduling, competition, and execution processes of each execution unit under limited hardware resource constraints, achieving fine-grained extrapolation of the end-to-end execution behavior of the task. By synchronously modeling asynchronous dependencies and hardware resource constraints, the shortcomings of traditional linear superposition methods in ignoring parallel overlap and resource competition are effectively overcome, significantly improving the accuracy of performance prediction for AI computing tasks. Simultaneously, the graph traversal-based performance simulation algorithm maintains high efficiency while ensuring prediction accuracy and can accurately identify and locate performance bottlenecks caused by specific resource saturation.

[0099] Combination Figure 7 As shown, this embodiment of the disclosure provides an electronic device (e.g., a computer, controller, etc.) 700, which includes a memory 701, a processor 702, a communication interface 703, and a bus 704. The memory 701, processor 702, and communication interface 703 are interconnected via the bus 704.

[0100] The memory 701 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).

[0101] The memory 701 can store a program. When the program stored in the memory 701 is executed by the processor 702, the processor 702 and the communication interface 703 are used to execute the various steps of the performance prediction method for artificial intelligence computing tasks according to the embodiments of this application.

[0102] Processor 702 is a circuit with signal processing capabilities. In one implementation, processor 702 can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, processor 702 can implement certain functions through the logical relationships of hardware circuits. These logical relationships of hardware circuits are fixed or reconfigurable. For example, processor 702 can be a hardware circuit implemented as an ASIC or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a type of ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), or deep learning processing unit (DPU). The processor 702 is used to execute related programs to implement the functions required by the units in the performance prediction apparatus for artificial intelligence computing tasks in the embodiments of this application, or to execute the performance prediction method for artificial intelligence computing tasks in the method embodiments of this application.

[0103] As can be seen, each module in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor types.

[0104] Furthermore, the modules in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these modules are integrated together as a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or for implementing the functions of the modules of the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0105] The communication interface 703 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the device 700 and other devices or communication networks. For example, data can be acquired through the communication interface 703.

[0106] Bus 704 may include a pathway for transmitting information between various components of device 700 (e.g., memory 701, processor 702, communication interface 703).

[0107] It should be noted that, although Figure 7 The performance prediction device 700 for artificial intelligence computing tasks shown only illustrates the memory, processor, and communication interface. However, those skilled in the art should understand that in specific implementations, the device 700 may also include other components necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that the device 700 may also include hardware components for implementing other additional functions. Moreover, those skilled in the art should understand that the device 700 may only include the components necessary for implementing the embodiments of this application, and may not necessarily include... Figure 7 All the devices shown.

[0108] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of any of the above methods.

[0109] This application also provides a computer program product containing instructions. When the computer program product is run on a computer or processor, it causes the computer or processor to perform one or more steps of any of the methods described above.

[0110] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0111] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0112] The foregoing description and accompanying drawings fully illustrate embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included or substituted for parts and features of other embodiments. The scope of the embodiments of this disclosure includes the entire scope of the claims and all available equivalents of the claims. While the terms “first,” “second,” etc., may be used in this application to describe elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be called a second element without changing the meaning of the description, and similarly, a second element may be called a first element, provided that all occurrences of “first element” are consistently renamed and all occurrences of “second element” are consistently renamed. First and second elements are both elements, but may not be the same element. Moreover, the terminology used in this application is only for describing embodiments and is not intended to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. Similarly, the term “and / or” as used herein means including one or more of the associated listed any and all possible combinations. Additionally, when used herein, the terms “comprise” and its variations “comprises” and / or “comprising” refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase “comprising an…” does not exclude the presence of additional identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0113] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0114] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A performance prediction method for artificial intelligence computing tasks, characterized by, include: The artificial intelligence computing task is broken down into multiple execution units, and the basic performance data of each execution unit on the target hardware is determined. Based on the asynchronous dependencies between execution units, an asynchronous execution dependency graph is constructed; wherein, the vertices of the asynchronous execution dependency graph represent execution units and are labeled with the hardware resource tags required for execution, and the directed edges of the graph represent the asynchronous dependencies between execution units and are labeled with the dependency type; The asynchronous execution dependency graph, basic performance data, and artificial intelligence hardware parameters are taken as input, and the hardware-aware performance simulation algorithm is used to simulate the performance and output the performance prediction results.

2. The performance prediction method according to claim 1, characterized in that, Based on the asynchronous dependencies between execution units, an asynchronous execution dependency graph is constructed, including: Based on the target parallel strategy and pipeline strategy, select the target dependency template from the predefined dependency pattern library; Based on the target dependency template, an asynchronous execution dependency graph is generated according to the execution unit and the hardware resource tags marked by the execution unit.

3. The performance prediction method according to claim 2, characterized in that, Based on the target dependency template, an asynchronous execution dependency graph is generated according to the execution unit and the hardware resource tags annotated by the execution unit, including: The execution unit is treated as a vertex in the asynchronous execution dependency graph, and hardware resource labels are associated with the graph vertices; Based on the dependency pattern in the target dependency template, establish directed edges between graph vertices and label the directed edges with dependency types. The instantiated graph vertex set and directed edge set are organized to construct a directed graph data structure, thus obtaining an asynchronous execution dependency graph.

4. The performance prediction method according to claim 1, characterized in that, Taking asynchronous execution dependency graphs, basic performance data, and AI hardware parameters as input, a hardware-aware performance simulation algorithm is used to simulate the performance and output performance prediction results, including: Initialize the hardware resource status table, the queue of pending executions, the queue of currently executing tasks, and the cumulative time. Add the vertices in the asynchronous execution dependency graph that have no preceding dependencies to the execution queue, and update the hardware resource status table to the occupied state; Based on fundamental performance data and AI hardware parameters, iterative simulations are performed on the queue to be executed. With both the executing queue and the pending queue empty, the performance prediction results are output based on the cumulative time of the iterative simulation.

5. The performance prediction method according to claim 4, characterized in that, Based on fundamental performance data and AI hardware parameters, iterative simulations are performed on the queue to be executed, including: From the queue of vertices to be executed, select the graph vertices that meet the dependency conditions and whose hardware resources corresponding to the labeled hardware resource tags are all in an idle state at the current simulation time, and use them as the vertices of the graph to be executed. Based on basic performance data and AI hardware parameters, calculate the estimated execution time when the vertices of the graph to be executed occupy hardware resources; Based on the estimated execution time, allocate execution time periods to the vertices of the graph to be executed, and update the hardware resource status table to the occupied status; After the simulation of the vertex to be executed is completed, hardware resources are released, and the successor vertices of the vertices to be executed in the asynchronous execution dependency graph are added to the execution queue.

6. The performance prediction method according to claim 5, characterized in that, When the vertices of the graph to be executed are vertices of a computational class graph, the estimated execution time when the vertices of the graph to be executed occupy hardware resources is calculated based on basic performance data and artificial intelligence hardware parameters, including: Obtain the number of floating-point operations and data accesses corresponding to the vertices of the computation class graph from the basic performance data; Based on the peak computing power of the computing unit identified by the hardware resource tag in the artificial intelligence hardware parameters, the first duration of computing limitation is determined according to the number of floating-point operations. Based on the effective bandwidth of the memory system associated with hardware resource tags in the artificial intelligence hardware parameters, the second duration of memory access restriction is determined according to the data access volume. The larger of the first and second durations is determined as the estimated execution time for calculating the vertices of the class graph.

7. The performance prediction method according to any one of claims 1 to 6, characterized in that, Also includes: Based on the performance bottleneck location information in the performance prediction results, adjust the configuration parameters of the target parallel strategy and pipeline strategy to generate a new strategy configuration. Based on the new strategy configuration, the subgraph structure of the asynchronous execution dependency graph that is directly affected by parameter adjustments is partially reconstructed to obtain the updated asynchronous execution dependency graph; Incremental local resimulation is performed based on the updated asynchronous execution dependency graph to obtain optimized performance prediction results.

8. The performance prediction method according to claim 7, characterized in that, Based on the new strategy configuration, the subgraph structure directly affected by parameter adjustments in the asynchronous execution dependency graph is partially reconstructed to obtain the updated asynchronous execution dependency graph, including: Based on the new strategy configuration, generate a target subgraph that conforms to the dependency pattern in the target dependency template; Remove the original subgraph corresponding to the target subgraph in the asynchronous execution dependency graph, and connect the target subgraph to the asynchronous execution dependency graph to obtain the updated asynchronous execution dependency graph.

9. A performance prediction system for artificial intelligence computing tasks, comprising: The disassembly module is configured to break down artificial intelligence computing tasks into multiple execution units and determine the basic performance data of each execution unit on the target hardware. The building module, which communicates with the disassembly module, is configured to construct an asynchronous execution dependency graph based on the asynchronous dependencies between execution units. The vertices of the asynchronous execution dependency graph represent execution units and are labeled with the hardware resources required for execution. The directed edges of the graph represent the asynchronous dependencies between execution units and are labeled with the dependency types. The simulation module, which communicates with the building module, is configured to take asynchronous execution dependency graphs, basic performance data, and artificial intelligence hardware parameters as inputs, simulate them using hardware-aware performance simulation algorithms, and output performance prediction results.

10. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the performance prediction method for artificial intelligence computing tasks as described in any one of claims 1 to 8.

Images