Npu performance evaluation method, device and equipment under soc multi-channel and medium

Abstract

A SoC multi-channel NPU performance evaluation method, device, equipment and medium, including dividing the model graph corresponding to the target neural network model to be run on the NPU to be evaluated to obtain a plurality of network model task segments; the execution process of each network model task segment is decomposed to obtain a plurality of execution stages with pipelining characteristics; the control rules of each execution stage are determined based on the preset attribute parameters of the network model task segment; the resource competition and concurrent scheduling rules between different execution stages are determined according to the key performance parameters of the NPU to be evaluated; the target neural network model is simulated and run based on the network model task segment and the execution stage through the above rules; the performance evaluation result of the NPU to be evaluated is determined based on the simulation running result, the key performance parameters and the SoC environment parameters carried by the NPU to be evaluated, so as to effectively improve the efficiency and performance fidelity of NPU performance evaluation under SoC multi-channel.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence performance evaluation technology, specifically to a method, apparatus, device, and medium for evaluating the performance of an NPU (Neural Processing Unit) under a multi-path SoC (System on Chip). Background Technology

[0002] In the SoC design verification and performance analysis phase, there are generally two technical approaches for NPU performance evaluation: one is to directly use the NPU RTL (Register Transfer Level) model, synthesis netlist, or hardware simulator / accelerator environment for performance analysis. This approach can realistically reflect hardware behavior, but in complex SoC scenarios, as the model size increases, the business graph becomes more complex, and the concurrency scenarios increase, the simulation speed drops significantly, and the consumption of simulation and debugging resources is large, making it difficult to support large-scale regression and rapid performance iteration; the other approach is to use a pure software functional model or idealized behavioral model for evaluation. This approach is faster, but it usually focuses more on whether the functional path is open, while it does not adequately characterize the key data paths inside the NPU, the pipeline stage dependencies, and the overlapping execution relationships between multiple task segments, making it difficult to accurately represent the real performance bottlenecks in complex SoC scenarios.

[0003] For complex applications such as autonomous driving, visual perception, and large model inference, the network to be evaluated usually contains a large number of layers and operators, and it does not run in isolation in the SoC, but works together with DDR (Double Data Rate) access, on-chip interconnects, other processing units, and software scheduling. It is evident that the two conventional technical approaches mentioned above cannot effectively achieve NPU performance evaluation in complex multi-path scenarios of SoC. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for evaluating the performance of NPU under SoC multi-path conditions, which can effectively improve the efficiency and performance fidelity of NPU performance evaluation under SoC multi-path conditions.

[0005] In a first aspect, embodiments of this application provide a method for evaluating NPU performance under SoC multi-path conditions, including: The model graph corresponding to the target neural network model that needs to run on the NPU to be evaluated is divided into multiple network model task segments. The execution process of each network model task segment is decomposed to obtain multiple execution stages with pipeline characteristics; The control rules for each execution stage are determined based on the preset attribute parameters of the task segments in the network model. Determine the resource contention and concurrent scheduling rules between different execution stages based on the key performance parameters corresponding to the NPU to be evaluated; The target neural network model is controlled by resource contention and concurrent scheduling rules and control rules, and simulation operation is performed based on the network model task segments and execution phases. The performance evaluation results of the NPU to be evaluated are determined based on the simulation results, key performance parameters, and environmental parameters corresponding to the SoC on which the NPU is mounted.

[0006] In conjunction with the first aspect, in one implementation, the multiple execution stages with flow characteristics are, in sequence, a coefficient loading stage, an input data transport stage, a calculation execution stage, and an output data transport stage. The coefficient loading stage is used to characterize the behavior of loading the weight parameters required for the network model task segment from external memory into the NPU internal coefficient buffer. The input data transfer stage is used to characterize the behavior of transferring the data required for the network model task segment from external memory to the NPU input buffer; The computation execution phase is used to characterize the core computational operations corresponding to the task segments of the network model. The output data transfer stage is used to characterize the behavior of writing the output of the network model task segment from the NPU output buffer back to external memory.

[0007] In conjunction with the first aspect, in one implementation, the resource contention concurrency scheduling rules include read bandwidth contention, write bandwidth contention, on-chip interconnect bandwidth occupancy, cache occupancy, storage channel occupancy, mutual exclusion and overlap between different execution stages, and a concurrency scheduling mechanism.

[0008] In conjunction with the first aspect, in one implementation, the control rules include startup conditions, input-output relationships, duration calculation rules, resource occupancy types, concurrency levels, and state transition rules.

[0009] In conjunction with the first aspect, in one implementation, the step of dividing the model graph corresponding to the target neural network model to be run on the NPU to be evaluated into multiple network model task segments includes: Based on a preset decomposition strategy, the model graph corresponding to the target neural network model to be run on the NPU to be evaluated is divided into multiple network model task segments. The preset decomposition strategy includes layer structure, operator type, data dependency relationship and data block size.

[0010] Secondly, embodiments of this application provide an NPU performance evaluation device under SoC multi-path support, comprising: The task segmentation module is used to divide the model graph corresponding to the target neural network model that needs to run on the NPU to be evaluated, and obtain multiple network model task segments. The stage decomposition module is used to decompose the execution process of each network model task segment to obtain multiple execution stages with pipeline characteristics. The rule determination module is used to determine the control rules for each execution stage based on the preset attribute parameters of the task segment of the network model; and to determine the resource contention and concurrent scheduling rules between different execution stages based on the key performance parameters corresponding to the NPU to be evaluated. The simulation control module is used to control the target neural network model to perform simulation operation based on the network model task segments and execution phases through resource contention and concurrent scheduling rules and control rules; The performance evaluation module is used to determine the performance evaluation result of the NPU to be evaluated based on the simulation results, key performance parameters, and environmental parameters corresponding to the SoC on which the NPU to be evaluated is mounted.

[0011] In conjunction with the second aspect, in one implementation, the multiple execution stages with flow characteristics are, in sequence, a coefficient loading stage, an input data transport stage, a calculation execution stage, and an output data transport stage. The coefficient loading stage is used to characterize the behavior of loading the weight parameters required for the network model task segment from external memory into the NPU internal coefficient buffer. The input data transfer stage is used to characterize the behavior of transferring the data required for the network model task segment from external memory to the NPU input buffer; The computation execution phase is used to characterize the core computational operations corresponding to the task segments of the network model. The output data transfer stage is used to characterize the behavior of writing the output of the network model task segment from the NPU output buffer back to external memory.

[0012] In conjunction with the second aspect, in one implementation, the resource contention concurrency scheduling rules include read bandwidth contention, write bandwidth contention, on-chip interconnect bandwidth occupancy, cache occupancy, storage channel occupancy, mutual exclusion and overlap between different execution stages, and a concurrency scheduling mechanism.

[0013] In conjunction with the second aspect, in one implementation, the control rules include startup conditions, input-output relationships, duration calculation rules, resource occupancy types, concurrency levels, and state transition rules.

[0014] In conjunction with the second aspect, in one implementation, the task segmentation module is specifically used for: Based on a preset decomposition strategy, the model graph corresponding to the target neural network model to be run on the NPU to be evaluated is divided into multiple network model task segments. The preset decomposition strategy includes layer structure, operator type, data dependency relationship and data block size.

[0015] Thirdly, embodiments of this application provide a SoC multi-path NPU performance evaluation device, which includes a processor, a memory, and an SoC multi-path NPU performance evaluation program stored in the memory and executable by the processor. When the SoC multi-path NPU performance evaluation program is executed by the processor, it implements the steps of the SoC multi-path NPU performance evaluation method described above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a SoC multi-path NPU performance evaluation program, wherein when the SoC multi-path NPU performance evaluation program is executed by a processor, it implements the steps of the aforementioned SoC multi-path NPU performance evaluation method.

[0017] The beneficial effects of the technical solutions provided in this application include: The model graph corresponding to the target neural network model to be run on the NPU to be evaluated is divided into multiple network model task segments. State tracking and time progression are performed on a segment-by-segment basis, allowing for macro-level control of task progress while preserving key performance characteristics at the micro-level. The execution process of each network model task segment is decomposed into multiple execution stages with pipelined characteristics. Control rules for each execution stage are determined based on preset attribute parameters of the network model task segments to accurately reflect the hardware data path. Simultaneously, resource contention and concurrent scheduling rules between different execution stages are determined according to the key performance parameters corresponding to the NPU to be evaluated. These resource contention and concurrent scheduling rules, along with the control rules, are then used to... The control target neural network model is simulated based on the network model task segments and execution phases to avoid cycle-by-cycle simulation of the complete RTL. While ensuring that key performance behaviors are representative, it significantly improves the efficiency of SoC-level performance evaluation and can better reflect multi-path concurrency, pipeline overlap, and resource contention behaviors in complex SoC scenarios. Compared with pure software ideal models, it has higher performance fidelity. Finally, based on the simulation results, key performance parameters, and environmental parameters corresponding to the SoC on which the NPU to be evaluated is mounted, the performance evaluation results of the NPU to be evaluated are accurately obtained. It can be seen that this application can effectively improve the efficiency and performance fidelity of NPU performance evaluation under SoC multi-path conditions. Attached Figure Description

[0018] Figure 1This is a flowchart illustrating an embodiment of the NPU performance evaluation method under SoC multi-path in this application; Figure 2 This is a schematic diagram of the overall process of the NPU performance evaluation method under SoC multi-path in the embodiment of this application; Figure 3 This is a functional module diagram of an embodiment of the SoC multi-path NPU performance evaluation device of this application; Figure 4 This is a schematic diagram of the hardware structure of the NPU performance evaluation device under SoC multi-path in the embodiments of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0020] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.

[0021] SoC: refers to a system that integrates processor, storage, interconnect, and dedicated accelerator functions onto a single chip.

[0022] NPU: Used to perform AI-related computing tasks such as convolution, matrix operations, activation, and manipulation.

[0023] Segment: A network model task segment formed by dividing the network model graph corresponding to the neural network model to be executed according to strategies such as layers, operators, or data dependencies. It is the basic unit for scheduling and performance simulation in this application.

[0024] COEF: Coefficient Loading Phase, used to load the weight parameters required by the current Segment from external storage such as DDR into the NPU's internal coefficient cache.

[0025] IDMA: Input Data Transfer Stage, used to transfer the input feature data required for the current Segment operation to the NPU input buffer.

[0026] PROC: The computation execution phase, used to execute the core computation operations corresponding to the current Segment.

[0027] ODMA: Output Data Transfer Stage, used to write the current Segment calculation result from the output buffer back to external storage such as DDR.

[0028] DDR (Double Data Rate) memory: It is a typical object of external memory access by the NPU.

[0029] Multi-path: refers to the weight loading path, input transport path, calculation path, and output write-back path involved in the NPU during execution, as well as the concurrency and competition relationships generated by these paths in complex SoC scenarios.

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0031] In a first aspect, embodiments of this application provide a method for evaluating the performance of an NPU under a multi-path SoC.

[0032] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the NPU performance evaluation method for SoC multi-path under this application. Figure 1 As shown, the NPU performance evaluation methods under SoC multi-path conditions include: Step S10: Divide the model graph corresponding to the target neural network model that needs to run on the NPU to be evaluated to obtain multiple network model task segments.

[0033] In this exemplary embodiment, the NPU to be evaluated refers to the NPU object whose performance needs to be evaluated; the target neural network model refers to the neural network model running on the NPU to be evaluated, such as a convolutional neural network or a large language model; the model graph corresponding to the target neural network model is a topological description of the target neural network model, that is, it describes the computational structure and data flow of the network; based on this, see... Figure 2 As shown, this embodiment segments the model graph corresponding to the target neural network model to obtain multiple network model task segments. It can be seen that this embodiment can control the task progress macroscopically and retain key performance characteristics microscopically by tracking the state and advancing the time in units of task segments.

[0034] Furthermore, in one embodiment, the step of dividing the model graph corresponding to the target neural network model to be run on the NPU to be evaluated into multiple network model task segments includes: Based on a preset decomposition strategy, the model graph corresponding to the target neural network model to be run on the NPU to be evaluated is divided into multiple network model task segments. The preset decomposition strategy includes layer structure, operator type, data dependency relationship and data block size.

[0035] As an example, in this embodiment, the preset decomposition strategy includes layer structure, operator type, data dependency, data block size, or other strategies that can achieve task segment decomposition. Based on this, the model graph corresponding to the target neural network model is divided into multiple segments according to the layer structure, operator type, data dependency, data block size, or other preset decomposition strategy. For example, the preset decomposition strategy is "segmentation according to independently schedulable operator combinations". Suppose that when segmenting a sub-network containing "convolution + activation + convolution", the first convolution layer and its subsequent activation operator can be segmented into Segment1, the second convolution layer can be segmented into Segment2, and the post-processing result can be segmented into Segment3.

[0036] Step S20: Decompose the execution process of each network model task segment to obtain multiple execution stages with pipelined characteristics; the multiple execution stages with pipelined characteristics are, in order, the coefficient loading stage, the input data transport stage, the computation execution stage, and the output data transport stage; wherein, the coefficient loading stage is used to characterize the behavior of loading the weight parameters required by the network model task segment from external memory to the NPU internal coefficient buffer; the input data transport stage is used to characterize the behavior of transporting the input data required by the network model task segment from external memory to the NPU input buffer; the computation execution stage is used to characterize the execution of the core computation operation corresponding to the network model task segment; and the output data transport stage is used to characterize the behavior of writing the output result of the network model task segment from the NPU output buffer back to external memory.

[0037] As an example, in this embodiment, the execution process of each segment is decomposed, resulting in four types of execution stage objects with pipelined characteristics: the coefficient loading stage (COEF), the input data transport stage (IDMA), the computation execution stage (PROC), and the output data transport stage (ODMA); wherein, see Figure 2 As shown, the COEF phase is used to model the weight loading behavior (i.e., logical weight prefetching behavior), which is used to model the behavior of loading the weight parameters required by the current segment from external storage such as DDR to the NPU coefficient buffer; the IDMA phase is used to model the input data moving behavior, which is used to model the behavior of moving the input data required by the current segment from external storage such as DDR to the NPU input buffer; the PROC phase is used to model the core computation behavior, which is used to model the execution behavior of the current segment on the core computation array of the NPU; and the ODMA phase is used to model the output result write-back behavior, which is used to model the behavior of writing the output result of the current segment from the NPU output buffer back to external storage such as DDR.

[0038] Step S30: Determine the control rules for each execution stage based on the preset attribute parameters of the task segment of the network model; wherein, the control rules include the start conditions, input-output relationship, duration calculation rules, resource type, concurrency level, and state transition rules.

[0039] As an example, in this embodiment, the preset attribute parameters corresponding to each segment include, but are not limited to, the input data size, weight size, output data size, computational load, priority, required resource type, and dependencies (such as the predecessor segment identifier, the successor segment identifier, whether DDR needs to be written back, whether it is allowed to overlap with other segments at different stages, etc.).

[0040] Furthermore, taking the aforementioned Segment1 as an example, if it corresponds to a 3×3 convolution plus activation operation, the input feature map size is 56×56×64, the weight tensor size is 3×3×64×128, and the output feature map size is 56×56×128, then the input data scale of Segment1 is determined by the input feature map size and data precision, the weight scale is determined by the weight tensor size and weight quantization bit width, the output data scale is determined by the output feature map size and output data precision, the computational cost is determined by the output feature map size, convolution kernel size, number of input channels, and number of output channels, the predecessor dependency is that the output data of the previous Segment is valid, and the successor dependency is that the output result of the current Segment can be consumed by the subsequent Segment.

[0041] It is worth noting that the specific application relationships of the above-mentioned Segment attribute parameters in subsequent embodiments include the following: the input data scale is used to calculate the duration of the IDMA stage and to determine whether the input buffer is sufficient to hold the input data of this Segment; the weight scale is used to calculate the duration of the COEF stage and to determine whether the coefficient buffer can hold the weight parameters of this Segment; the output data scale is used to calculate the duration of the ODMA stage and to determine whether the output buffer meets the write-back requirement; the computational cost is used to calculate the duration of the PROC stage; and the preceding and following dependencies are used to determine the startable time of the current Segment, the ready time of the subsequent Segment, and whether multiple Segments are allowed to form pipeline overlap, etc.

[0042] Based on this, control rules for each type of stage object can be constructed using the attribute parameters of the aforementioned Segment. These control rules include startup conditions, input-output relationships, duration calculation rules, resource types, concurrency levels, and state transition rules. For example, taking the aforementioned Segment1 as an example, for the COEF stage, its startup conditions are: Segment1 has entered the executable state, the remaining space in the coefficient buffer is not less than the weight size of Segment1, the read DMA channel is available, and the DDR read bandwidth and on-chip interconnect bandwidth allow for this weight transfer; its input is the weight size parameter; its output is the "weight ready" state; its duration is determined by the weight size and the effective weight loading bandwidth; its occupied resources include at least DDR read bandwidth, on-chip interconnect read bandwidth, read DMA channel, and coefficient buffer space; after it completes the relevant operations, the weight ready flag of Segment1 is set.

[0043] For the IDMA stage, the start conditions are as follows: the input data of Segment1 has been generated by the predecessor or is valid in DDR; the remaining space of the input buffer is sufficient to accommodate the input data required by Segment1; the read DMA channel is available and the DDR read bandwidth and on-chip interconnect bandwidth allow this data transfer; its input is the input data size parameter; its output is the "input data ready" state; its duration is determined by the input data size and the valid input transfer bandwidth; its occupied resources include at least DDR read bandwidth, on-chip interconnect read bandwidth, read DMA channel, and input buffer space; after it completes the relevant operations, the input ready flag of Segment1 is set.

[0044] For the PROC phase, the startup conditions are as follows: the weight ready flag of Segment1 is set, the input ready flag is set, the computation core is available, and the internal intermediate buffer is allocable; its inputs are computational parameters and effective computational throughput parameters; its output is the "computation result has been generated" state; its duration is determined by both the computational quantity and the effective computational throughput; its resources include at least the computation core and the internal intermediate buffer; after it completes the relevant operations, the computation completion flag of Segment1 is set, and the output result is marked as writable.

[0045] For the ODMA stage, the start conditions are: the calculation completion flag of Segment1 is set, there is a valid result in the output buffer, the write DMA channel is available, and the DDR write bandwidth and on-chip interconnect bandwidth allow the result to be written back; its input is the output data size parameter; its output is the "result write-back complete" status; its duration is determined by the output data size and the effective output write-back bandwidth; its occupied resources include at least DDR write bandwidth, on-chip interconnect write bandwidth, write DMA channel, and output buffer; after it completes the relevant operations, Segment1 is marked as complete and the buffer and DMA resources it occupies are released.

[0046] Furthermore, assuming a target SoC scenario, the maximum load bandwidth of COEF is 20GB / s, the maximum transfer bandwidth of IDMA is 12GB / s, the maximum write-back bandwidth of ODMA is 10GB / s, the effective computation throughput of PROC is 4TOPS, the available read bandwidth of DDR is 24GB / s, the available write bandwidth of DDR is 12GB / s, the available bandwidth of on-chip interconnect is 16GB / s, the number of read DMA channels is 1, the number of write DMA channels is 1, the coefficient buffer capacity is 256KB, the input buffer capacity is 1MB, and the output buffer capacity is 1MB, then the duration of the COEF phase is determined by "weight scale / effective weight load bandwidth". The duration of the decision is determined by the quotient of the weight size and the effective weight loading bandwidth, where the effective weight loading bandwidth is the smaller of the maximum loading bandwidth of COEF, the available read bandwidth of DDR, and the available bandwidth of on-chip interconnect. The duration of IDMA is determined by "input data size / effective input transport bandwidth". The duration of PROC is determined by "computational load / effective computational throughput". The duration of ODMA is determined by "output data size / effective output write-back bandwidth". It is worth noting that since the number of read DMA channels is 1, COEF and IDMA cannot occupy the read DMA channel at the same time. The priority of execution needs to be determined in subsequent steps according to the arbitration rules.

[0047] Step S40: Determine the resource contention and concurrent scheduling rules between different execution stages based on the key performance parameters corresponding to the NPU to be evaluated; wherein, the resource contention and concurrent scheduling rules include read bandwidth contention, write bandwidth contention, on-chip interconnect bandwidth occupancy, cache occupancy, storage channel occupancy, mutual exclusion and overlap between different execution stages, and concurrent scheduling mechanism.

[0048] As an example, in this embodiment, key performance parameters of the NPU side will be extracted from the IP, design specifications, performance characterization documents or historical test data of the NPU to be evaluated; at the same time, environmental parameters of the scenario side will be extracted from the target SoC platform or target simulation scenario on which the NPU to be evaluated is mounted; and the two will be organized into a unified parameter set to drive subsequent segment modeling, stage latency calculation, resource contention modeling, concurrency scheduling and performance analysis.

[0049] Key performance parameters on the NPU side include, but are not limited to, pipeline stage division method, maximum weight loading bandwidth of the COEF stage, maximum input transport bandwidth of the IDMA stage, effective computation throughput of the PROC stage, maximum output write-back bandwidth of the ODMA stage, coefficient buffer capacity, input buffer capacity, output buffer capacity, number of DMA channels, maximum number of supported concurrent transactions, stage mutual exclusion, stage overlap enabling, stage scheduling priority, read-write arbitration constraints, basic startup overhead, and basic cleanup overhead. It should be noted that the maximum weight loading bandwidth of the COEF stage, maximum input transport bandwidth of the IDMA stage, effective computation throughput of the PROC stage, maximum output write-back bandwidth of the ODMA stage, maximum number of supported concurrent transactions, basic startup overhead, and basic cleanup overhead are mainly used for subsequent performance analysis; the pipeline stage division method, coefficient buffer capacity, input buffer capacity, output buffer capacity, number of DMA channels, stage mutual exclusion, stage overlap enabling, stage scheduling priority, and read-write arbitration constraints are mainly used for subsequent segment modeling, stage latency calculation, resource contention modeling, and concurrent scheduling.

[0050] In this embodiment, resource contention relationships in complex SoC scenarios will be further established based on key performance parameters of the NPU side. These include, but are not limited to, read bandwidth contention relationships (such as DDR read bandwidth contention), write bandwidth contention relationships (such as DDR write bandwidth contention), on-chip interconnect bandwidth occupancy relationships, cache occupancy relationships (such as internal cache or buffer occupancy), storage channel occupancy relationships (such as DMA channel occupancy), mutual exclusion and overlap relationships between different execution stages, and concurrent scheduling mechanisms. Through the above resource contention relationship model, the simulation can not only reflect the ideal latency of a single segment, but also the waiting, blocking, and backoff behaviors in multi-segment concurrent scenarios.

[0051] The ideal latency of a single segment can be obtained by summing the durations of the stages under contention-free conditions. For example, assuming that the weight size of the aforementioned Segment1 is 144KB, the input data size is 392KB, the output data size is 784KB, the computational load is 231M operations, the effective bandwidth of COEF is 16GB / s, the effective bandwidth of IDMA is 12GB / s, the effective bandwidth of ODMA is 10GB / s, and the effective throughput of PROC is 4TOPS, then the COEF duration of Segment1 is obtained from 144KB / 16GB / s, the IDMA duration is obtained from 392KB / 12GB / s, the PROC duration is obtained from 231M / 4TOPS, and the ODMA duration is obtained from 784KB / 10GB / s. The total time obtained when the four stages are executed sequentially is the ideal latency of Segment1 under the scenario of no resource contention, no waiting, and no blocking.

[0052] In multi-segment concurrent scenarios, the resource contention model described above is used to represent waiting, blocking, and backoff behaviors. Taking Segment1 and Segment2 as examples, if they share the same DDR read path and the same read DMA channel, when Segment1 is in the IDMA stage, although the COEF stage of Segment2 has met the dependency conditions, it cannot start immediately because the read DMA channel is occupied by Segment1. Instead, it enters a waiting state for resources. When the IDMA of Segment1 is completed, the scheduler re-determines whether to grant the required read DMA channel to Segment2 based on priority or arbitration rules. If the on-chip interconnect bandwidth is still insufficient at this time, the COEF stage of Segment2 continues to wait for resources and will retry to start at the next scheduling time. The above "retry to start" process is the backoff behavior.

[0053] Furthermore, if Segment1's ODMA is occupying on-chip interconnect write bandwidth, while Segment2's IDMA is simultaneously requesting on-chip interconnect read bandwidth, although the corresponding path directions are different, if the total on-chip interconnect bandwidth is limited, the effective input transport bandwidth of Segment2 will decrease, resulting in a corresponding extension of its duration, thus demonstrating the waiting and blocking effects introduced by multi-path concurrent access in complex SoC scenarios.

[0054] Based on this, this embodiment establishes a concurrent scheduling mechanism for multiple segments, i.e., see [link to documentation]. Figure 2As shown, the scheduler determines when each segment enters a stage of COEF, IDMA, PROC, or ODMA based on the dependencies of each segment, the stage completion status, the resource availability status, and the preset priority rules, and whether different segments are allowed to overlap in different stages. It is worth noting that the concurrent scheduling mechanism in this embodiment supports the next segment entering the COEF or IDMA stage while the previous segment is in the PROC stage, thereby simulating the pipelined processing behavior of the NPU.

[0055] To enable overlapping execution of different segments at different stages, this embodiment can predefine a stage overlap relationship matrix. This matrix indicates whether different stage combinations can run simultaneously under resource-allowed conditions. For example, it can be defined that "the PROC stage of the previous segment is allowed to overlap with the COEF stage of the next segment," and "the PROC stage of the previous segment is allowed to overlap with the IDMA stage of the next segment." If COEF and IDMA share the same read DMA channel, they are not allowed to overlap in the same time slice. If two ODMA stages share the same write DMA channel, the two ODMA stages are not allowed to overlap. If the computing core itself does not support multi-task concurrency, the two PROC stages are not allowed to overlap.

[0056] Step S50: Control the target neural network model to run the simulation based on the network model task segments and execution phases by using resource contention concurrency scheduling rules and control rules.

[0057] In this exemplary embodiment, the target neural network model is controlled to run in simulation based on the aforementioned resource contention concurrency scheduling rules and control rules, according to the network model task segments and execution phases. Under the time-progression mechanism, the state of multiple segments is continuously updated. Specifically, at each time-progression point or event trigger point, the scheduler will schedule each segment through control signals, and execute the following logic: First, check whether the predecessor dependencies of each segment are satisfied; then, put the segments whose dependencies are satisfied but have not yet started into the ready queue; then, determine which segments can enter the running state based on the stage overlap relationship matrix, the current resource occupancy table, the stage priority, and the arbitration rules; if a stage satisfies the dependency but has insufficient resources, the stage enters the waiting resource state; when a stage is completed, the resource occupancy table is updated immediately, and the ready judgment of the subsequent stage and the subsequent segment is triggered.

[0058] Specifically, assuming Segment1 is executing PROC and Segment2 has satisfied its predecessor dependency; if the aforementioned stage overlap matrix allows "Segment1's PROC" and "Segment2's COEF" to overlap, and the read DMA channel is idle and the coefficient buffer capacity is sufficient, then Segment2 can enter the COEF stage; if Segment2 subsequently attempts to enter the IDMA stage, but the read DMA channel is still occupied by other read paths, or the on-chip interconnect read bandwidth is insufficient, then Segment2's IDMA stage enters a resource waiting state; when the corresponding resources are released, the scheduler switches Segment2's IDMA stage to the running state. Through this method, controllable overlap of different segments at different stages can be achieved, thereby simulating the pipelined processing behavior of the NPU.

[0059] Step S60: Determine the performance evaluation result of the NPU to be evaluated based on the simulation results, key performance parameters, and environmental parameters corresponding to the SoC on which the NPU to be evaluated is mounted.

[0060] As an example, in this embodiment, the environmental parameters on the scenario side include, but are not limited to, DDR available read bandwidth, DDR available write bandwidth, on-chip interconnect available bandwidth, on-chip interconnect average additional latency, the proportion of background services (i.e., services that access DDR outside of the NPU) occupying the DDR read path, the proportion of background services occupying the DDR write path, the proportion of background services occupying the on-chip interconnect, DMA channel reservation policy, SoC operating frequency or frequency multiplier, task arrival method, and QoS (Quality of Service) weight configuration.

[0061] After the simulation is completed, the simulation results will be obtained. These results include actual test data corresponding to the scene-side environmental parameters and the NPU-side key performance parameters. The scene-side environmental parameters and the NPU-side key performance parameters will be used as theoretical true values. By comparing and analyzing the theoretical true values ​​with the actual test data, the performance evaluation results of the NPU to be evaluated can be determined.

[0062] Understandably, traditional solutions lack sufficient support for NPU intellectual property reuse across projects and SoC platforms, often requiring the rebuilding of performance evaluation models for different chip environments, resulting in poor reusability. However, in this embodiment, when the same or similar NPU IP is applied to different SoC projects, only the set of key performance parameters and the scenario-side environmental parameters need to be adjusted to reuse the same abstract performance simulation framework to complete the evaluation of new projects. For example, in SoC-A, the available DDR read bandwidth is 24GB / s, and the background services consume relatively little on-chip interconnect bandwidth, so the COEF and IDMA stages of Segment 1 can obtain higher effective bandwidth. However, in SoC-B, the available DDR read bandwidth is 16GB / s, and the CPU or GPU background services consume an additional 30% of the on-chip interconnect bandwidth, so the duration of the COEF, IDMA, and ODMA stages of Segment 1 will be extended. The above changes do not require modification of the segmentation method or the structure of the four-stage objects. The migration from SoC-A to SoC-B can be completed simply by replacing the scenario-side environmental parameters, thus demonstrating the advantages of this embodiment in NPU IP reuse and cross-project performance evaluation.

[0063] In summary, this embodiment combines key NPU performance parameters with scenario-side environmental parameters to achieve performance improvement without relying on a complete NPU. Under the premise of RTL cycle-by-cycle simulation, this embodiment abstractly represents the key performance behaviors of the NPU, avoiding cycle-by-cycle simulation of the complete RTL. This significantly improves the efficiency of SoC-level performance evaluation while ensuring the characterizability of key performance behaviors, and enables rapid migration and reuse across different SoC projects and different intellectual property reuse scenarios, thereby reducing repetitive modeling workload. Furthermore, traditional solutions struggle to simultaneously express the concurrent advancement, waiting, blocking, and write-back behaviors of multiple task segments at different stages, failing to fully reflect the multi-path execution characteristics of complex SoC scenarios. This embodiment decomposes the NPU execution process into stages such as COEF, IDMA, PROC, and ODMA, and establishes a multi-segment concurrent scheduling mechanism. This effectively represents and reflects the multi-path concurrency, pipeline overlap, and resource contention behaviors such as weight loading, input handling, computation execution, and output write-back in complex SoC scenarios, outputting instructive performance evaluation results with higher performance fidelity compared to pure software ideal models. In addition, this embodiment can be used for early performance prediction in the design verification stage, as well as for architecture optimization, parameter tuning, scenario analysis, and regression evaluation, demonstrating strong engineering practical value.

[0064] Secondly, embodiments of this application also provide an NPU performance evaluation device under SoC multi-path conditions.

[0065] In one embodiment, reference is made to Figure 3 , Figure 3This is a functional module diagram of an embodiment of the NPU performance evaluation device under SoC multi-path in this application. Figure 3 As shown, the NPU performance evaluation device under SoC multi-path includes: The task segmentation module is used to divide the model graph corresponding to the target neural network model that needs to run on the NPU to be evaluated, and obtain multiple network model task segments. The stage decomposition module is used to decompose the execution process of each network model task segment to obtain multiple execution stages with pipeline characteristics. The rule determination module is used to determine the control rules for each execution stage based on the preset attribute parameters of the task segment of the network model; and to determine the resource contention and concurrent scheduling rules between different execution stages based on the key performance parameters corresponding to the NPU to be evaluated. The simulation control module is used to control the target neural network model to perform simulation operation based on the network model task segments and execution phases through resource contention and concurrent scheduling rules and control rules; The performance evaluation module is used to determine the performance evaluation result of the NPU to be evaluated based on the simulation results, key performance parameters, and environmental parameters corresponding to the SoC on which the NPU to be evaluated is mounted.

[0066] Furthermore, in one embodiment, the multiple execution stages with pipeline characteristics are, in sequence, a coefficient loading stage, an input data transport stage, a calculation execution stage, and an output data transport stage; The coefficient loading stage is used to characterize the behavior of loading the weight parameters required for the network model task segment from external memory into the NPU internal coefficient buffer. The input data transfer stage is used to characterize the behavior of transferring the data required for the network model task segment from external memory to the NPU input buffer; The computation execution phase is used to characterize the core computational operations corresponding to the task segments of the network model. The output data transfer stage is used to characterize the behavior of writing the output of the network model task segment from the NPU output buffer back to external memory.

[0067] Furthermore, in one embodiment, the resource contention concurrency scheduling rules include read bandwidth contention, write bandwidth contention, on-chip interconnect bandwidth occupancy, cache occupancy, storage channel occupancy, mutual exclusion and overlap between different execution stages, and a concurrency scheduling mechanism.

[0068] Furthermore, in one embodiment, the control rules include startup conditions, input-output relationships, duration calculation rules, resource usage types, concurrency levels, and state transition rules.

[0069] Furthermore, in one embodiment, the task segmentation module is specifically used for: Based on a preset decomposition strategy, the model graph corresponding to the target neural network model to be run on the NPU to be evaluated is divided into multiple network model task segments. The preset decomposition strategy includes layer structure, operator type, data dependency relationship and data block size.

[0070] The functions of each module in the above-mentioned SoC multi-path NPU performance evaluation device correspond to the steps in the above-mentioned SoC multi-path NPU performance evaluation method embodiment, and their functions and implementation processes will not be described in detail here.

[0071] Thirdly, embodiments of this application provide a SoC multi-path NPU performance evaluation device, which can be a personal computer (PC), laptop, server, or other device with data processing capabilities.

[0072] Reference Figure 4 , Figure 4 This is a schematic diagram of the hardware structure of the SoC multi-path NPU performance evaluation device involved in the embodiments of this application. In the embodiments of this application, the SoC multi-path NPU performance evaluation device may include a processor, memory, communication interface, and communication bus.

[0073] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0074] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within the NPU performance evaluation device in a SoC multi-path environment, as well as interfaces used for interconnecting the NPU performance evaluation device with other devices (such as other computing devices or user devices). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user devices can be displays, keyboards, etc.

[0075] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0076] The processor can be a general-purpose processor, which can call the SoC multi-path NPU performance evaluation program stored in memory and execute the SoC multi-path NPU performance evaluation method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the SoC multi-path NPU performance evaluation program is called can refer to the various embodiments of the SoC multi-path NPU performance evaluation method of this application, and will not be repeated here.

[0077] Those skilled in the art will understand that Figure 4 The hardware structure shown does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0078] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0079] The present application stores a SoC multi-path NPU performance evaluation program on a readable storage medium, wherein when the SoC multi-path NPU performance evaluation program is executed by a processor, it implements the steps of the SoC multi-path NPU performance evaluation method described above.

[0080] The method implemented when the NPU performance evaluation program under SoC multi-path is executed can be referred to in the various embodiments of the SoC multi-path NPU performance evaluation method of this application, and will not be repeated here.

[0081] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0082] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0083] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0084] In the description of the embodiments of this application, "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, in the description of the embodiments of this application, "multiple" refers to two or more.

[0085] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0087] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for evaluating NPU performance under SoC multi-path conditions, characterized in that, include: The model graph corresponding to the target neural network model that needs to run on the NPU to be evaluated is divided into multiple network model task segments. The execution process of each network model task segment is decomposed to obtain multiple execution stages with pipeline characteristics; The control rules for each execution stage are determined based on the preset attribute parameters of the task segments in the network model. Determine the resource contention and concurrent scheduling rules between different execution stages based on the key performance parameters corresponding to the NPU to be evaluated; The target neural network model is controlled by resource contention and concurrent scheduling rules and control rules, and simulation operation is performed based on the network model task segments and execution phases. The performance evaluation results of the NPU to be evaluated are determined based on the simulation results, key performance parameters, and environmental parameters corresponding to the SoC on which the NPU is mounted.

2. The method for evaluating NPU performance under SoC multi-path as described in claim 1, characterized in that: The multiple execution stages with flow characteristics are, in sequence, the coefficient loading stage, the input data transport stage, the calculation execution stage, and the output data transport stage; The coefficient loading stage is used to characterize the behavior of loading the weight parameters required for the network model task segment from external memory into the NPU internal coefficient buffer. The input data transfer stage is used to characterize the behavior of transferring the data required for the network model task segment from external memory to the NPU input buffer; The computation execution phase is used to characterize the core computational operations corresponding to the task segments of the network model. The output data transfer stage characterizes the behavior of writing the output of the network model task segment from the NPU output buffer back to external memory.

3. The method for evaluating NPU performance under SoC multi-path as described in claim 1, characterized in that: The resource contention and concurrent scheduling rules include read bandwidth contention, write bandwidth contention, on-chip interconnect bandwidth occupancy, cache occupancy, storage channel occupancy, mutual exclusion and overlap between different execution stages, and concurrent scheduling mechanisms.

4. The method for evaluating NPU performance under SoC multi-path as described in claim 1, characterized in that: The control rules include startup conditions, input-output relationships, duration calculation rules, resource types, concurrency levels, and state transition rules.

5. The method for evaluating NPU performance under SoC multi-path as described in claim 1, characterized in that, The process involves dividing the model graph corresponding to the target neural network model that needs to run on the NPU to be evaluated into multiple network model task segments, including: Based on a preset decomposition strategy, the model graph corresponding to the target neural network model to be run on the NPU to be evaluated is divided into multiple network model task segments. The preset decomposition strategy includes layer structure, operator type, data dependency relationship and data block size.

6. A device for evaluating NPU performance under SoC multi-path conditions, characterized in that, include: The task segmentation module is used to divide the model graph corresponding to the target neural network model that needs to run on the NPU to be evaluated, and obtain multiple network model task segments. The stage decomposition module is used to decompose the execution process of each network model task segment to obtain multiple execution stages with pipeline characteristics. The rule determination module is used to determine the control rules for each execution stage based on the preset attribute parameters of the task segment of the network model; and to determine the resource contention and concurrent scheduling rules between different execution stages based on the key performance parameters corresponding to the NPU to be evaluated. The simulation control module is used to control the target neural network model to perform simulation operation based on the network model task segments and execution phases through resource contention and concurrent scheduling rules and control rules; The performance evaluation module is used to determine the performance evaluation result of the NPU to be evaluated based on the simulation results, key performance parameters, and environmental parameters corresponding to the SoC on which the NPU to be evaluated is mounted.

7. The NPU performance evaluation device under SoC multi-path as described in claim 6, characterized in that: The multiple execution stages with flow characteristics are, in sequence, the coefficient loading stage, the input data transport stage, the calculation execution stage, and the output data transport stage; The coefficient loading stage is used to characterize the behavior of loading the weight parameters required for the network model task segment from external memory into the NPU internal coefficient buffer. The input data transfer stage is used to characterize the behavior of transferring the data required for the network model task segment from external memory to the NPU input buffer; The computation execution phase is used to characterize the core computational operations corresponding to the task segments of the network model. The output data transfer stage characterizes the behavior of writing the output of the network model task segment from the NPU output buffer back to external memory.

8. The NPU performance evaluation device under SoC multi-path as described in claim 6, characterized in that: The resource contention and concurrent scheduling rules include read bandwidth contention, write bandwidth contention, on-chip interconnect bandwidth occupancy, cache occupancy, storage channel occupancy, mutual exclusion and overlap between different execution stages, and concurrent scheduling mechanisms.

9. A device for evaluating NPU performance under SoC multi-path conditions, characterized in that, The SoC multi-path NPU performance evaluation device includes a processor, a memory, and an SoC multi-path NPU performance evaluation program stored in the memory and executable by the processor. When the SoC multi-path NPU performance evaluation program is executed by the processor, it implements the steps of the SoC multi-path NPU performance evaluation method as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a SoC multi-path NPU performance evaluation program, wherein when the SoC multi-path NPU performance evaluation program is executed by the processor, it implements the steps of the SoC multi-path NPU performance evaluation method as described in any one of claims 1 to 5.

Images