Performance bottleneck analysis method

Abstract

The present disclosure provides a performance bottleneck analysis method for analyzing the performance bottleneck of a target system. The performance bottleneck analysis method includes: traversing a software schedule tree from subtree to root, calculating the data traffic and execution count of each of multiple computing operations of the software suitable for running on the target system; and, based on the data traffic and execution count, calculating the equivalent cycles of each of multiple hardware modules of the target system to identify the hardware bottleneck of the target system.

Description

Technical Field

[0001] The present disclosure relates to an analysis of an electronic system, and more particularly to a performance bottleneck analysis method. Background Art

[0002] Performance analysis is a crucial step in electronic system development. It helps designers understand whether the target system is achieving its rated efficiency. By identifying which hardware module (or modules) is the performance bottleneck, designers can optimize the target system and / or adjust hardware parameters based on the bottleneck analysis results. Summary of the Invention

[0003] The present disclosure provides a performance bottleneck analysis method for finding the hardware bottleneck of a target system.

[0004] In an embodiment of the present disclosure, the performance bottleneck analysis method is used to analyze the performance bottleneck of a target system. The performance bottleneck analysis method includes: traversing the software schedule tree from subtree to root, calculating the data traffic and execution count of each of multiple computing operations of the software suitable for running on the target system; and calculating the equivalent cycles of each of multiple hardware modules of the target system based on the data traffic and execution count to identify the hardware bottleneck of the target system.

[0005] Based on the above, the software to be tested can be run on the target system. In some practical application examples, the target system can be a multi-accelerator system or other hardware system, and the software to be tested can include computing software for processing tensors. In an embodiment according to the present disclosure, the performance bottleneck analysis method first traverses the scheduling tree of the software to be tested to calculate the number of executions of each task node (operation operation) of the scheduling tree, and calculates the data flow (read-in and write-out data flow) of each task node (operation operation). Different task nodes (operation operations) will use the corresponding hardware modules of the target system. Based on the data flow and execution number of a certain target task node (operation operation), the performance bottleneck analysis method can calculate the equivalent cycle of the corresponding hardware module of the target task node. For example, dividing the data flow by the data transmission bandwidth can obtain the "load store cycles" of the target task node (operation operation) as the equivalent cycle. The more cycles a hardware module has, the more likely it is to become a performance bottleneck. After calculating the equivalent cycle of each hardware module, the hardware bottleneck of the target system can be easily found. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 4 is a flow chart of a performance bottleneck analysis method according to an embodiment of the present disclosure.

[0007] Figure 2 FIG1 is a schematic diagram of a scheduling tree for convolutional neural network computing software according to an embodiment of the present disclosure.

[0008] Figure 3 It is a flowchart of a performance bottleneck analysis method according to another embodiment of the present disclosure.

[0009] Figure 4 FIG. 1 is a diagram illustrating a task-level pipeline with a synchronous channel according to an embodiment of the present disclosure.

[0010] Description of Reference Numerals

[0011] S110, S120, S210, S220, S230: Steps

[0012] S205, S210, S220, S225, S245: loop nodes

[0013] S215, S230, S235, S240, S250: Task nodes

[0014] bn_relu, conv, load_activation, load_weight, reduce: tasks DETAILED DESCRIPTION

[0015] Reference will now be made in detail to exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0016] The term "coupled (or connected)" used in the entire specification of this case (including the claims) may refer to any direct or indirect means of connection. For example, if the text describes a first device coupled (or connected) to a second device, it should be interpreted as the first device being directly connected to the second device, or the first device being indirectly connected to the second device through other devices or some means of connection. The terms "first", "second", etc. mentioned in the entire specification of this case (including the claims) are used to name the components (element), and are not used to limit the upper or lower limit of the number of components, nor are they used to limit the order of components. In addition, wherever possible, components / members / steps with the same numbers in the drawings and embodiments represent the same or similar parts. Components / members / steps with the same numbers or the same terms in different embodiments can refer to the relevant descriptions of each other.

[0017] The performance bottleneck analysis method will be described using the following embodiments. In some practical application examples, the performance bottleneck analysis method can be implemented as software, such as electronic design automation (EDA) software, computer-aided design (CAD) software, or other integrated circuit design software. The software can be recorded / stored on a "non-transitory computer-readable medium." In some embodiments, the non-transitory computer-readable medium includes, for example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, and / or a storage device. The storage device includes a hard disk drive (HDD), a solid-state drive (SSD), or other storage device. A central processing unit (CPU), a controller, a microcontroller, or a microprocessor can read and execute the software from the non-transitory computer-readable medium, thereby implementing the relevant functions of the performance bottleneck analysis method.

[0018] The performance bottleneck analysis method can analyze any electronic system (hereinafter referred to as the target system). In some practical application examples, the target system can be a multi-accelerator system or other hardware system. The target system can run the software to be tested. The software to be tested can include any computing software that processes tensors. In some practical application examples, the software to be tested can include convolutional neural network (CNN) computing software or other application software.

[0019] Figure 1 4 is a flow chart of a performance bottleneck analysis method according to an embodiment of the present disclosure. Figure 1 Step S110 may traverse a schedule tree of the software (software under test) from subtree to root, calculating the data flow and execution count of each of a plurality of computing operations of the software under test suitable for running on the target system. For example, the software under test may include convolutional neural network (CNN) computing software.

[0020] Figure 2 FIG. 1 is a schematic diagram of a scheduling tree for convolutional neural network computing software (software under test) according to an embodiment of the present disclosure. Figure 2 The shown scheduling tree includes a plurality of nodes (Vertex). Figure 2 The nodes shown include loop nodes, sequence nodes, and task nodes. Loop nodes can also be called iteration nodes. For example, Figure 2 The nodes S205, S210, S220, S225 and S245 are all loop nodes. A sequence node is a linear sequence of multiple nodes. For example, Figure 2 The nodes S230 and S235 are in the same sequence node, and Figure 2 The nodes S215, S220, S240 and S245 are in another sequence node. All nodes in the same sequence node will be executed sequentially. Task nodes can also be called intrinsic nodes. For example, Figure 2 The nodes (computation operations) S215 , S230 , S235 , S240 , and S250 shown are all task nodes.

[0021] Task node S215 can execute a task (operation) named "load_weight" to load the weight matrix into a hardware module of the target system, such as a buffer of a tensor core. Task node S230 can execute a task (operation) named "load_activation" to load the activation matrix into another hardware module of the target system, such as another buffer of a tensor core. Task node S235 can execute a task (operation) named "conv" to use another hardware module of the target system (such as the General Matrix Multiplication (GEMM) circuit of the tensor core) to perform convolution calculations on the weight matrix and the activation matrix to generate a convolved data matrix. Task node S240 can execute a task (operation) named "reduce" to perform a reduction operation on the convolved data matrix to generate a reduced-dimensional data matrix. The task node S250 may execute a task (operation) named “bn_relu” to perform batch normalization calculation and activation calculation (eg, ReLU) on the dimensionally reduced data matrix.

[0022] Please refer to Figure 1 and Figure 2 Step S110 can be traversed from the subtree to the root. Figure 2The scheduling tree shown calculates the data flow and execution count for each of the task nodes (operations) S215, S230, S235, S240, and S250. For example, by traversing the subtree to the root, the "conv" task (task node S235) is repeatedly triggered / executed 128 times by loop node S225, which is repeatedly triggered / executed 3136 times by loop node S220, which is repeatedly triggered / executed 128 times by loop node S210, and which is repeatedly triggered / executed twice by loop node S205. Therefore, the execution count of the "conv" task (operation, i.e., task node S235) is 128*3136*128*2=102760448 times. Each time task node S235 is executed, a convolved data matrix is generated and stored in a buffer. Assuming the data size of a convolved data matrix is MN, the data flow of task node S235 (operation operation) is 102760448*MN. The calculation of the data flow and execution times of other task nodes S215, S230, S240, and S250 can refer to the relevant description of the data flow and execution times of task node S235 and are therefore not repeated here.

[0023] Figure 1 The step S120 shown can calculate the equivalent cycles of each hardware module of the target system based on the data flow and execution times of different task nodes (computational operations) to find the hardware bottleneck of the target system. For example, the target system may include a tensor core with a general matrix multiplication (GEMM) circuit, an A buffer and a B buffer, and a vector core with multiple computing units (CUs). Step S120 can calculate the number of cycles of executing multiple computing operations (such as Figure 2 The equivalent cycle of each hardware module in the process of different task nodes shown in FIG.

[0024] Step S120 can calculate load_cycles:=total_traffic / hardware_bandwidth, where load_cycles represents the equivalent number of cycles for the software to access the target hardware module, total_traffic represents the total amount of data accessed by the software to the target hardware module, and hardware_bandwidth represents the access bandwidth of the target hardware module. For example, the data traffic calculated in step S110 may include Figure 2 The total amount of data accessed by different task nodes (computational operations) to a target hardware module (e.g., the A buffer in the tensor core) among multiple hardware modules of the target system. Step S120 can calculate ldA = Adata / hbm_bufa, where ldA (equivalent to load_cycles) represents the number of cycles used to access the A buffer in the tensor core (the equivalent number of cycles used by the software under test to access the target hardware module), Adata (equivalent to total_traffic) represents the total amount of data accessed to the A buffer (the total amount of data accessed by the software under test to the target hardware module), and hbm_bufa (equivalent to hardware_bandwidth) represents the hardware bus width used to move data from the High Bandwidth Memory (HBM) to the A buffer (the access bandwidth of the target hardware module). Similarly, step S120 can calculate ldB = Bdata / l1P5_bufb, where ldB (equivalent to load_cycles) represents the number of cycles to access the B buffer in the tensor core (the equivalent number of cycles for the software under test to access the target hardware module), Bdata (equivalent to total_traffic) represents the total amount of data accessed to the B buffer (the total amount of data accessed by the software to the target hardware module), and l1P5_bufb (equivalent to hardware_bandwidth) represents the hardware bus width for moving data from the cache to the B buffer (the access bandwidth of the target hardware module). The total data amount Adata and the total data amount Bdata can be obtained through the calculation in step S110.

[0025] The number of executions calculated in step S110 may include Figure 2The total number of times a target hardware module (e.g., a GEMM circuit in a tensor core) from among multiple hardware modules of the target system is used in different task nodes (computational operations) is shown. Step S120 may calculate computing_cycles:=total_ops / hardware_ops_per_cycle, where computing_cycles represents the number of equivalent cycles that the software uses the target hardware module, total_ops represents the total number of times the software uses the target hardware module, and hardware_ops_per_cycle represents the maximum number of multiple computing operations that the target hardware module can execute in one cycle. For example, step S120 can calculate GEMM=MAC_ops / opspercycle, where GEMM (equivalent to computing_cycles) represents the number of cycles in which the GEMM circuit in the tensor core performs computing operations (the equivalent number of cycles in which the software under test uses the target hardware module), MAC_ops (equivalent to total_ops) represents the total number of times the GEMM circuit is used (the total number of times the software under test uses the target hardware module), and opspercycle (equivalent to hardware_ops_per_cycle) represents the hardware data path width (the maximum number of multiple computing operations performed by the target hardware module in one cycle). The hardware data path width opspercycle can be understood as the maximum number of multiple computing operations performed by the GEMM circuit in one cycle. Step S120 can also calculate GEMM_wr = GEMM_result / wrpercycle, where GEMM_wr represents the number of cycles in which the GEMM circuit in the Tensor Core writes the convolved data matrix to the buffer, GEMM_result represents the total data throughput of the GEMM circuit writing data (the convolved data matrix), and wrpercycle represents the hardware data path width. The hardware data path width wrpercycle can be understood as the maximum amount of data written by the GEMM circuit in one cycle. The total number of MAC_ops and the total data throughput GEMM_result can be obtained by the calculation in step S110.

[0026] Step S120 may calculate at least one first equivalent cycle count of at least one memory module in the target hardware module accessed by the software under test, and at least one second equivalent cycle count of at least one hardware computing module in the target hardware module used by the software under test. Step S120 may use the maximum of the at least one first equivalent cycle count and the at least one second equivalent cycle count as the equivalent cycle count of the target hardware module. For example, step S120 may determine the maximum of the cycle counts ldA, ldB, GEMM, and GEMM_wr as the equivalent cycle count tcore_cycle of the Tensor Core.

[0027] By the same token, step S120 can also calculate load_matrix = total_load_traffic / bandwidth_of_memory_block, store_matrix = total_store_traffic / bandwidth_of_memory_block, and vector_cycle = vector_ops / ops_per_cycle, where load_matrix represents the number of cycles in which the vector core loads the matrix, total_load_traffic represents the total amount of data loaded by the vector core for matrix loading, bandwidth_of_memory_block represents the hardware bus width for matrix loading / storing back by the vector core, store_matrix represents the number of cycles in which the vector core stores back the matrix, total_store_traffic represents the total amount of data stored back by the vector core for matrix storage, vector_cycle represents the number of cycles in which the vector core performs computing operations, vector_ops represents the total number of times the vector core is used, and ops_per_cycle represents the maximum number of multiple computing operations performed by the vector core in one cycle. The total data volume total_load_traffic, the total data volume total_store_traffic, and the total number of usage vector_ops can be obtained through the calculation in step S110. Step S120 can find the maximum of the cycle numbers load_matrix, store_matrix, and vector_cycle as the equivalent cycle number vector_cycle of the vector core. Step S120 can also find the maximum of the equivalent cycle numbers "tcore_cycle", "vector_cycle", "gemm_wr_mem_cycle+stm", and "gemm_wr_tlr+ldm" as the equivalent cycle number of the target system. Among them, "gemm_wr_mem_cycle+stm" and "gemm_wr_tlr+ldm" represent the equivalent cycle number of the contention bandwidth portion between the tensor core and the vector core.

[0028] In summary, in an embodiment according to the present disclosure, the performance bottleneck analysis method can first traverse the scheduling tree of the software to be tested to calculate the number of executions of each task node (operation operation) of the scheduling tree, and calculate the data flow (read-in and write-out data flow) of each task node (operation operation). Different task nodes (operation operations) will use the corresponding hardware modules of the target system. Based on the data flow and number of executions of a certain target task node (operation operation), the performance bottleneck analysis method can calculate the equivalent cycle of the corresponding hardware module of the target task node. For example, dividing the data flow by the data transmission bandwidth can obtain the "loadstore cycles" of the target task node (operation operation) as the equivalent cycle. Generally speaking, the more cycles a hardware module has, the more likely it is to become a performance bottleneck. After calculating the equivalent cycle of each hardware module, the hardware bottleneck of the target system can be easily found.

[0029] In some embodiments, the equivalent cycles calculated in step S120 can be recorded in a document to facilitate hardware / software designers in identifying performance bottlenecks (hardware bottlenecks and / or software bottlenecks). In some embodiments, the equivalent cycles calculated in step S120 can be presented to hardware / software designers in any manner to assist them in identifying performance bottlenecks. For example, the equivalent cycles calculated in step S120 can be entered into a table. For example, assuming that step S110 calculates that the number of executions (total usage times) of the "conv" task (calculation operation) of the software under test is x times and the hardware data path width of the GEMM circuit used by the "conv" task is y, the table can present the equivalent number of cycles "x / y" calculated by the GEMM circuit in step S120 for performing the "conv" task. Similarly, the table can present the equivalent number of cycles for different hardware modules of the target system to perform different tasks. The hardware / software designer can observe the equivalent cycle numbers of different hardware modules and different tasks in the table, and then find out the hardware bottleneck of the target system and / or the software bottleneck of the software to be tested.

[0030] Figure 3 It is a flowchart of a performance bottleneck analysis method according to another embodiment of the present disclosure. Figure 3 The steps S310 and S320 shown can refer to Figure 1 The related descriptions of steps S110 and S120 are analogous and will not be repeated here. Figure 3In the illustrated embodiment, step S330 can use a task-level pipeline with synchronization channels to visualize the equivalent cycles of different computational operations of the software under test. The visualization method (chart layout) of step S330 can be set according to actual design or application requirements.

[0031] Figure 4 FIG. 1 is a diagram illustrating a task-level pipeline with a synchronous channel according to an embodiment of the present disclosure. Figure 4 The horizontal axis shown represents time (equivalent number of cycles) and the vertical axis represents tasks. Figure 4 A task-level long-pole diagram is shown for different tasks. Figure 2 、 Figure 3 and Figure 4 Based on the calculation results of steps S310 and S320, Figure 4 In the illustrated embodiment, step S330 plots tasks such as "load_weight," "load_activation," "conv," "reduce," and "bn_relu" at different positions on the timeline according to their execution time. The lengths of these tasks correspond to the equivalent cycles of the different tasks calculated in step S320. Hardware / software designers can observe the equivalent cycles of different tasks in the task-level pipeline diagram to identify software bottlenecks in the software under test.

[0032] In summary, Figure 3 The performance bottleneck analysis method shown can be implemented as an analysis model to analyze the root-cause of the performance of a target system (e.g., a multi-accelerator system). The input of the analysis model is hardware constraints and workload description. Hardware constraints include hardware hierarchy and arbitration constraints. Workload description includes a description of a schedule tree. The analysis model can automatically perform the analysis process. The output of the analysis model is task bottlenecks and hardware bottlenecks. Task bottlenecks can be presented by a task-level long-bar graph. The maximum value is usually the bottleneck. Users can optimize the kernel or adjust hardware parameters based on the observation of the visualized results.

[0033] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A performance bottleneck analysis method for analyzing the performance bottleneck of a target system, characterized in that: The performance bottleneck analysis method includes: traversing the scheduling tree of the software from subtree to root, calculating the data flow and execution times of each of a plurality of computing operations of the software suitable for running on the target system; and Based on the data flow and the execution times, an equivalent cycle of each of the multiple hardware modules of the target system is calculated to find the hardware bottleneck of the target system. The step of calculating the equivalent cycle of each of the plurality of hardware modules of the target system includes: By traversing the hierarchical structure of the target system from bottom to top, an equivalent cycle of each of the plurality of hardware modules in a process in which the target system executes the plurality of computing operations is calculated.

2. The performance bottleneck analysis method according to claim 1, characterized in that: The performance bottleneck analysis method further includes: A task-level pipeline with synchronization channels is used to visualize the equivalent cycles of the multiple computing operations.

3. The performance bottleneck analysis method according to claim 1, characterized in that: The data flow includes the total amount of data accessed by the software to a target hardware module among the multiple hardware modules. The performance bottleneck analysis method further includes: Calculate load_cycles:=total_traffic / hardware_bandwidth, where load_cycles represents the equivalent number of cycles for the software to access the target hardware module, total_traffic represents the total amount of data accessed by the software to the target hardware module, and hardware_bandwidth represents the access bandwidth of the target hardware module.

4. The performance bottleneck analysis method according to claim 1, characterized in that: The execution count includes a total number of times a target hardware module among the multiple hardware modules is used in the multiple computing operations of the software. The performance bottleneck analysis method further includes: Calculate computing_cycles:=total_ops / hardware_ops_per_cycle, where computing_cycles represents the number of equivalent cycles of the target hardware module used by the software, total_ops represents the total number of times the software uses the target hardware module, and hardware_ops_per_cycle represents the maximum number of multiple computing operations performed by the target hardware module in one cycle.

5. The performance bottleneck analysis method according to claim 1, characterized in that: The performance bottleneck analysis method further includes: calculating at least a first equivalent cycle number of the software accessing at least one memory module among the target hardware modules of the plurality of hardware modules; calculating at least a second equivalent number of cycles in which the software uses at least one hardware computing module in the target hardware module; and The maximum of the at least one first equivalent cycle number and the at least one second equivalent cycle number is taken as the equivalent cycle number of the target hardware module.

Images