Artificial intelligence accelerator pipeline performance analysis method and apparatus

Abstract

The present invention relates to artificial intelligence accelerator pipeline performance analysis methods and apparatus, wherein the computing device of the present invention is included in an integrated circuit device that includes a general purpose interconnect interface and other processing devices. The computing device interacts with the other processing devices to collectively perform a user-specified computing operation. The integrated circuit device can also include a storage device connected to the computing device and the other processing devices, respectively, for data storage of the computing device and the other processing devices.

Description

Technical Field

[0001] This invention generally relates to the field of computers. More specifically, this invention relates to a method and apparatus for performance analysis of artificial intelligence accelerator pipelines. Background Technology

[0002] As computer systems become increasingly complex, processors experience more and more performance events, which can lead to performance bottlenecks. To identify these bottlenecks, developers need to collect as much runtime data as possible to optimize the computer system's performance.

[0003] Patent No. CN106557400A provides a method for dynamic data collection in a device, comprising: collecting data on high-level performance events through device hardware, wherein the high-level performance events indicate alternative causes of performance bottlenecks; determining a first performance event among the high-level performance events causing performance bottlenecks based on real-time analysis of the data; reconfiguring the device hardware to collect additional data on lower-level performance events below the first performance event, wherein the lower-level performance events indicate additional alternative causes that are more specific than alternative causes; and performing the above-described steps of collection, determination, and reconfiguration while the device continuously executes an application.

[0004] The aforementioned existing technologies require continuous execution of the application when inferring performance bottlenecks, and the high-level and low-level performance events are inferred based on experience. Furthermore, they are designed only for processors such as CPUs, GPUs, and DSPs, and do not provide an effective performance detection method for the unique structure of artificial intelligence accelerators. Summary of the Invention

[0005] In order to at least partially solve the technical problems mentioned in the background art, the present invention provides a method and device for performance analysis of artificial intelligence accelerator pipelines.

[0006] In one aspect, the present invention discloses a processor core for analyzing pipeline performance, comprising a control module, an arithmetic module, and a storage module. The control module, arithmetic module, and storage module correspond to multiple events in the pipeline. These events are combined according to different conditions, and the multiple combinations are mutually exclusive and categorized as front-end bottlenecks and back-end bottlenecks. The processor core also includes multiple performance counters, each associated with a combination. When multiple combinations meet corresponding conditions, the corresponding performance counter counts.

[0007] In another aspect, the present invention discloses an artificial intelligence accelerator, including the aforementioned processor core.

[0008] In another aspect, the present invention discloses an integrated circuit device including the aforementioned artificial intelligence accelerator.

[0009] In another aspect, the present invention discloses a board that includes the aforementioned integrated circuit device.

[0010] In another aspect, this invention discloses a performance analysis method for an artificial intelligence accelerator, which includes a processor core. The method includes: dividing the processor core pipeline into multiple events; forming multiple combinations based on the multiple events, wherein the multiple combinations are mutually exclusive and categorized as front-end bottlenecks and back-end bottlenecks; configuring performance counters associated with the multiple combinations; determining whether the multiple combinations meet corresponding conditions; if so, driving the corresponding performance counters to count; and evaluating the front-end bottlenecks and back-end bottlenecks based on the count values ​​of the performance counters.

[0011] In another aspect, the present invention discloses a computer-readable storage medium storing computer program code for an artificial intelligence accelerator performance analysis method, wherein the computer program code, when run by a processing device, executes the method described above.

[0012] The present invention has the following technical effects:

[0013] 1. This invention configures performance counters for all combinations of events that may cause performance bottlenecks. Each time a performance analysis is performed, the combination of interest can be selected. Each analysis does not require reconfiguration of hardware; only the specific performance counter needs to be read. The performance analysis scheme of this invention is quite lightweight and flexible.

[0014] 2. The performance analysis scheme of the present invention unfolds in a multi-layered manner from top to bottom. The events in each layer are mutually exclusive and are divided into front-end bottlenecks and back-end bottlenecks, which can logically and quickly locate performance bottlenecks.

[0015] 3. This invention defines event combinations specifically for the unique structure of AI accelerators, enabling direct identification of performance bottlenecks in AI accelerators. Attached Figure Description

[0016] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts. Wherein:

[0017] Figure 1 This is a structural diagram of the board card according to an embodiment of the present invention;

[0018] Figure 2 This is a structural diagram illustrating an integrated circuit device according to an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram illustrating the internal structure of a computing device according to an embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram illustrating the internal structure of the processor core according to an embodiment of the present invention;

[0021] Figure 5 This is a schematic diagram illustrating how the production line is layered from top to bottom according to an embodiment of the present invention;

[0022] Figure 6 This is a schematic diagram illustrating the performance statistics of layer 0 in an embodiment of the present invention;

[0023] Figure 7 This is a schematic diagram illustrating the performance statistics of the backend bottleneck in the first layer of this invention.

[0024] Figure 8 This is a flowchart illustrating another embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It should be understood that the terms "first," "second," "third," and "fourth," etc., in the claims, specification, and drawings of this invention are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" used in the specification and claims of this invention indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0027] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0028] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection."

[0029] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] Most existing AI accelerators employ a parallel computing design, equipped with dozens or even hundreds of processor cores, each capable of independent computation and possessing its own pipeline. However, as AI accelerators become increasingly complex, inefficiencies can still occur when performing various algorithms due to hardware-software incompatibility. In many cases, pipelines develop "bubbles of idleness," which become performance bottlenecks, preventing the hardware from fully realizing its potential.

[0031] This invention addresses the unique architecture of AI accelerators, particularly proposing a scheme for analyzing pipeline performance, specifically the processor cores. AI accelerator processor cores generally include control, computation, and storage modules. This invention divides the AI ​​accelerator pipeline into multiple events, categorizing them into combinations based on different conditions. These combinations are mutually exclusive and categorized as front-end and back-end bottlenecks. This invention configures multiple performance counters, each associated with one of these combinations. When a combination meets its corresponding conditions, the relevant performance counter counts. Developers can selectively read information from specific performance counters to hierarchically evaluate performance bottlenecks from top to bottom.

[0032] Figure 1 A schematic diagram of the structure of a board 10 according to an embodiment of the present invention is shown. Figure 1 As shown, board 10 includes chip 101, which is a system-on-chip (SoC) integrating one or more combined processing units. These combined processing units are artificial intelligence computing units used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in cloud intelligence. A significant characteristic of cloud intelligence applications is the large volume of input data, placing high demands on the platform's storage and computing capabilities. Board 10 in this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and powerful computing capabilities.

[0033] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. The calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface forms, such as a PCIe interface.

[0034] The board 10 also includes a storage device 104 for storing data, which includes one or more memory cells 105. The storage device 104 is connected to and transmits data with the controller 106 and the chip 101 via a bus. The controller 106 in the board 10 is configured to regulate the state of the chip 101. Therefore, in one application scenario, the controller 106 may include a microcontroller (MCU).

[0035] Figure 2 This is a structural diagram illustrating the combined processing device in chip 101 of this embodiment. (As shown) Figure 2 As shown, the combined processing device 20 includes a computing device 201, an interface device 202, a processing device 203, and off-chip memory 204.

[0036] The computing device 201 is an artificial intelligence accelerator configured to execute user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0037] Interface device 202 is used to transmit data and control commands between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to on-chip storage device of computing device 201. Further, computing device 201 can obtain control commands from processing device 203 via interface device 202 and write them to on-chip control cache of computing device 201. Alternatively or optionally, interface device 202 can also read data from storage device of computing device 201 and transmit it to processing device 203.

[0038] The processing device 203, as a general-purpose processing device, performs basic controls including but not limited to data transfer, and turning on and / or stopping the computing device 201. Depending on the implementation, the processing device 203 may be one or more types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or other general-purpose and / or special-purpose processors.

[0039] The off-chip memory 204 is used to store data to be processed. It is DDR memory, typically 16G or larger, and is used to store data of computing device 201 and / or processing device 203.

[0040] Figure 3A schematic diagram of the internal structure of computing device 201 is shown. Computing device 201 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 201 in the figure adopts a multi-core hierarchical structure design. As a system-on-a-chip, computing device 201 includes multiple clusters, and each cluster includes multiple processor cores. In other words, computing device 201 is constructed in a hierarchical structure of system-on-a-chip, cluster, and processor core.

[0041] From the perspective of system-on-a-chip hierarchy, such as Figure 3 As shown, the computing device 201 includes an external storage controller 301, a peripheral communication module 302, an on-chip interconnect module 303, a synchronization module 304, and multiple clusters 305.

[0042] There can be multiple external storage controllers 301; two are shown as an example in the figure. These controllers are used to access external storage devices, such as those issued by the processor core, in response to access requests from the processor core. Figure 2 The external memory 204 allows for reading or writing data from external memory. The peripheral communication module 302 receives control signals from the processing device 203 via the interface device 202, initiating the computing device 201 to execute tasks. The on-chip interconnect module 303 connects the external storage controller 301, the peripheral communication module 302, and multiple clusters 305, enabling data and control signal transmission between modules. The synchronization module 304 is a global barrier controller (GBC) used to coordinate the work progress of each cluster, ensuring information synchronization. The multiple clusters 305 are the computing core of the computing device 201; four are exemplarily shown in the figure. With hardware development, the computing device 201 of this invention can also include eight, sixteen, sixty-four, or even more clusters 305. The clusters 305 are used to efficiently execute deep learning algorithms.

[0043] From the perspective of cluster hierarchy, such as Figure 3 As shown, each cluster 305 includes multiple processor cores (IPU cores) 306 and one memory core (MEM core) 307.

[0044] Four processor cores 306 are shown exemplarily in the figure, but the present invention does not limit the number of processor cores 306. Its internal architecture is as follows: Figure 4 As shown. Each processor core 306 includes three main modules: a control module 41, an arithmetic module 42, and a storage module 43.

[0045] The control module 41 coordinates and controls the operation of the computation module 42 and the storage module 43 to complete the deep learning task. It includes an instruction fetch unit (IFU) 411, an instruction decode unit (IDU) 412, and a cache 413. The instruction fetch unit 411 fetches instructions from the processing device 203, and the instruction decode unit 412 decodes the fetched instructions and temporarily stores the decoding results (micro-instructions) in the cache 413. At appropriate times, the micro-instructions in the cache 413 are sent as control information to the computation module 42 and the storage module 43.

[0046] The computation module 42 includes a vector operation unit 421 and a matrix operation unit 422. The vector operation unit 421 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit 422 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution.

[0047] Storage module 43 is used to store or move related data, including neuron RAM (NRAM) 431, weight RAM (WRAM) 432, input / output direct memory access (IODMA) 433, and move direct memory access (MVDMA) 434. NRAM 431 is used to store feature maps for computation by processor core 306 and intermediate results after computation; WRAM 432 is used to store the weights of the deep learning network; IODMA 433 controls the access of NRAM 431 / WRAM 432 to external memory 204 through broadcast bus 309; MVDMA 434 controls the access of NRAM 431 / WRAM 432 to SRAM 308.

[0048] Back Figure 3 The storage core 307 is primarily used for storage and communication, namely storing shared data or intermediate results among processor cores 306, and for communication between the execution cluster 305 and off-chip memory 204, communication between clusters 305, and communication between processor cores 306. In other embodiments, the storage core 307 has scalar operation capabilities and is used to perform scalar operations.

[0049] Storage core 307 includes a shared memory unit (SRAM) 308, a broadcast bus 309, a cluster direct memory access (CDMA) module 310, and a global direct memory access (GDMA) module 311. SRAM 308 acts as a high-performance data relay station. Data multiplexed between different processor cores 306 within the same cluster 305 does not need to be obtained from the off-chip memory 204 by each processor core 306 individually. Instead, it is relayed between processor cores 306 via SRAM 308. Storage core 307 only needs to quickly distribute the multiplexed data from SRAM 308 to multiple processor cores 306, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip I / O access.

[0050] Broadcast bus 309, CDMA 310, and GDMA 311 are used to perform communication between processor cores 306, communication between clusters 305, and data transfer between cluster 305 and external memory 204, respectively. These will be explained below.

[0051] The broadcast bus 309 is used to complete high-speed communication between the processor cores 306 within the cluster 305. In this embodiment, the broadcast bus 309 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point (i.e., data transmission from one processor core to another) data transmission. Multicast is a communication method that transmits a piece of data from SRAM 308 to several specific processor cores 306. Broadcast is a communication method that transmits a piece of data from SRAM 308 to all processor cores 306, and is a special case of multicast.

[0052] CDMA 310 is used to control SRAM 308 access between different clusters 305 within the same computing device 201.

[0053] GDMA 311 works in conjunction with external memory controller 301 to control access to external memory 204 from SRAM 308 of cluster 305, or to read data from external memory 204 into SRAM 308. As described above, communication between external memory 204 and NRAM 431 or WRAM 432 can be achieved through two channels. The first channel is direct communication between external memory 204 and NRAM 431 or WRAM 432 via IODAM 433; the second channel involves first transferring data between external memory 204 and SRAM 308 via GDMA 311, and then transferring data between SRAM 308 and NRAM 431 or WRAM 432 via MVDMA 434. Although the second channel appears to require more components and has a longer data flow, in some embodiments, the bandwidth of the second channel is actually much greater than that of the first channel. Therefore, communication between the off-chip memory 204 and the NRAM 431 or WRAM 432 may be more efficient through the second channel. Embodiments of the present invention can select the data transmission channel based on their hardware capabilities.

[0054] In other embodiments, the functions of GDMA 311 and IODMA 433 can be integrated into the same component. For ease of description, GDMA 311 and IODMA 433 are considered different components. For those skilled in the art, any component that performs similar functions and achieves similar technical effects to this invention falls within the scope of protection of this invention. Furthermore, the functions of GDMA 311, IODMA 433, CDMA 310, and MVDMA 434 can also be implemented by the same component.

[0055] This embodiment analyzes the pipeline performance of the processor core 306 in the computing device 201, focusing on the individual and collaborative performance of the control module 41, the arithmetic module 42, and the storage module 43. This embodiment divides the processor core 306 pipeline into multiple layers and multiple events. Developers can, based on the events of interest, hierarchically combine them according to different conditions. These combinations are mutually exclusive and categorized as front-end bottlenecks and back-end bottlenecks, corresponding to the individual or collaborative performance of the control module 41, the arithmetic module 42, and the storage module 43. For example, a front-end bottleneck might include the inability of the control module 41 to issue instructions, while a back-end bottleneck might include the inability of the arithmetic module 42 to perform calculations or the inability of the storage module 43 to access memory.

[0056] Figure 5This diagram illustrates a top-down layered pipeline in this embodiment. The pipeline of processor core 306 is divided into multiple layers from top to bottom, each layer consisting of a front-end and a back-end. As shown, the top layer of the processor core 306 pipeline is layer 0, and the entire pipeline is divided into a front-end F0 and a back-end B0. Layer 1 is the layer below layer 0. The front-end F0 of layer 0 can be further subdivided into front-end F1 and F2 of layer 1, and the back-end B0 of layer 0 can be subdivided into back-end B1 and B2 of layer 1. Layer 2 is the layer below layer 1, and the front-end F1 of layer 1 can be subdivided into front-end F3 and F4 of layer 2. This process continues, and this embodiment can be further divided into N layers, refining the front-end and back-end. Bottlenecks occurring at each layer's front-end and back-end are called front-end bottlenecks and back-end bottlenecks, respectively. Developers can arbitrarily define the front-end and back-end and plan combinations of events based on these definitions. These combinations are mutually exclusive, thus clearly identifying the causes of bottlenecks.

[0057] There may be dozens or even hundreds of events related to processor core 306. While these events are related to performance bottlenecks, they are sometimes not obvious. This embodiment generates combinations based on the logical relationships between multiple events, each combination reflecting an actual performance bottleneck, and these combinations are mutually exclusive.

[0058] Processor core 306 also includes multiple performance counters (not shown), directly configured within control module 41, arithmetic module 42, and storage module 43, respectively, and associated with multiple combinations of pipelines in control module 41, arithmetic module 42, and storage module 43. More specifically, these performance counters are associated with combinations obtained from the pipeline layering of processor core 306; performance counters at the same layer are grouped into the performance counters of the previous layer based on the front and back ends of the pipeline, and then expanded into the performance counters of the next layer. Figure 5 For example, the performance counters of front-end F1 and front-end F2 in layer 1 are summarized into the performance counter of front-end F0 in layer 0, while the performance counter of front-end F1 in layer 1 is expanded to include the performance counters of front-end F3 and front-end F4 in layer 2.

[0059] The events in this embodiment include running events and idle events caused by specific factors, such as control pipeline events subdivided by control pipeline time in control module 41, operation pipeline events subdivided by operation type in operation module 42, and memory access pipeline events subdivided by memory access type or memory access location in storage module 43.

[0060] This embodiment exemplarily includes at least performance counters at layers 0 to 2. The layer 0 performance counter is associated with a combination of instruction issuance events of the control module 41, and operation memory access events of the arithmetic module 42 and the storage module 43. The layer 1 performance counter is the lower-level expanded performance counter of the layer 0 performance counter, and is associated with instruction fetch blocking events and downstream blocking events in the instruction issuance events, as well as a combination of operation events of the arithmetic module 42 and memory access events of the storage module 43. The layer 2 performance counter is the lower-level expanded performance counter of the layer 1 performance counter, and is associated with a combination of at least one of the following events: at least one of convolution operation events and scalar operation events and vector operation events in the arithmetic module 42; memory access events of on-chip or off-chip storage in the storage module 43; and data input / output events or data transfer events in the storage module 43. The second-level performance counters are also associated with instruction fetch blocking events and downstream instruction blocking events in the computation events, instruction fetch blocking events and downstream instruction blocking events in the memory access events, and combinations of events related to the computation time of the computation module 42 as a percentage of the total time, or the memory access time of the storage module 43 as a percentage of the total time. The relationship between each level of events and the performance counters will be explained in detail below.

[0061] Figure 6 This diagram illustrates the performance statistics timing for Layer 0. Since Layer 0 is the topmost layer, it also reflects the performance event combinations of the overall pipeline 60 of processor core 306. In this embodiment, Layer 0 is defined as the instruction issuance event 601 of control module 41 and the memory access event 602 of arithmetic module 42 or memory module 43. Figure 6 As shown, instruction issuance event 601 and computation / memory access event 602 divide the entire pipeline 60 into four event combinations. The first combination refers to instruction issuance event 601 running while computation / memory access event 602 is idle; the second combination refers to instruction issuance event 601 being idle while computation / memory access event 602 is idle; the third combination refers to instruction issuance event 601 being idle while computation / memory access event 602 runs; and the fourth combination refers to instruction issuance event 601 running while computation / memory access event 602 runs, which is the union of instruction issuance event 601 and computation / memory access event 602. After this grouping, the combinations at level 0 are mutually exclusive, meaning that the first, second, third, and fourth combinations do not overlap.

[0062] Based on the above grouping, this embodiment defines the first and second combinations as the front-end bottlenecks of layer 0, and the third and fourth combinations as the back-end bottlenecks of layer 0. Each combination in layer 0 is configured with a performance counter to calculate the average occurrence rate of that combination. The performance counters in layer 0 include first to fourth performance counters, wherein the first performance counter is used to count the front-end bottlenecks of layer 0, and counts when the condition of the first combination (instruction issuance event 601 is running and computation memory access event 602 is idle) is met; the second performance counter is used to count the front-end bottlenecks of layer 0, and counts when the condition of the second combination (instruction issuance event 601 is idle and computation memory access event 602 is idle) is met; the third performance counter is used to count the back-end bottlenecks of layer 0, and counts when the condition of the third combination (instruction issuance event 601 is idle and computation memory access event 602 is running) is met; the fourth performance counter is used to count the back-end bottlenecks of layer 0, and counts when the condition of the fourth combination (instruction issuance event 601 is running and computation memory access event 602 is running) is met.

[0063] In summary, layer 0 of this embodiment uses four performance counters to count four mutually exclusive event combinations.

[0064] Layer 1 is the layer below Layer 0, and it is also divided into front-end bottlenecks and back-end bottlenecks. Specifically, the front-end bottlenecks in Layer 1 are a further subdivision of the front-end bottlenecks in Layer 0. These bottlenecks include instruction fetch blocking events and downstream blocking events, such as cache misses during instruction fetch leading to excessively long pipeline interruption times and branch prediction errors, undoing register dependencies, excessive latency in the transmission of synchronization instructions within or between cores, and improper configuration of synchronization instructions. The back-end bottlenecks in Layer 1 are a further subdivision of the back-end bottlenecks in Layer 0. These bottlenecks are combinations of computation events from the arithmetic module 42 and memory access events from the storage module 43, such as the running and idle states of the arithmetic module 42 and / or the storage module 43. Table 1 shows the combinations in Layer 1.

[0065]

[0066]

[0067] Table 1

[0068] This embodiment configures 7 performance counters for the first layer of combinations: the fifth to the eleventh performance counters, which count the fifth to eleventh combinations respectively.

[0069] The fifth performance counter is used to count the front-end bottleneck of the first layer. Under the premise that the first performance counter counts, the fifth performance counter is associated with the fifth combination, namely the situation where the arithmetic module 42 and the storage module 43 cannot receive instructions because the instruction queue is full. Specifically, the fifth performance counter counts when the following conditions are met: microinstructions are being sent in the cache 413, and the instruction queue of the arithmetic module 42 or the storage module 43 is full.

[0070] The sixth performance counter is used to count the front-end bottleneck of the first layer. Under the premise that the first performance counter counts, the sixth performance counter is associated with the sixth combination, that is, the arithmetic module 42 and the storage module 43 cannot receive instructions due to synchronization reasons. Specifically, the sixth performance counter counts when the following conditions are met: microinstructions in the cache 413 are being sent, and the arithmetic module 42 or the storage module 43 is in a synchronized state.

[0071] The seventh performance counter is used to count the front-end bottleneck of the first layer. Under the premise that the second performance counter counts, the seventh performance counter is associated with the seventh combination, that is, when a buffer loss occurs during instruction fetch, resulting in excessive pipeline interruption time. Specifically, when the condition of buffer loss or branch prediction error causing pipeline interruption time to exceed the threshold is met, the seventh performance counter counts.

[0072] The eighth performance counter is used to count the front-end bottleneck of the first layer. Under the premise that the second performance counter counts, the eighth performance counter is associated with the eighth combination, namely instruction wait register dedependency. Specifically, the eighth performance counter counts when the condition that the pipeline interruption time exceeds the threshold is met due to instruction wait register dedependency.

[0073] Figure 7 The diagram shows the performance statistics of the backend bottleneck in layer 1. The backend in layer 1 is related to computation event 701 and memory access event 702. As shown in the figure, the backend bottleneck in layer 1 is mutually exclusive into the ninth group, the tenth group and the eleventh group.

[0074] The ninth performance counter is used to count the back-end bottleneck of the first layer. Under the premise that the third or fourth performance counter counts, the ninth performance counter is associated with the ninth combination, that is, the operation event and the memory access event run simultaneously. Specifically, the ninth performance counter counts when the conditions for the operation module 42 to run and the storage module 43 to run are both met.

[0075] The tenth performance counter is used to count the backend bottleneck of the first layer. Under the premise that the third or fourth performance counter counts, the tenth performance counter is associated with the tenth combination, that is, the operation event is running and the memory access event is idle. Specifically, the tenth performance counter counts when the conditions of the operation module 42 running and the storage module 43 being idle are met.

[0076] The eleventh performance counter is used to count the backend bottleneck of the first layer. Under the premise that the third or fourth performance counter counts, the eleventh performance counter is associated with the eleventh combination, that is, the operation event is idle and the memory access event is running. Specifically, when the conditions of the operation module 42 being idle and the storage module 43 running are met, the eleventh performance counter counts.

[0077] In summary, the first layer of this embodiment uses seven performance counters to count seven mutually exclusive event combinations.

[0078] Layer 2 is the layer below Layer 1, and it is also divided into front-end bottlenecks and back-end bottlenecks. That is, the front-end bottleneck of Layer 2 is a further subdivision of the front-end bottleneck of Layer 1, and the back-end bottleneck of Layer 2 is a further subdivision of the back-end bottleneck of Layer 1. For example, the back-end computation event bottleneck of Layer 1 can be further subdivided into two mutually exclusive back-end bottlenecks in Layer 2: whether memory access events are blocked and whether the computation events themselves are efficient enough. Memory access events may be blocked due to instruction issuance bottlenecks (such as insufficient instruction queue depth in storage module 43) and the inability to issue instructions due to synchronization issues. The back-end memory access event bottleneck of Layer 1 can be further subdivided into two mutually exclusive back-end bottlenecks in Layer 2: whether computation events are blocked and whether the memory access events themselves are efficient enough. Since the back-end bottleneck of Layer 2 focuses on the details of computation events and memory access events, the performance counter count for the back-end bottleneck of Layer 2 is based on the count of any one of the ninth, tenth, or eleventh performance counters. Table 2 shows some combinations of the back-end of Layer 2 in this embodiment.

[0079]

[0080] Table 2

[0081] This embodiment configures 10 performance counters for the backend of the second layer in Table 2: the twelfth performance counter to the twenty-first performance counter, which count the twelfth to the twenty-first combinations respectively.

[0082] The twelfth performance counter is associated with the twelfth combination, which is when a cache miss occurs during instruction fetching, causing excessive pipeline interruption time. Under the premise that the tenth performance counter is counting, the twelfth performance counter counts when the idle time of the arithmetic module 42 or the storage module 43 exceeds the threshold and the conditions for cache miss of 413 are met.

[0083] The thirteenth performance counter is associated with the thirteenth combination, namely instruction wait register dedependency. Under the premise that the tenth performance counter counts, the thirteenth performance counter counts when the conditions that the idle time of the dedependency event of storage module 43 exceeds the threshold are met.

[0084] The fourteenth performance counter is associated with the fourteenth combination, that is, when the storage module 43 cannot receive instructions because the instruction queue is full, the fourteenth performance counter counts, provided that the tenth performance counter is counting, and when the cache 413 is sending memory access micro-instructions and the conditions that the queue of the storage module 43 is full are met.

[0085] The fifteenth performance counter is associated with the fifteenth combination, namely, when the storage module 43 cannot receive instructions due to synchronization reasons, the fifteenth performance counter counts under the premise that the tenth performance counter counts, and the following conditions are met: the memory access microinstruction of the cache 413 is being sent; the storage module 43 cannot receive the microinstruction; the storage module 43 is in a synchronization state.

[0086] The sixteenth performance counter is used to associate with the sixteenth combination, namely the number of instruction execution cycles of the arithmetic module 42 / the total number of cycles. Under the premise that the first, second and tenth performance counters are counting simultaneously, the sixteenth performance counter counts when the condition that the time for the arithmetic module 42 to run instructions accounts for at least a certain proportion of the total time is met.

[0087] The seventeenth performance counter is associated with the seventeenth combination, which is when a buffer loss occurs during instruction fetching, causing excessive pipeline interruption time. Under the premise that the eleventh performance counter counts, when the idle time of the arithmetic module 42 exceeds the threshold and all conditions for buffer loss are met, the seventeenth performance counter counts.

[0088] The eighteenth performance counter is associated with the eighteenth combination, namely the instruction wait register dedependency of the arithmetic module 42. Under the premise that the eleventh performance counter counts, the eighteenth performance counter counts when the idle time of the dedependency event of the arithmetic module 42 exceeds the threshold condition is met.

[0089] The nineteenth performance counter is associated with the nineteenth combination, that is, when the arithmetic module 42 cannot receive instructions because the instruction queue is full, and the conditions of the arithmetic microinstruction being sent in the cache 413 and the queue of the arithmetic module 42 being full are met, the nineteenth performance counter counts.

[0090] The twentieth performance counter is associated with the twentieth combination, that is, the operation module 42 cannot receive instructions due to synchronization reasons. Under the premise that the eleventh performance counter counts, the twentieth performance counter counts when the following conditions are met: the operation microinstruction of the buffer 413 is being sent; the operation module 42 cannot receive the operation microinstruction; the operation module 42 is in a synchronization state.

[0091] The 21st performance counter is associated with the 21st combination, namely the number of instruction execution cycles of storage module 43 / total number of cycles. Under the premise that the first, second and eleventh performance counters are counting simultaneously, the 21st performance counter counts when the condition that the time for storage module 43 to run instructions accounts for at least a certain proportion of the total time is met.

[0092] In summary, the backend of the second layer in this embodiment exemplarily utilizes 10 performance counters to count 10 mutually exclusive event combinations respectively.

[0093] In this embodiment, the performance counter counts are used to reflect the average occurrence rate of the processor core 306 pipeline during operation. For example, if the average occurrence rate of front-end F0 at layer 0 is too low, the hardware usage of the layer below front-end F0 at layer 0 can be further examined, that is, the average occurrence rates of front-end F1 and F2 at layer 1 can be checked. If it is found that the average occurrence rate of F2 at layer 1 is too low, the average occurrence rates of front-end F5 and F6 at layer 2 can be further examined. By tracing down in this way, the key cause of the performance bottleneck can be clearly identified.

[0094] More specifically, over a period of time (e.g., 100 cycles), when a combination meets certain conditions, a corresponding performance counter is counted, and the count value represents the average occurrence rate of that event combination during this period. Developers can determine the number of performance counters based on hardware resources and appropriately select which performance counter information to read, performing a top-down hierarchical evaluation of performance bottlenecks. Because this embodiment hierarchically layers the pipeline from top to bottom, the performance counters are also distributed from top to bottom to analyze performance bottlenecks hierarchically.

[0095] For example, developers would expect Figure 7 The higher the average occurrence rate of the ninth combination, the better. This is because the simultaneous operation of the arithmetic module 42 and the storage module 43 indicates that arithmetic events and memory access events are parallel, and the higher the degree of parallelism, the higher the efficiency of the processor core 306. Therefore, developers can choose to read the count value of the ninth performance counter to determine whether its average occurrence rate meets expectations.

[0096] The above combinations are merely examples. Those skilled in the art can deduce other mutually exclusive combinations of events at the same level, such as the front-end event combinations in the second layer mentioned above. They can also continue to layer down, such as front-end bottlenecks and back-end bottlenecks in the third layer. This embodiment does not limit the combinations or the number of layers.

[0097] This embodiment is designed for an artificial intelligence accelerator. By setting multiple performance counters on the hardware and selecting the combination of events to be monitored, the hardware does not need to be reconfigured. The pipeline unfolds in a multi-layered manner from top to bottom, with each layer's combination being mutually exclusive and divided into front-end bottlenecks and back-end bottlenecks, allowing for logical and rapid identification of performance bottlenecks.

[0098] Another embodiment of the present invention is a method for performance analysis of artificial intelligence accelerators, specifically for... Figures 1 to 4 The pipeline performance of the processor core 306 in the computing device 201 is analyzed, focusing on the individual and collaborative performance of the control module 41, the arithmetic module 42, and the storage module 43. This embodiment also divides the pipeline of the processor core 306 into multiple layers and multiple events. Developers can, based on the events of interest, hierarchically combine them according to different conditions. These combinations are mutually exclusive and categorized as front-end bottlenecks and back-end bottlenecks, corresponding to the individual or collaborative performance of the control module 41, the arithmetic module 42, and the storage module 43. Figure 8 A flowchart illustrating this embodiment is shown.

[0099] In step 801, the pipeline of processor core 306 is divided into multiple events. The events in this embodiment include running events and idle events caused by specific factors, such as control pipeline events subdivided by control pipeline time in control module 41, arithmetic pipeline events subdivided by arithmetic type in arithmetic module 42, and memory access pipeline events subdivided by memory access type or memory access location in memory module 43.

[0100] This embodiment divides the pipeline into at least three layers: Layer 0, Layer 1, and Layer 2. Layer 0 is the top layer, reflecting the performance event combination of the overall pipeline of processor core 306. In this embodiment, Layer 0 is defined as the front-end bottleneck of instruction issuance events of control module 41, and the back-end bottleneck of computation and memory access events of arithmetic module 42 or storage module 43.

[0101] Layer 1 is the layer below Layer 0, and it is also divided into front-end bottlenecks and back-end bottlenecks. Specifically, the front-end bottlenecks in Layer 1 are a further subdivision of the front-end bottlenecks in Layer 0. The front-end bottlenecks in Layer 1 are instruction fetch blocking events and downstream blocking events, such as buffer loss during instruction fetching leading to excessively long pipeline interruption time and branch prediction errors, undoing register dependencies, excessively long transmission delays of synchronization instructions within or between cores, and unreasonable configuration of synchronization instructions.

[0102] The second layer is the layer below the first layer, and includes events such as convolution operation events, scalar operation events, and vector operation events in the operation module 42; memory access events for on-chip or off-chip storage in the storage module 43; data input / output events or data transfer events in the storage module 43; instruction fetch blocking events and downstream instruction blocking events in the operation events; instruction fetch blocking events and downstream instruction blocking events associated with memory access events; operation time of the operation module 42 as a percentage of the total time; and memory access time of the storage module 43 as a percentage of the total time.

[0103] In step 802, multiple combinations of events are considered. These combinations are mutually exclusive and categorized into front-end bottlenecks and back-end bottlenecks. Examples include: a combination of instruction issuance events from the control module 41 at layer 0, and memory access events from the computation module 42 and storage module 43; a combination of instruction fetch blocking events and downstream blocking events from the instruction issuance events at layer 1, and a combination of computation events from the computation module 42 and memory access events from the storage module 43; at least one of convolution operation events, scalar operation events, and vector operation events from the computation module 42 at layer 2; memory access events from on-chip or off-chip storage in the storage module 43; data input / output events or data transfer events in the storage module 43; combinations of instruction fetch blocking events and downstream blocking events from the computation events, instruction fetch blocking events and downstream blocking events from the memory access events, the proportion of computation time of the computation module 42 to the total time, or the proportion of memory access time of the storage module 43 to the total time.

[0104] For example, this embodiment specifically divides the production line into the following combinations.

[0105] The first group refers to a situation where the instruction issuance event is running and the computation / memory access event is idle; the second group refers to a situation where both the instruction issuance event and the computation / memory access event are idle; the third group refers to a situation where both the instruction issuance event and the computation / memory access event are running; and the fourth group refers to a situation where both the instruction issuance event and the computation / memory access event are running. After this grouping, the combinations at level 0 are mutually exclusive, meaning that the first, second, third, and fourth groups do not overlap. Based on the above grouping, this embodiment defines the first and second groups as the front-end bottlenecks of level 0, and the third and fourth groups as the back-end bottlenecks of level 0.

[0106] The front-end bottlenecks in Layer 1 include the fifth combination, namely, the situation where the arithmetic module 42 and the storage module 43 cannot receive instructions due to a full instruction queue; the sixth combination, namely, the situation where the arithmetic module 42 and the storage module 43 cannot receive instructions due to synchronization reasons; the seventh combination, namely, the situation where buffer loss occurs during instruction fetching, resulting in excessively long pipeline interruption time; and the eighth combination, namely, the situation where instructions wait for register dedependency.

[0107] The backend bottleneck at level 1 is a further subdivision of the backend bottleneck at level 0. The backend bottleneck at level 1 is a combination of computation events from the computation module 42 and memory access events from the storage module 43, such as the running and idle states of the computation module 42 and / or the storage module 43. The backend bottlenecks at level 1 include the ninth combination, where computation events and memory access events run simultaneously; the tenth combination, where computation events run and memory access events are idle; and the eleventh combination, where computation events are idle and memory access events run.

[0108] Layer 2 is the layer below Layer 1, and it is also divided into front-end bottlenecks and back-end bottlenecks. That is, the front-end bottleneck in Layer 2 is a further subdivision of the front-end bottleneck in Layer 1, and the back-end bottleneck in Layer 2 is a further subdivision of the back-end bottleneck in Layer 1. For example, the back-end computation event bottleneck in Layer 1 can be further subdivided into two mutually exclusive back-end bottlenecks in Layer 2: whether memory access events are blocked and whether the computation events themselves are efficient enough. Memory access events may be blocked due to instruction issuance bottlenecks (such as insufficient instruction queue depth in storage module 43) and synchronization issues preventing instruction issuance. Similarly, the back-end memory access event bottleneck in Layer 1 can be further subdivided into two mutually exclusive back-end bottlenecks in Layer 2: whether computation events are blocked and whether the memory access events themselves are efficient enough.

[0109] The backend of Layer 2 includes: the twelfth combination, i.e., a buffer loss during instruction fetching causing excessive pipeline interruption time; the thirteenth combination, i.e., instruction waiting register dedependency; the fourteenth combination, i.e., storage module 43 cannot receive instructions because the instruction queue is full; the fifteenth combination, i.e., storage module 43 cannot receive instructions due to synchronization reasons; the sixteenth combination, i.e., the number of instruction execution cycles of arithmetic module 42 / total number of cycles; the seventeenth combination, i.e., a buffer loss during instruction fetching causing excessive pipeline interruption time; the eighteenth combination, i.e., instruction waiting register dedependency of arithmetic module 42; the nineteenth combination, i.e., arithmetic module 42 cannot receive instructions because the instruction queue is full; the twentieth combination, i.e., arithmetic module 42 cannot receive instructions due to synchronization reasons; and the twenty-first combination, i.e., the number of instruction execution cycles of storage module 43 / total number of cycles.

[0110] The above combinations are merely examples; those skilled in the art can deduce other mutually exclusive combinations of events at the same level.

[0111] In step 803, performance counters are configured to be associated with multiple combinations. These performance counters are directly set within the control module 41, the arithmetic module 42, and the storage module 43, and are respectively associated with the aforementioned combinations.

[0112] More specifically, configure the first performance counter to be associated with the first group; configure the second performance counter to be associated with the second group; configure the third performance counter to be associated with the third group; configure the fourth performance counter to be associated with the fourth group; configure the fifth performance counter to be associated with the fifth group; configure the sixth performance counter to be associated with the sixth group; configure the seventh performance counter to be associated with the seventh group; configure the eighth performance counter to be associated with the eighth group; configure the ninth performance counter to be associated with the ninth group; configure the tenth performance counter to be associated with the tenth group; configure the eleventh performance counter to be associated with the eleventh group; configure the twelfth performance counter to be associated with the twelfth group; configure the thirteenth performance counter to be associated with the thirteenth group; configure the fourteenth performance counter to be associated with the fourteenth group; configure the fifteenth performance counter to be associated with the fifteenth group; configure the sixteenth performance counter to be associated with the sixteenth group; configure the seventeenth performance counter to be associated with the seventeenth group; configure the eighteenth performance counter to be associated with the eighteenth group; configure the nineteenth performance counter to be associated with the nineteenth group; configure the twentieth performance counter to be associated with the twentieth group; configure the twenty-first performance counter to be associated with the twenty-first group.

[0113] Developers can determine the number of performance counters based on hardware resources and appropriately select which performance counter information to read.

[0114] In step 804, it is determined whether multiple combinations meet the corresponding conditions. Specifically, at level 0, it is determined whether the condition that the instruction issuance event of the first combination is running and the operation memory access event is idle is met; whether the condition that the instruction issuance event of the second combination is idle and the operation memory access event is idle is met; whether the condition that the instruction issuance event of the third combination is idle and the operation memory access event is running is met; and whether the condition that the instruction issuance event of the fourth combination is running and the operation memory access event is running is met.

[0115] In the first layer, it is determined whether the conditions are met for micro-instruction sending in the cache 413 of the fifth combination and the instruction queue of the arithmetic module 42 or the storage module 43 being full; whether the conditions are met for micro-instruction sending in the cache 413 of the sixth combination and the arithmetic module 42 or the storage module 43 being in a synchronized state; whether the conditions are met for the seventh combination where a buffer loss or branch prediction error causes pipeline interruption time to exceed a threshold; whether the conditions are met for the eighth combination where instruction waiting for register dedependency causes pipeline interruption time to exceed a threshold; whether the conditions are met for the ninth combination where the arithmetic module 42 is running and the storage module 43 is running; whether the conditions are met for the tenth combination where the arithmetic module 42 is running and the storage module 43 is idle; and whether the conditions are met for the eleventh combination where the arithmetic module 42 is idle and the storage module 43 is running.

[0116] In the second layer, it is determined whether the idle time of the twelfth combination's operation module 42 or storage module 43 exceeds a threshold and the cache 413 is lost; whether the idle time of the disdependency event of the thirteenth combination's storage module 43 exceeds a threshold; whether the conditions are met for the fourteenth combination's cache 413 to be sending memory access micro-instructions and the queue of storage module 43 to be full; whether the conditions are met for the fifteenth combination's cache 413 to be sending memory access micro-instructions, storage module 43 to be unable to receive the micro-instructions, and storage module 43 to be in a synchronized state; and whether the time spent by the sixteenth combination's operation module 42 running instructions accounts for at least a specific proportion of all time. The conditions are as follows: whether the idle time of the seventeenth combination's operation module 42 exceeds the threshold and the cache loss occurs; whether the idle time of the disdependency event of the eighteenth combination's operation module 42 exceeds the threshold; whether the operation micro-instruction of the nineteenth combination's cache 413 is being sent and the queue of the operation module 42 is full; whether the operation micro-instruction of the twentieth combination's cache 413 is being sent, the operation module 42 cannot receive operation micro-instructions, and the operation module 42 is in a synchronization state; and whether the time spent by the twenty-first combination's storage module 43 running instructions accounts for at least a specific proportion of all time.

[0117] For any combination, if the corresponding condition is not met, step 805 is executed to stop the corresponding performance counter from counting; that is, the corresponding performance counter does not count. If the corresponding condition is met, step 806 is executed to start the corresponding performance counter from counting.

[0118] In step 807, front-end and back-end bottlenecks are evaluated based on the performance counter counts. In this embodiment, the performance counter counts reflect the average occurrence rate of the pipeline of processor core 306 during operation. More specifically, this embodiment can repeatedly execute steps 804 to 806 over a period of time (e.g., 100 cycles). When a combination meets a corresponding condition, the corresponding performance counter counts, and the count value represents the average occurrence rate of that event combination during this period. Because this embodiment layers the pipeline from top to bottom, the performance counters are also distributed from top to bottom, facilitating layered analysis of front-end and back-end bottlenecks by those skilled in the art.

[0119] This embodiment targets an artificial intelligence accelerator, which utilizes performance counters to select the combination of events of interest. Each analysis does not require hardware reconfiguration, and its pipeline unfolds in multiple layers from top to bottom. The combinations of each layer are mutually exclusive and are divided into front-end bottlenecks and back-end bottlenecks, which can logically and quickly locate performance bottlenecks.

[0120] Another embodiment of the present invention is a computer-readable storage medium storing computer program code for a method of analyzing the performance of an artificial intelligence accelerator. When the computer program code is run by a processor, it executes the methods described in the preceding embodiments. In some implementation scenarios, the integrated units described above can be implemented as software program modules. If implemented as software program modules and sold or used as independent products, the integrated units can be stored in a computer-readable memory. Based on this, when the solution of the present invention is embodied in the form of a software product (e.g., a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to cause a computer device (e.g., a personal computer, a server, or a network device) to execute some or all of the steps of the methods described in the embodiments of the present invention. The aforementioned memory may include, but is not limited to, various media capable of storing program code, such as USB flash drives, flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0121] The present invention has the following technical advantages: Firstly, it configures performance counters for event combinations that may cause performance bottlenecks. Each time a performance analysis is performed, the desired combination can be selected. Each analysis does not require hardware reconfiguration; only the specific performance counter needs to be read. This makes the performance analysis scheme of the present invention quite lightweight and flexible. Secondly, the performance analysis scheme of the present invention unfolds in a multi-layered manner from top to bottom. Events at each layer are mutually exclusive and divided into front-end and back-end bottlenecks, allowing for logical and rapid location of performance bottlenecks. Thirdly, the present invention defines event combinations specifically for the unique structure of AI accelerators, enabling direct discovery of performance bottlenecks in AI accelerators.

[0122] Depending on the application scenario, the electronic devices or apparatus of the present invention may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The electronic devices or apparatus of the present invention can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare. Furthermore, the electronic devices or apparatus of the present invention can also be used in application scenarios related to artificial intelligence, big data, and / or cloud computing, such as cloud computing, edge computing, and terminal computing. In one or more embodiments, the high-computing-power electronic devices or apparatuses according to the present invention can be applied to cloud devices (e.g., cloud servers), while the low-power electronic devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling and collaborative work of end-to-cloud or cloud-edge-end integration.

[0123] It should be noted that, for the sake of brevity, this invention describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solution of this invention is not limited to the order of the described actions. Therefore, based on the disclosure or teachings of this invention, those skilled in the art will understand that some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this invention can be considered as optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions of this invention. In addition, depending on the solution, the description of some embodiments of this invention also has different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this invention can also refer to the relevant descriptions of other embodiments.

[0124] In terms of specific implementation, based on the disclosure and teachings of this invention, those skilled in the art will understand that the several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various units in the electronic device or device embodiments described above, this document has divided them based on logical functions, but in actual implementation, there may be other ways of division. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection utilizing an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

[0125] In this invention, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same position or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this invention. Additionally, in some scenarios, multiple units in the embodiments of this invention may be integrated into one unit or each unit may exist physically independently.

[0126] In other implementation scenarios, the integrated units described above can also be implemented in hardware, i.e., as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the circuit's hardware structure may include, but is not limited to, physical devices, which may include, but are not limited to, transistors or memristors. Therefore, the various devices described herein (e.g., computing devices or other processing devices) can be implemented using appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage units or storage devices can be any suitable storage medium (including magnetic storage media or magneto-optical storage media), such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM.

[0127] The foregoing can be better understood in accordance with the following terms:

[0128] Clause A1. A processor core for analyzing pipeline performance, comprising a control module, an arithmetic module, and a storage module, wherein the control module, the arithmetic module, and the storage module correspond to multiple events in the pipeline, the multiple events being combined according to different conditions, the multiple combinations being mutually exclusive and divided into front-end bottlenecks and back-end bottlenecks, characterized in that the processor core further comprises multiple performance counters, each associated with one of the multiple combinations, wherein when the multiple combinations meet corresponding conditions, the corresponding performance counter counts.

[0129] Clause A2. The processor core according to Clause A1, wherein the performance counter is associated with a combination of multiple events in multiple layers obtained by the pipeline layering, and the performance counter of a single layer is summarized into the performance counter of the previous layer and expanded into the performance counter of the next layer according to the pipeline.

[0130] Clause A3. The processor core as described in Clause A2, wherein the plurality of events includes running events and idle events caused by specific factors.

[0131] Clause A4. The processor core as described in Clause A2, wherein the plurality of events includes control pipeline events subdivided by control pipeline time, arithmetic pipeline events subdivided by arithmetic type, and memory pipeline events subdivided by memory access type or memory access location.

[0132] Clause A5. For the processor core described in Clause A1, the front-end bottleneck includes the inability to issue instructions, and the back-end bottleneck includes the inability to perform computation or access memory.

[0133] Clause A6. The processor core according to Clause A1, wherein the plurality of performance counters includes a level 0 performance counter, the level 0 performance counter being associated with a combination of instruction issuance events of the control module, computational memory access events of the computational module and the storage module.

[0134] Clause A7. The processor core according to Clause A6, wherein the plurality of performance counters includes a first-level performance counter, the first-level performance counter being a lower-level expanded performance counter of the 0th-level performance counter, the first-level performance counter being associated with a combination of an instruction fetch blocking event and a downstream blocking event in the instruction dispatch event.

[0135] Clause A8. The processor core according to Clause A6, wherein the plurality of performance counters includes a first-level performance counter, the first-level performance counter being a lower-level expanded performance counter of the 0th-level performance counter, the first-level performance counter being associated with a combination of computation events of the computing module and memory access events of the storage module.

[0136] Clause A9. The processor core according to Clause A8, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of at least one of the following events: a convolution operation event and at least one of a scalar operation event and a vector operation event in the computation module; a memory access event of on-chip or off-chip storage in the storage module; a data input / output event or a data transfer event in the storage module.

[0137] Clause A10. The processor core according to Clause A7, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of instruction fetch blocking events and instruction downstream blocking events in the computation event.

[0138] Clause A11. The processor core according to Clause A7, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of instruction fetch blocking events and instruction downstream blocking events in the memory access event.

[0139] Clause A12. The processor core according to Clause A7, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of a computation time ratio of the computation module to the total time or a memory access time ratio of the memory module to the total time.

[0140] Clause A13. The processor core according to Clause A6, wherein the plurality of performance counters includes a first performance counter for counting the front-end bottleneck of the layer 0, the first performance counter counting when the instruction issue event is executed and the condition of the operation memory access event being idle is met.

[0141] Clause A14. The processor core according to Clause A6, wherein the plurality of performance counters includes a second performance counter for counting the front-end bottleneck of the 0th layer, the second performance counter counting when the conditions of the instruction issuance event being idle and the computation memory access event being idle are met.

[0142] Clause A15. The processor core according to Clause A8, wherein the plurality of performance counters includes a third performance counter for counting the back-end bottleneck of the 0th layer, the third performance counter counting when the instruction issuance event is idle and the conditions for the operation memory access event to run are met.

[0143] Clause A16. The processor core according to Clause A15, wherein the plurality of performance counters includes a fourth performance counter for counting the back-end bottleneck of the 0th layer, the fourth performance counter counting when the instruction issue event is executed and the conditions for the operation memory access event are met.

[0144] Clause A17. The processor core according to Clause A13, wherein the control module includes a cache for storing microinstructions, and the plurality of performance counters includes a fifth performance counter for counting the front-end bottleneck of the first layer, wherein, provided that the first performance counter is counting, the fifth performance counter counts when the following conditions are met: the microinstruction in the cache is being sent; the instruction queue of the arithmetic module or the storage module is full.

[0145] Clause A18. The processor core according to Clause A13, wherein the control module includes a cache for storing microinstructions, and the plurality of performance counters includes a sixth performance counter for counting the front-end bottleneck of the first layer, wherein, provided that the first performance counter is counting, the sixth performance counter counts when the following conditions are met: the microinstruction in the cache is being sent; the arithmetic module or the storage module is in a synchronization state.

[0146] Clause A19. The processor core according to Clause A14, wherein the plurality of performance counters includes a seventh performance counter for counting the front-end bottleneck of the first layer, wherein, provided that the second performance counter counts, the seventh performance counter counts when the following condition is met: a cache miss or branch prediction error occurs causing the pipeline interruption time to exceed a threshold.

[0147] Clause A20. The processor core according to Clause A14, wherein the plurality of performance counters includes an eighth performance counter for counting the front-end bottleneck of the first layer, wherein the eighth performance counter counts on the premise that the second performance counter counts, and the following condition is met: instruction wait register dedependency causes the pipeline interruption time to exceed a threshold.

[0148] Clause A21. The processor core according to Clause A16, wherein the plurality of performance counters includes a ninth performance counter for counting the back-end bottleneck of the first layer, wherein the ninth performance counter counts on the premise that the third or fourth performance counter counts, and the following conditions are met: the operation event runs; the memory access event runs.

[0149] Clause A22. The processor core according to Clause A16, wherein the plurality of performance counters includes a tenth performance counter for counting the back-end bottleneck of the first layer, wherein the tenth performance counter counts on the premise that the third performance counter counts, and the following conditions are met: the computation event is running; the memory access event is idle.

[0150] Clause A23. The processor core according to Clause A16, wherein the plurality of performance counters includes an eleventh performance counter for counting the back-end bottleneck of the first layer, wherein the eleventh performance counter counts provided that the third performance counter counts, and the following conditions are met: the computation event is idle; the memory access event is running.

[0151] Clause A24. An artificial intelligence accelerator comprising a processor core as described in any one of Clauses A1 to A23.

[0152] Clause A25. An integrated circuit device including an artificial intelligence accelerator as described in Clause A24.

[0153] Clause A26. A board including an integrated circuit device as described in Clause A25.

[0154] Clause A27. A method for performance analysis of an artificial intelligence accelerator, the artificial intelligence accelerator including a processor core, characterized in that the method includes: dividing the pipeline of the processor core into multiple events; forming multiple combinations based on the multiple events, the multiple combinations being mutually exclusive and divided into front-end bottlenecks and back-end bottlenecks; configuring performance counters associated with the multiple combinations; determining whether the multiple combinations meet corresponding conditions; if so, driving the corresponding performance counters to count; and evaluating the front-end bottlenecks and the back-end bottlenecks based on the count values ​​of the performance counters.

[0155] Clause A28. The method according to Clause A27, wherein the single-layer performance counter is summarized to the performance counter of the previous layer and expanded to the performance counter of the next layer according to the pipeline.

[0156] Clause A29. The method described in Clause A28, wherein the plurality of events includes operational events and idle events caused by specific factors.

[0157] Clause A30. The method described in Clause A28, wherein the plurality of events includes control pipeline events segmented by control pipeline time, operation pipeline events segmented by operation type, and memory pipeline events segmented by memory access type or memory access location.

[0158] Clause A31. The method described in Clause A27, wherein the front-end bottleneck includes the inability to issue instructions, and the back-end bottleneck includes the inability to perform computation or access memory.

[0159] Clause A32. The method according to Clause A27, wherein the plurality of performance counters includes a level 0 performance counter, the level 0 performance counter being associated with a combination of instruction issuance events of the control module, computational memory access events of the computational module and the storage module.

[0160] Clause A33. The method according to Clause A32, wherein the plurality of performance counters includes a first-level performance counter, the first-level performance counter being a lower-level expanded performance counter of the 0th-level performance counter, the first-level performance counter being associated with a combination of an instruction fetch blocking event and a downstream blocking event in the instruction issuance event.

[0161] Clause A34. The method according to Clause A32, wherein the plurality of performance counters includes a first-level performance counter, the first-level performance counter being a lower-level expanded performance counter of the 0th-level performance counter, the first-level performance counter being associated with a combination of computation events of the computation module and memory access events of the storage module.

[0162] Clause A35. The method according to Clause A34, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of at least one of the following events: a convolution operation event and at least one of a scalar operation event and a vector operation event in the computation module; a memory access event of on-chip or off-chip storage in the storage module; a data input / output event or a data transfer event in the storage module.

[0163] Clause A36. The method according to Clause A33, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of instruction fetch blocking events and instruction downstream blocking events in the computation event.

[0164] Clause A37. The method according to Clause A33, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of instruction fetch blocking events and instruction downstream blocking events in the memory access event.

[0165] Clause A38. The method according to Clause A33, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of a computation time ratio of the computation module to the total time or a memory access time ratio of the storage module to the total time.

[0166] Clause A39. A computer-readable storage medium having stored thereon computer program code of a method for analyzing the performance of an artificial intelligence accelerator, wherein when the computer program code is run by a processing device, it performs the method described in any one of Clauses A27 to A38.

[0167] The embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A processor core for analyzing pipeline performance, comprising a control module, a computation module, and a storage module, wherein the control module, the computation module, and the storage module correspond to multiple events in the pipeline, the multiple events are logically combined into multiple combinations, the multiple combinations are mutually exclusive and are divided into front-end bottlenecks and back-end bottlenecks, characterized in that... The processor core also includes multiple performance counters, which are respectively associated with the multiple combinations. When the multiple combinations meet the corresponding counting conditions, the corresponding performance counter counts. The performance counters are associated with multiple combinations of multi-layer events obtained from the pipeline layering, and the performance counters of a single layer are summarized into the performance counters of the previous layer and expanded into the performance counters of the next layer according to the pipeline. The multiple events mentioned include control pipeline events subdivided by control pipeline time, operation pipeline events subdivided by operation type, and memory access pipeline events subdivided by memory access type or memory access location.

2. The processor core according to claim 1, wherein the plurality of events includes running events and idle events caused by specific factors.

3. The processor core according to claim 1, wherein the front-end bottleneck includes the inability to issue instructions, and the back-end bottleneck includes the inability to perform computation or access memory.

4. The processor core according to claim 1, wherein the plurality of performance counters includes a level 0 performance counter, the level 0 performance counter being associated with a combination of instruction issuance events of the control module, and operation memory access events of the arithmetic module and the storage module.

5. The processor core according to claim 4, wherein the plurality of performance counters includes a first-level performance counter, the first-level performance counter being a lower-level expanded performance counter of the 0th-level performance counter, and the first-level performance counter being associated with a combination of the instruction fetch blocking event and the instruction downstream blocking event in the instruction dispatch event.

6. The processor core according to claim 4, wherein the plurality of performance counters includes a first-level performance counter, the first-level performance counter being a lower-level expanded performance counter of the 0th-level performance counter, and the first-level performance counter being associated with a combination of computation events of the computation module and memory access events of the storage module.

7. The processor core of claim 6, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of at least one of the following events: The operation module includes at least one of the following: convolution operation event, scalar operation event, and vector operation event; Memory access events of on-chip or off-chip storage in the storage module; The storage module contains data input / output events or data transfer events.

8. The processor core according to claim 6, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, and the second-level performance counter being associated with a combination of instruction fetch blocking events and instruction downstream blocking events in the computation events.

9. The processor core according to claim 6, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, and the second-level performance counter being associated with a combination of instruction fetch blocking events and instruction downstream blocking events in the memory access events.

10. The processor core according to claim 5, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, and the second-level performance counter is associated with a combination of events representing the proportion of computation time of the computing module to the total computation time, or a combination of events representing the proportion of memory access time of the storage module to the total memory access time.

11. The processor core according to claim 4, wherein the plurality of performance counters includes a first performance counter for counting the front-end bottleneck of layer 0, wherein the first performance counter counts when the instruction issuance event is executed and the condition of the operation memory access event being idle is met.

12. The processor core according to claim 4, wherein the plurality of performance counters includes a second performance counter for counting the front-end bottleneck of layer 0, wherein the second performance counter counts when the condition of the instruction issuance event being idle and the operation memory access event being idle is met.

13. The processor core according to claim 6, wherein the plurality of performance counters includes a third performance counter for counting back-end bottlenecks of layer 0, wherein the third performance counter counts when the instruction issuance event is idle and the conditions for the operation memory access event to run are met.

14. The processor core of claim 13, wherein the plurality of performance counters includes a fourth performance counter for counting back-end bottlenecks of layer 0, wherein the fourth performance counter counts when the instruction issuance event is executed and the conditions for the operation memory access event are met.

15. The processor core of claim 11, wherein the control module includes a cache for temporary microinstructions, and the plurality of performance counters includes a fifth performance counter for counting the front-end bottleneck of the first layer, wherein, provided that the first performance counter is counting, the fifth performance counter counts when the following condition is met: The microinstruction in the cache is being sent; The instruction queue of the computing module or the storage module is full.

16. The processor core of claim 11, wherein the control module includes a cache for temporary microinstructions, and the plurality of performance counters includes a sixth performance counter for counting the front-end bottleneck of the first layer, wherein, provided that the first performance counter is counting, the sixth performance counter counts when the following condition is met: The microinstruction in the cache is being sent; The computing module or the storage module is in a synchronized state.

17. The processor core of claim 12, wherein the plurality of performance counters includes a seventh performance counter for counting the front-end bottleneck of the first layer, wherein, provided that the second performance counter is counting, the seventh performance counter counts when the following condition is met: A cache loss or branch prediction error causes the pipeline to be interrupted for a time exceeding a threshold.

18. The processor core of claim 12, wherein the plurality of performance counters includes an eighth performance counter for counting the front-end bottleneck of the first layer, and the eighth performance counter counts while the second performance counter is counting, provided that the following condition is met: Instruction wait register dedependency caused the pipeline interruption time to exceed the threshold.

19. The processor core of claim 14, wherein the plurality of performance counters includes a ninth performance counter for counting back-end bottlenecks of the first layer, and wherein the ninth performance counter counts provided that the third or fourth performance counter counts, and the following conditions are met: The computational event is executed; The memory access event is executed.

20. The processor core of claim 14, wherein the plurality of performance counters includes a tenth performance counter for counting back-end bottlenecks of the first layer, wherein the tenth performance counter counts provided that the third performance counter counts, and the tenth performance counter counts when all of the following conditions are met: The computational event is executed; The memory access event is idle.

21. The processor core of claim 14, wherein the plurality of performance counters includes an eleventh performance counter for counting back-end bottlenecks of the first layer, and wherein, provided that the third performance counter is counting, the eleventh performance counter counts when all of the following conditions are met: The computational events are idle; The memory access event is executed.

22. An artificial intelligence accelerator, comprising a processor core according to any one of claims 1 to 21.

23. An integrated circuit device comprising the artificial intelligence accelerator according to claim 22.

24. A board comprising the integrated circuit device according to claim 23.

25. A method for performance analysis of an artificial intelligence accelerator, the artificial intelligence accelerator comprising a processor core, the processor core comprising a control module, a computing module, and a storage module, characterized in that, The method includes: The pipeline of the processor core is divided into multiple events; The multiple events form multiple combinations, which are mutually exclusive and are divided into front-end bottlenecks and back-end bottlenecks. Configure multiple performance counters to be associated with the multiple combinations; Determine whether the multiple combinations satisfy the corresponding counting conditions; If so, the corresponding performance counter will be activated. The front-end bottleneck and the back-end bottleneck are evaluated based on the count values ​​of the performance counters; The single-layer performance counter is summarized into the performance counter of the upper layer according to the pipeline and expanded into the performance counter of the lower layer; The multiple events mentioned include control pipeline events subdivided by control pipeline time, operation pipeline events subdivided by operation type, and memory access pipeline events subdivided by memory access type or memory access location.

26. The method of claim 25, wherein the plurality of events includes operational events and idle events caused by specific factors.

27. The method according to claim 25, wherein the front-end bottleneck includes the inability to issue instructions, and the back-end bottleneck includes the inability to perform computation or access memory.

28. The method of claim 25, wherein the plurality of performance counters includes a level 0 performance counter, the level 0 performance counter being associated with a combination of instruction issuance events of the control module, and computation memory access events of the computation module and the storage module.

29. The method of claim 28, wherein the plurality of performance counters includes a first-level performance counter, the first-level performance counter being a lower-level expanded performance counter of the 0th-level performance counter, and the first-level performance counter being associated with a combination of an instruction fetch blocking event and an instruction downstream blocking event in the instruction issuance event.

30. The method of claim 28, wherein the plurality of performance counters includes a first-level performance counter, the first-level performance counter being a lower-level expanded performance counter of the 0th-level performance counter, and the first-level performance counter being associated with a combination of computation events of the computation module and memory access events of the storage module.

31. The method of claim 30, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, the second-level performance counter being associated with a combination of at least one of the following events: The operation module includes at least one of the following: convolution operation event, scalar operation event, and vector operation event; Memory access events of on-chip or off-chip storage in the storage module; The storage module contains data input / output events or data transfer events.

32. The method of claim 30, wherein the plurality of performance counters includes a second-layer performance counter, the second-layer performance counter being a lower-level expanded performance counter of the first-layer performance counter, and the second-layer performance counter being associated with a combination of instruction fetch blocking events and instruction downstream blocking events in the computation event.

33. The method of claim 30, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, and the second-level performance counter being associated with a combination of instruction fetch blocking events and instruction downstream blocking events in the memory access events.

34. The method according to claim 29, wherein the plurality of performance counters includes a second-level performance counter, the second-level performance counter being a lower-level expanded performance counter of the first-level performance counter, and the second-level performance counter is associated with a combination of events representing the proportion of computation time of the computing module to the total computation time, or a combination of events representing the proportion of memory access time of the storage module to the total memory access time.

35. A computer-readable storage medium having stored thereon computer program code for a method of analyzing the performance of an artificial intelligence accelerator, wherein when the computer program code is run by a processing device, it performs the method according to any one of claims 25 to 34.

Images