A cache performance detection method, device and storage medium

Abstract

Embodiments of the present application provide a cache performance detection method and device and a storage medium, which can be applied to the field of artificial intelligence chip technology. In the method, the active rate of the load-store cache is obtained based on the number of active periods of the load-store cache and the number of active periods of an application program running on the artificial intelligence chip; the blocking condition of the interface component in the load-store cache is obtained based on the blocking rate of the interface component in the request channel dimension and the data channel dimension; the blocking condition of the communication component in the load-store cache is obtained based on the blocking rate of the communication component in the data receiving channel dimension and the return channel dimension; and the performance detection result of the load-store cache is obtained based on the active rate, the blocking condition of the interface component and the blocking condition of the communication component, so as to realize the detection of the blocking condition of each component from the whole load-store cache. Such a layered detection method improves the coverage of the load-store cache and thus improves the accuracy of performance detection.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence chip technology, and in particular to a cache performance testing method, device and storage medium. Background Technology

[0002] The Load-Store Cache (LSC) in a Graphics Processing Unit (GPU) is a hardware module specifically responsible for handling load and store operations. It connects the GPU computing core with global memory, preventing the GPU computing core from directly accessing the high-latency global memory. Therefore, the status of the Load-Store Cache directly reflects whether the data supply in the GPU is smooth. Analyzing the LSC can pinpoint memory bottlenecks and optimize data access patterns, which is key to fully leveraging the peak performance of the GPU.

[0003] In related technologies, performance counters provided by hardware modules in the GPU are used to simply count the overall cache hit rate, number of read and write operations, and other indicators. These indicators are then used to perform performance testing on the loading and storage cache to obtain the corresponding test results.

[0004] However, single metrics such as cache hit rate and number of read / write operations have certain limitations and lack the ability to deeply analyze the performance bottlenecks of complex loading and storage caches, resulting in low accuracy in performance testing of loading and storage caches. Summary of the Invention

[0005] This invention provides a cache performance testing method, device, and storage medium to improve the accuracy of performance analysis of loaded storage cache.

[0006] On the one hand, embodiments of this application provide a cache performance testing method, the method comprising:

[0007] The activity rate of the loading and storage cache is obtained based on the number of active cycles of the loading and storage cache and the number of active cycles of the application running on the artificial intelligence chip.

[0008] Based on the interface components in the loading and storage cache, the blocking status of the interface components is obtained in terms of blocking rates in both the request channel and data channel dimensions.

[0009] Based on the communication components in the loading and storage cache, the blocking status of the communication components is obtained in terms of blocking rates in the data receiving channel dimension and the return channel dimension, respectively.

[0010] Based on the activity rate, the blocking status of the interface component, and the blocking status of the communication component, the performance test results of loading the storage cache are obtained.

[0011] On one hand, embodiments of this application provide a cache performance testing device, the device comprising:

[0012] An active module is used to obtain the activity rate of the loading storage cache based on the number of active cycles of the loading storage cache and the number of active cycles of the application running on the artificial intelligence chip;

[0013] The interface module is used to obtain the blocking status of the interface component based on the blocking rate of the interface component in the loading storage cache in the request channel dimension and the data channel dimension, respectively.

[0014] The communication module is used to obtain the blocking status of the communication component based on the blocking rate of the communication component in the data receiving channel dimension and the return channel dimension, respectively, based on the loading and storage cache.

[0015] The detection module is used to obtain the performance detection results of the loading storage cache based on the activity rate, the blocking status of the interface component, and the blocking status of the communication component.

[0016] The detection module is also used for:

[0017] The full blocking rate of the first-level cache is obtained based on the number of stalled cycles when the first-level cache is full and the number of active cycles when loading the storage cache.

[0018] The full blocking rate of the shared cache is obtained based on the number of stalled cycles when the shared cache is full and the number of active cycles when loading the storage cache.

[0019] The detection module is also used for:

[0020] Get the number of standby cycles in the L1 cache caused by read-write conflicts;

[0021] The blocking rate caused by the read-write conflict is obtained based on the number of stalled cycles of the first-level cache and the number of active cycles of the load storage cache.

[0022] The detection module is also used for:

[0023] Get the number of standby cycles in the L1 cache caused by read-write conflicts;

[0024] The blocking rate caused by the read-write conflict is obtained based on the number of stalled cycles of the first-level cache and the number of active cycles of the load storage cache.

[0025] Based on the cache hit rate, cache reuse rate and cache full load rate of the first-level cache within a preset time period, the overall turbulence score of the first-level cache is obtained.

[0026] The cache efficiency of the first-level cache is obtained based on the cache hit rate, the blocking rate caused by the read-write conflict, and the turbulence comprehensive score.

[0027] The detection module is specifically used for:

[0028] Determine the number of hits and misses from the total number of operations on the first-level cache, and obtain the number of cache blocks that are allowed to be evicted from the first-level cache;

[0029] The cache hit rate is obtained based on the number of hits and the total number of operations; and the cache reuse rate is obtained based on the number of misses and the number of cache blocks.

[0030] The number of full-load cycles when the amount of data in the first-level cache is lower than the full-load baseline is obtained, and the cache full-load rate is determined based on the number of full-load cycles and the number of active cycles of loading the storage cache.

[0031] The detection module is also used for:

[0032] Based on the cache full load rate and the cache hit rate, the efficiency of the first-level cache under pressure is determined.

[0033] The detection module is also used for:

[0034] The number of the first cycles in which write conflicts occurred in the shared cache within a preset time period is obtained, as well as the number of the first bytes written to the shared cache within the preset time period;

[0035] The write conflict rate is obtained based on the first cycle number and the first byte number;

[0036] Obtain the second number of cycles in which read conflicts occur in the shared cache within the preset time period, and the second number of bytes read from the shared cache within the preset time period;

[0037] The read conflict rate is obtained based on the second cycle number and the second byte number;

[0038] Based on the write conflict rate and the read conflict rate, the total conflict rate of the shared cache is obtained;

[0039] The caching efficiency of the shared cache is obtained based on the total conflict rate of the shared cache.

[0040] The detection module is specifically used for:

[0041] The third cycle number of atomic operation conflicts that occurred in the shared cache within the preset time period is obtained, as well as the third byte number of the atomic operation executed within the preset time period; the atomic operation is the smallest uninterruptible execution unit in a concurrent scenario.

[0042] The atomic collision rate is obtained based on the third period number and the third byte number.

[0043] The total conflict rate of the shared cache is obtained by weighted summing of the write conflict rate, the read conflict rate, and the atomic conflict rate.

[0044] On one hand, embodiments of this application provide a computer device, including:

[0045] Memory, used to store program instructions;

[0046] An artificial intelligence chip is used to run program instructions stored in the memory and execute the steps of the above-mentioned cache performance detection method according to the obtained program.

[0047] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer to perform the steps of the above-described cache performance detection method.

[0048] On one hand, embodiments of this application provide a computer program product, including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described cache performance detection method.

[0049] In this embodiment, the activity rate of the load storage cache is obtained based on the number of active cycles of the load storage cache and the number of active cycles of the application running on the artificial intelligence chip, enabling overall detection of the load storage cache. Based on the interface components in the load storage cache, the blocking rate is obtained in both the request channel and data channel dimensions to determine the blocking status of the interface components. Similarly, based on the communication components in the load storage cache, the blocking rate is obtained in both the data receiving channel and return channel dimensions to determine the blocking status of the communication components. This in-depth detection of the blocking status of multiple components under the load storage cache across different dimensions improves the coverage of performance detection for multiple components. Therefore, based on the activity rate, the blocking status of interface components, and the blocking status of communication components, the performance detection results of the load storage cache are obtained, improving the accuracy of load storage cache performance detection. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of a chip architecture provided in an embodiment of this application;

[0052] Figure 2 A flowchart illustrating a cache performance testing method provided in this application embodiment. Figure 1 ;

[0053] Figure 3 A flowchart illustrating a cache performance testing method provided in this application embodiment. Figure 2 ;

[0054] Figure 4 A flowchart illustrating a cache performance testing method provided in this application embodiment. Figure 3 ;

[0055] Figure 5 A flowchart illustrating a cache performance testing method provided in this application embodiment. Figure 4 ;

[0056] Figure 6 A flowchart illustrating a cache performance testing method provided in this application embodiment. Figure 5 ;

[0057] Figure 7 A flowchart illustrating a cache performance testing method provided in this application embodiment. Figure 6 ;

[0058] Figure 8 This is a schematic diagram of the structure of a cache performance testing device provided in an embodiment of this application;

[0059] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0061] refer to Figure 1 This is a structural diagram of an artificial intelligence chip applicable to the embodiments of this application. The artificial intelligence chip 100 includes at least: a video memory 101 and a plurality of computing units 102, wherein the computing units 102 may be a streaming processing cluster (SPC).

[0062] The video memory 101 can be high-bandwidth memory (HBM) or other types of memory.

[0063] The computing unit 102 includes a load storage cache 103 and multiple execution units 104. The execution units 104 are used to handle parallel computing tasks, such as retrieving stored data from the load storage cache 103 for reduction and other computational tasks. The load storage cache 103 mainly includes a level 1 cache 105 and a shared cache 106.

[0064] In addition to the structure described above, the artificial intelligence chip 100 in this application may also include other structures, and this application does not specifically limit such structures.

[0065] Artificial intelligence chips 100 can be: Graphics Processing Unit (GPU), General-purpose computing on graphics processing units (GPGPU), Domain Specific Architecture (DSA), etc.

[0066] In view of this, this application is based on Figure 1 The diagram of the AI ​​chip architecture shown illustrates a method for detecting cache performance, applicable to various scenarios such as image processing, speech processing, and text processing. The physical meaning of loading cached data can differ across these scenarios.

[0067] For example, in a text processing scenario, cached data can be text data used in tasks such as text generation and text recognition.

[0068] For example, in a speech processing scenario, cached data can be speech data used in tasks such as speech enhancement, speech recognition, and speech synthesis.

[0069] For example, in image processing scenarios, cached data can be image data used in tasks such as image preprocessing, image segmentation, and object detection.

[0070] It should be noted that in this application, "first" and "second" are used for descriptive purposes only and should not be construed as indicating relative importance or implying the number of technical features indicated.

[0071] The following section details the process of cache performance testing methods. (See also...) Figure 2 This method is executed by a computer device, which includes a central processing unit and... Figure 1 The artificial intelligence chip shown includes the following steps:

[0072] Step 201: Obtain the activity rate of loading the storage cache based on the number of active cycles of loading the storage cache and the number of active cycles of the application running on the artificial intelligence chip.

[0073] Specifically, the Central Processing Unit (CPU) obtains the number of active cycles of the load cache and the number of active cycles of the application running on the AI ​​chip. The number of active cycles of the load cache refers to the number of clock cycles in which the load cache actually performs data read and write operations; the number of active cycles of the application refers to the effective clock cycles in which the application executes instructions on the AI ​​chip; where clock cycle is the smallest unit of time for the GPU. The ratio of the number of active cycles of the load cache to the number of active cycles of the application is used as the load cache activity rate to roughly assess the overall workload of the load cache. The specific calculation formula for the load cache activity rate is shown in the following formula (1):

[0074] (1)

[0075] in, This indicates the activity rate of loading the storage cache; This indicates the number of active cycles for loading the storage cache; This indicates the number of active cycles of the application.

[0076] Step 202: Based on the interface components loaded from the storage cache, obtain the blocking status of the interface components in terms of blocking rates in both the request channel and data channel dimensions.

[0077] Specifically, the interface component is the Cache Block Interface (CBI) in the loading storage cache. It is a dedicated interface in the loading storage cache responsible for transferring data blocks between different cache levels, such as the data block transfer between the first-level cache and the second-level cache.

[0078] refer to Figure 3 Since the same request corresponds to multiple data, the blocking status of the interface component is detected from two dimensions: the request channel dimension and the data channel dimension. Specifically, the blocking rate of the data channel dimension is the proportion of the number of pauses caused by the interface component processing data transmission relative to the total amount of each type of data; the blocking rate of the request channel dimension is the proportion of the number of pauses caused by the interface component processing request transmission relative to the total amount of each type of request. The formula for calculating the blocking rate of the interface component in the request channel dimension is shown in the following formula (2):

[0079] (2)

[0080] in, This indicates the blocking rate of the interface component at the request channel level; This indicates the number of pause cycles caused by the interface component processing index requests; This represents the number of index requests sent to the interface component to 'migrate Global Memory (GLM) data to the shared cache (GSM)'. This indicates the number of index requests sent to the interface component to 'read first-level cache data'; This indicates the number of index requests sent to the interface component to 'write data to the first-level cache'; This indicates the number of index requests sent to the interface component where the data is located at the low byte offset of the cache line; This indicates the number of index requests sent to the interface component to write multidimensional data structures (such as two-dimensional arrays and tensors) to the cache; This represents the number of index requests sent to the interface component to 'migrate shared cache (GSM) data to global memory (GLM)'. This indicates the number of index requests for the 'comparison-type cache' sent to the interface component; This indicates the number of index requests sent to the interface component for 'multidimensional data barrier synchronization'; This indicates the number of index requests for 'system-level barrier synchronization' sent to the interface component; This indicates the number of index requests sent to the interface component for 'loading multidimensional data and performing reduction operations'; This indicates the number of index requests for 'multidimensional data I / O storage operations' sent to the interface component.

[0081] The formula for calculating the blocking rate of the interface component in the data channel dimension is shown in the following formula (3):

[0082] (3)

[0083] in, This indicates the blocking rate of the interface component in the data channel dimension; This indicates the number of pause cycles caused by the interface component processing data transmission; This represents the total number of ticks of data flow from the 'Shared Buffer (GSM) to Global Memory (GLM)' transmitted to the interface component; This represents the total number of ticks of data streams related to atomic operations transmitted to the interface component. This represents the total number of ticks of the data stream associated with L1 cache write operations transmitted to the interface component. This represents the total number of ticks of data streams related to cache write operations of multidimensional data structures (such as tensors and two-dimensional arrays) transmitted to the interface component. This represents the total number of ticks of data streams related to reduction operations (such as summation, finding the maximum value, and finding the average) transmitted to the interface component. This represents the total number of ticks of data streams related to multidimensional data reduction operations (such as tensor global pooling and column-wise reduction of two-dimensional arrays) transmitted to the interface component. This indicates the number of index requests for the 'comparison cache' sent to the interface component.

[0084] In this embodiment of the application, the load storage cache of the artificial intelligence chip contains 128 bytes per tick.

[0085] Step 203: Based on the communication components loaded from the storage cache, obtain the blocking status of the communication components in terms of blocking rates in both the data receiving channel and return channel dimensions.

[0086] Specifically, in a multi-GPU system, any two GPUs can act as communication components that can directly communicate with each other, forming a peer entity relationship. This means that they can establish a communication link and transmit data directly without going through a third party. The blocking rate of the load cache in the GPU that acts as the communication component in the data receiving channel and the return channel is then obtained.

[0087] When GPU1 and GPU2 establish communication, it indicates that they have established a peer relationship, where GPU2 is any GPU other than GPU1. Considering GPU1 as peer Peer0 and GPU2 as peer Peer1, the blocking rate of peer Peer0 in the data receive channel dimension refers to the blocking rate when peer Peer1 sends data to peer Peer0, while the blocking rate of peer Peer0 in the return channel dimension refers to the blocking rate when peer Peer0 responds to peer Peer1; similarly, the blocking rate of peer Peer1 in the data receive channel dimension refers to the blocking rate when peer Peer0 sends data to peer Peer1, and the blocking rate of peer Peer1 in the return channel dimension refers to the blocking rate when peer Peer1 responds to peer Peer0.

[0088] For peer Peer0, the formula for calculating the total number of requests it receives is as follows:

[0089] (4)

[0090] in, This indicates the total number of requests received by peer 0; This indicates the number of atomic operation requests received by peer Peer0; This represents the total number of ticks of the asynchronous storage operation-related data stream received by peer Peer0; This represents the total number of ticks in the asynchronous reduction operation-related data stream received by peer Peer0; This indicates the number of memory barrier synchronization requests received by peer Peer0; This indicates the number of general synchronization barrier requests sent by peer Peer0; This represents the total number of ticks in which data is transferred from the shared buffer of peer Peer1 to the shared buffer of peer Peer0. This represents the total number of ticks in which data is transferred from the global memory of peer Peer1 to the shared buffer of peer Peer0.

[0091] Furthermore, based on formula (4), the total number of requests received by peer Peer0 is obtained, and the blocking rate of peer Peer0 in the data receiving channel dimension is calculated. The specific calculation formula is shown in the following formula (5):

[0092] (5)

[0093] in, This represents the blocking rate of peer 0 in the data receiving channel dimension; This indicates the total number of requests received by peer Peer0; This indicates the number of cycles that peer 0 stalled due to data inconsistency.

[0094] Furthermore, the number of stall cycles caused by Peer0 responding to Peer1 due to data consistency issues such as confirming receipts or verifying data results is obtained. Based on the number of stall cycles of Peer0 and the number of active cycles of loading the storage cache, the blocking rate of Peer0 in the return channel dimension is determined. The specific calculation formula is shown in the following formula (6):

[0095] (6)

[0096] in, This indicates the blocking rate of peer 0 in the return channel dimension; This represents the number of stagnation cycles in the return channel dimension for peer 0; This indicates the number of active cycles for loading the storage cache.

[0097] Similarly, for Peer1, the formula for calculating the total number of requests it receives is shown in the following formula (7):

[0098] (7)

[0099] in, This indicates the total number of requests received by peer Peer1; This indicates the number of atomic operation requests received by peer Peer1; This represents the total number of ticks in the asynchronous storage operation-related data stream received by peer Peer1; This represents the total number of ticks in the asynchronous reduction operation-related data stream received by peer Peer1; This indicates the number of memory barrier synchronization requests received by peer Peer1; This indicates the number of general synchronization barrier requests sent by peer Peer1; This represents the total number of ticks in which data is transferred from the shared buffer of peer 0 to the shared buffer of peer 1. This represents the total number of ticks in which data is transferred from the global memory of peer Peer0 to the shared buffer of peer Peer1.

[0100] The blocking rate of peer 1 in the data receiving channel dimension is specifically calculated using the following formula (8):

[0101] (8)

[0102] in, This represents the blocking rate of peer 1 in the data receiving channel dimension; This represents the total number of requests received by peer Peer1; This represents the number of cycles that peer 1 stalled due to data inconsistency.

[0103] The blocking rate of peer 1 on the return channel dimension is specifically calculated using the following formula (9):

[0104] (9)

[0105] in, This represents the blocking rate of peer 1 in the return channel dimension; This represents the number of stall cycles for peer1 in the return channel dimension; This indicates the number of active cycles for loading the storage cache.

[0106] In addition, for the load memory caches in different computing units of the same GPU, the total number of stall cycles caused by busy transmission links or unread data during data interaction of each load memory cache, and the total number of ticks of the data stream interaction of each load memory cache are obtained. Based on the total number of stall cycles of each load memory cache and the total number of ticks of the data stream interaction in the same GPU, the local blocking rate is determined. The specific calculation formula is shown in the following formula (10):

[0107] (10)

[0108] in, This indicates the blocking rate of each local storage cache being loaded; This indicates the total number of stall cycles for all local storage caches. This represents the total number of ticks of interactive data streams loaded from storage caches in each local computing unit.

[0109] Step 204: Based on the activity rate, the blocking status of the interface component, and the blocking status of the communication component, obtain the performance test results of loading the storage cache.

[0110] In this embodiment, the activity rate of the load storage cache is obtained based on the number of active cycles of the load storage cache and the number of active cycles of the application running on the artificial intelligence chip, enabling overall detection of the load storage cache. Based on the interface components in the load storage cache, the blocking rate of the interface components is obtained in the request channel dimension and the data channel dimension, respectively. Based on the communication components in the load storage cache, the blocking rate of the communication components is obtained in the data receiving channel dimension and the return channel dimension, respectively. This allows for in-depth detection of the blocking status of multiple components under the load storage cache in different dimensions, improving the coverage of performance detection of multiple components. Thus, based on the activity rate, the blocking status of interface components, and the blocking status of communication components, the performance detection results of the load storage cache are obtained, improving the accuracy of the performance detection of the load storage cache.

[0111] In some embodiments, the full blocking rate of the first-level cache is obtained based on the number of stalled cycles when the first-level cache is full and the number of active cycles when loading the storage cache; the full blocking rate of the shared cache is obtained based on the number of stalled cycles when the shared cache is full and the number of active cycles when loading the storage cache.

[0112] Specifically, the shared cache and the first-level cache share the same physical storage space in the loading storage cache. The proportion of the shared cache and the first-level cache in the same physical storage space can be configured before runtime through Memory-Mapped I / O (MMIO). Among them, the full state is the normal working state when the storage resources of the first-level cache or the shared cache are exhausted. In essence, it means that all cache lines are occupied by valid data.

[0113] For the first-level cache in the loading storage cache, the total number of clock cycles that cause the computing unit to be unable to submit new cache write requests and thus stall due to the first-level cache write buffer being full is obtained as the number of stalled cycles in the first-level cache full state. Combined with the number of active cycles of loading storage cache, the full blocking rate of the first-level cache is calculated. The specific calculation formula is shown in the following formula (11):

[0114] (11)

[0115] in, This indicates the full blocking rate of the L1 cache; This represents the number of pause cycles when the L1 cache is full; This indicates the number of active cycles for loading the storage cache.

[0116] For the shared cache in the loading storage cache, the number of stalled cycles caused by the write buffer in the shared cache being full, which prevents the computing unit from submitting new cache write requests, is obtained as the number of stalled cycles in a full shared cache state. Combined with the number of active cycles of loading the storage cache, the full blocking rate of the shared cache is calculated. The specific calculation formula is shown in the following formula (12):

[0117] (12)

[0118] in, This indicates the full blocking rate of the shared cache; This represents the number of stall cycles when the shared cache is full; This indicates the number of active cycles for loading the storage cache.

[0119] In some embodiments, the number of stalled cycles in the L1 cache caused by read-write conflicts is obtained; based on the number of stalled cycles in the L1 cache and the number of active cycles in loading the storage cache, the blocking rate caused by read-write conflicts is obtained.

[0120] Specifically, a read-after-write conflict (RAW) refers to a pipeline stall caused by a subsequent write operation to the same memory address in the processor pipeline, which prevents a previous read operation to that address from completing normally. The total number of clock cycles that prevents the computation unit from accessing the L1 cache due to a read-after-write conflict is obtained as the number of L1 cache stall cycles caused by the read-after-write conflict. Combined with the number of active cycles for loading the memory cache, the blocking rate caused by the read-after-write conflict is calculated. The specific calculation formula is shown in the following formula (13):

[0121] (13)

[0122] in, This indicates the blocking rate caused by read-write conflicts; This indicates the number of time the L1 cache is idle due to read-write conflicts; This indicates the number of active cycles for loading the storage cache.

[0123] Furthermore, it is also possible to obtain the total number of cache access requests that are stalled due to read-write conflicts, the number of write operations that bypass the first-level cache due to read-write conflicts in the first-level cache, the number of read operations, the number of atomic operations performed on the first-level cache, the number of multiple scattered read operations that are merged into a single read operation, and the number of multiple scattered write operations that are merged into a single write operation, thereby realizing the determination of the request blocking rate caused by read-write reread at the request dimension. The specific calculation formula is shown in the following formula (14):

[0124] (14)

[0125] in, This indicates the request blocking rate caused by read-after-write rereading; This represents the total number of cache access requests that were stalled due to read-write conflicts; This indicates the number of write operations that bypassed the first-level cache due to write conflicts; This indicates the number of read operations that bypassed the L1 cache due to read-after-write conflicts; This indicates the number of atomic operations performed on the first-level cache. This indicates the number of times multiple separate read operations are combined into a single read operation; This indicates the number of times multiple separate write operations are combined into a single write operation.

[0126] In some embodiments, the number of hits and misses are determined from the total number of operations on the L1 cache, and the number of cache blocks allowed to be evicted in the L1 cache is obtained; the cache hit rate is obtained based on the number of hits and the total number of operations; and the cache reuse rate is obtained based on the number of misses and the number of cache blocks; the number of full-load cycles when the data volume of the L1 cache is lower than the full-load baseline is obtained, and the cache full-load rate is determined based on the number of full-load cycles and the number of active cycles of loading the storage cache.

[0127] Specifically, cache blocks that can be evicted in the first-level cache refer to cache blocks in the first-level cache that can be selected and evicted by the replacement policy. These cache blocks are not locked by hardware or software, and space can be freed up by evicting them when the first-level cache is full or when new data needs to be loaded.

[0128] The total number of operations includes the number of read operations and the number of write operations. The cache hit rate of the first-level cache is calculated based on the number of read and write hits. The specific calculation formula is shown in the following formula (15):

[0129] (15)

[0130] in, Indicates cache hit rate; Indicates the total number of operations; Indicates the number of write operation hits; This indicates the number of hits in a read operation.

[0131] The number of misses is determined based on the number of hits in read / write operations and the total number of operations; the number of cache blocks allowed to be evicted is obtained to calculate the cache reuse rate, and the specific calculation formula is shown in the following formula (16):

[0132] (16)

[0133] in, Indicates cache reuse rate; Indicates the number of misses; Indicates the number of cache blocks determined by the normal eviction policy; Indicates the number of cache blocks marked as being evicted first; This indicates the number of cache blocks marked as the last evicted; This indicates the number of cache blocks that cannot be evicted through normal means but are evicted due to a full mask trigger.

[0134] Furthermore, the cache eviction rate of the first-level cache can be calculated based on the number of cache blocks allowed to be evicted. The specific calculation formula is shown in the following formula (17):

[0135] (17)

[0136] in, This represents the cache eviction rate of the first-level cache. Indicates the number of cache blocks that are allowed to be evicted from the L1 cache; This indicates the number of active cycles for loading the storage cache.

[0137] In addition, the formula for calculating the cache load rate is shown in the following formula (18):

[0138] (18)

[0139] in, This indicates the cache load rate of the first-level cache. This indicates that the amount of data in the L1 cache is lower than the number of full-load cycles compared to the full-load baseline. This indicates the number of active cycles for loading the storage cache.

[0140] Furthermore, the access and eviction status of the first-level cache can be combined to calculate the access to eviction ratio of the first-level cache. The specific calculation formula is shown in the following formula (19):

[0141] (19)

[0142] in, This indicates the ratio of accesses to evictions from the first-level cache. Indicates the number of cache blocks that are allowed to be evicted from the L1 cache; Indicates the number of write operation hits; This indicates the number of hits in a read operation.

[0143] In some embodiments, the number of paused cycles of the L1 cache caused by read-write conflicts is obtained; based on the number of paused cycles of the L1 cache and the number of active cycles of loading the storage cache, the blocking rate caused by read-write conflicts is obtained; based on the cache hit rate, cache reuse rate and cache full load rate of the L1 cache within a preset time period, the overall thrashing score of the L1 cache is obtained; based on the cache hit rate, the blocking rate caused by read-write conflicts and the overall thrashing score, the cache efficiency of the L1 cache is obtained.

[0144] Specifically, see Figure 4 The overall turbulence score of the L1 cache is obtained by multiplying the cache full load rate, the difference between unit 1 and cache hit rate (i.e., cache miss rate), and cache reuse rate.

[0145] The specific formula for calculating the overall thrashing score of the L1 cache is shown in the following formula (20):

[0146] (20)

[0147] in, This represents the overall thrashing score of the L1 cache. Indicates the cache full load rate; Indicates cache hit rate; This indicates the cache reuse rate.

[0148] See Figure 5 The cache efficiency of the first-level cache is obtained by multiplying the cache hit rate, the difference between unit 1 and the overall score of thrashing, and the blocking rate caused by unit 1 and read-write conflicts. The specific calculation formula for the cache efficiency of the first-level cache is shown in the following formula (21):

[0149] (twenty one)

[0150] in, This demonstrates the caching efficiency of the first-level cache. Indicates cache hit rate; This indicates the blocking rate caused by read-write conflicts; This indicates the overall score for bumpiness.

[0151] In some embodiments, the efficiency of the L1 cache under pressure is determined based on the cache load rate and cache hit rate.

[0152] Specifically, see Figure 4The efficiency of the L1 cache under pressure is obtained by multiplying the difference between unit 1 and cache full load rate with the cache hit rate. The efficiency of the L1 cache under pressure characterizes the efficiency of the L1 cache under a certain cache pressure. The cache pressure can be the efficiency when the amount of data in the L1 cache is higher than the full load baseline. The specific calculation formula for the efficiency under pressure is shown in the following formula (22):

[0153] (twenty two)

[0154] in, Indicates the efficiency of the L1 cache under pressure; Indicates cache hit rate; This indicates the cache full load rate.

[0155] In some embodiments, the number of first cycles in which write conflicts occur in the shared cache within a preset time period, and the number of first bytes written to the shared cache within the preset time period are obtained; the write conflict rate is obtained based on the first cycle number and the first byte number; the number of second cycles in which read conflicts occur in the shared cache within the preset time period, and the number of second bytes read from the shared cache within the preset time period are obtained; the read conflict rate is obtained based on the second cycle number and the second byte number; the total conflict rate of the shared cache is obtained based on the write conflict rate and the read conflict rate; and the cache efficiency of the shared cache is obtained based on the total conflict rate of the shared cache.

[0156] Specifically, the preset time period can be the time period for detecting the loading storage cache, which is set based on the actual situation, and this application does not impose specific limitations on it. The loading storage cache in this embodiment has two storage banks (bank0 and bank1), each containing multiple cache lines. When determining the conflict rate of the shared cache, information on various operations performed on storage banks bank0 and bank1 is obtained respectively.

[0157] The formula for calculating the write conflict rate of a shared cache is shown in the following formula (23):

[0158] (twenty three)

[0159] in, Indicates the write conflict rate of the shared cache; This represents the number of pause cycles that cause the memory bank0 to stall due to multiple threads simultaneously performing write operations on the same cache line in bank0. This represents the number of pause cycles that cause the memory bank1 to stall due to multiple threads simultaneously performing write operations on the same cache line in bank1. This indicates the total number of bytes written to the shared cache.

[0160] The formula for calculating the read conflict rate of the shared cache is shown in the following formula (24):

[0161] (twenty four)

[0162] in, This indicates the read conflict rate of the shared cache; This represents the number of pause cycles that cause the memory bank0 to stall due to multiple threads simultaneously performing read operations on the same cache line in bank0. This represents the number of pause cycles that cause the storage bank1 to stall due to multiple threads simultaneously performing read operations on the same cache line in bank1. This indicates the total number of bytes read from the shared cache.

[0163] In some embodiments, the number of third cycles in which atomic operation conflicts occur in the shared cache within a preset time period, and the number of third bytes in which atomic operations are executed within the preset time period are obtained; an atomic operation is the smallest uninterruptible execution unit in a concurrent scenario; the atomic conflict rate is obtained based on the number of third cycles and the number of third bytes; the total conflict rate of the shared cache is obtained by weighted summing of the write conflict rate, read conflict rate and atomic conflict rate.

[0164] Specifically, an atomic operation refers to an instruction operation in a multi-threaded or multi-process environment that cannot be interrupted during execution. It either executes completely or not at all, without switching to other threads in between. In other words, it is the smallest uninterruptible unit of execution in a concurrent scenario. Common atomic operations include read-modify-write operations, swap operations, and bitwise operations.

[0165] See Figure 6 The total conflict rate of the shared cache is obtained by weighting the read conflict rate, write conflict rate, and atomic conflict rate according to their respective byte counts. The formula for calculating the atomic conflict rate of the shared cache is shown in the following formula (25):

[0166] (25)

[0167] in, Indicates the atomic conflict rate of the shared cache; This represents the number of pause cycles that cause the memory bank0 to stall when multiple threads simultaneously perform atomic operations on the same cache line within bank0. This represents the number of pause cycles that cause the memory bank1 to stall when multiple threads simultaneously perform atomic operations on the same cache line in bank1. This indicates the total number of bytes read from the shared cache.

[0168] The total conflict rate of the shared cache can be obtained using the following formula (26):

[0169] (26)

[0170] in, This indicates the total conflict rate of the shared cache; Indicates the write conflict rate of the shared cache; This indicates the total number of bytes written to the shared cache; This indicates the read conflict rate of the shared cache; This indicates the total number of bytes read from the shared cache. Indicates the atomic conflict rate of the shared cache; This indicates the total number of bytes read from the shared cache.

[0171] Further, see Figure 7 The cache efficiency of the shared cache is obtained by the difference between unit 1 and the total conflict rate of the shared cache. The formula for calculating the cache efficiency of the shared cache is shown in the following formula (27):

[0172] (27)

[0173] in, Indicates the caching efficiency of the shared cache; This indicates the total conflict rate of the shared cache.

[0174] In some embodiments, the activity rate of each submodule is obtained to analyze the activity of loading the storage cache in a fine-grained manner.

[0175] Specifically, each submodule includes: a thread bundle shuffling unit, a level 1 cache, a hierarchical cache interconnection unit, a load memory unit, and a memory bank. Among them, the thread bundle shuffling unit is a hardware mechanism in the GPU that enables direct data exchange between threads within the same thread bundle, without the need for shared memory or global memory intermediaries, which can greatly improve the efficiency of inter-thread communication.

[0176] The formula for calculating the activity rate of the thread bundle shuffle unit is shown in the following formula (28):

[0177] (28)

[0178] in, This indicates the activity rate of the thread bundle shuffle unit; This indicates the number of active cycles of the thread bundle shuffle unit, which is the total number of clock cycles in which the thread bundle shuffle unit is in the "performing data exchange operation" state. This indicates the number of active cycles for loading the storage cache.

[0179] The formula for calculating the activity rate of the first-level cache is shown in the following formula (29):

[0180] (29)

[0181] in, This indicates the activity rate of the first-level cache; This indicates the number of active cycles of the L1 cache; This indicates the number of active cycles for loading the storage cache.

[0182] The formula for calculating the activity rate of the shared cache is shown in the following formula (30):

[0183] (30)

[0184] in, Indicates the activity rate of the shared cache; Indicates the number of active cycles of the shared cache; This indicates the number of active cycles for loading the storage cache.

[0185] The formula for calculating the activity rate of the hierarchical cache interconnect unit is shown in the following formula (31):

[0186] (31)

[0187] in, This indicates the activity rate of the hierarchical cache interconnect unit; This indicates the number of active cycles of the hierarchical cache interconnect unit; This indicates the number of active cycles for loading the storage cache.

[0188] The formula for calculating the activity rate of loaded storage units is shown in the following formula (32).

[0189] (32)

[0190] in, This indicates the activity rate of the loaded storage unit; This indicates the number of active cycles for loading memory units; This indicates the number of active cycles for loading the storage cache.

[0191] The number of active cycles for storage bank0 and storage bank1 are also obtained, and their activity rates are calculated respectively. The formula for calculating the activity rate of storage bank0 is shown in the following formula (33):

[0192] (33)

[0193] in, This indicates the activity rate of storage bank0; This indicates the number of active cycles for memory bank0; This indicates the number of active cycles for loading the storage cache.

[0194] The formula for calculating the activity rate of storage bank1 is shown in the following formula (34):

[0195] (34)

[0196] in, This indicates the activity rate of storage bank1; This indicates the number of active cycles for memory bank1; This indicates the number of active cycles for loading the storage cache.

[0197] In addition, request splitting events occurring in the first-level cache and shared cache are obtained. Request splitting events refer to the mechanism of dividing access requests into smaller units for processing. The main purpose is to resolve conflicts and improve resource utilization. This application calculates the request splitting rate of the first-level cache and shared cache respectively, and uses the request splitting rate as a separate dimension to perform performance testing on the first-level cache and shared cache.

[0198] The formula for calculating the request splitting rate of the shared cache is shown in the following formula (35):

[0199] (35)

[0200] in, Indicates the request splitting rate of the shared cache; This indicates the number of times a write operation performed on the shared cache is split. This indicates the number of times a read operation performed on the shared cache is split. This indicates the number of times the atomic operation performed on the shared cache is split. Indicates the number of write operations performed on the shared cache; This indicates the number of read operations performed on the shared cache. This indicates the number of atomic operations performed on the shared cache.

[0201] The formula for calculating the request splitting rate of the first-level cache is shown in the following formula (36):

[0202] (36)

[0203] in, This indicates the request splitting rate of the first-level cache; This indicates the number of times a write operation performed on the first-level cache was split. This indicates the number of times a read operation performed on the L1 cache is split. This indicates the number of times an atomic operation performed on the L1 cache is split. This represents the total number of cache access requests that were stalled due to read-write conflicts; This indicates the number of write operations that bypassed the first-level cache due to write conflicts; This indicates the number of read operations that bypassed the L1 cache due to read-after-write conflicts; This indicates the number of atomic operations performed on the first-level cache. This indicates the number of times multiple separate read operations are combined into a single read operation; This indicates the number of times multiple separate write operations are combined into a single write operation.

[0204] In this embodiment, the activity rate of the load storage cache, the activity rate of the thread bundle shuffling unit, the first-level cache, the hierarchical cache interconnection unit, the load storage unit, and the storage body can be detected. Through this hierarchical detection method, the anomalies of any sub-module can be accurately captured, thereby improving the accuracy of performance detection.

[0205] In some embodiments, the latency period of various operations for loading the storage cache is obtained, and the latency of loading the storage cache is detected.

[0206] Specifically, the various operations include: read operations, atomic operations; and also more granular operations such as multidimensional data access, reduction operations, cache block indexing, and various special instructions.

[0207] The formula for calculating the global memory loading latency is shown in the following formula (37):

[0208] (37)

[0209] in, This represents the global memory load latency, which is the average number of clock cycles consumed from reading data from global memory to loading it into the storage cache. This represents the total number of clock cycles consumed during the global memory loading operation, from the time the request is initiated until the data is successfully loaded. This represents the total number of samples for global memory load operations.

[0210] The formula for calculating the delay of atomic operations is shown in the following formula (38):

[0211] (38)

[0212] in, This represents the latency of atomic operations in global memory, specifically the average number of clock cycles required to execute a global memory atomic operation. This represents the total number of clock cycles consumed during the process of a global memory atomic operation, from the initiation of the request to its completion. This represents the total number of samples of global memory atomic operations.

[0213] The formula for calculating the latency of matrix loading dedicated instructions is shown in the following formula (39):

[0214] (39)

[0215] in, This indicates the latency of the matrix loading instruction, which is the average number of clock cycles consumed per execution of the matrix loading instruction. This represents the total number of clock cycles consumed from the initiation of the matrix loading instruction request to its completion. This represents the total sample of matrix loading-specific instructions.

[0216] The formula for calculating the latency of multidimensional data access is shown in the following formula (40):

[0217] (40)

[0218] in, This represents the latency of multidimensional data access, which is the average number of clock cycles consumed per multidimensional data access operation. This represents the total number of clock cycles consumed during the process of multidimensional data access, from the initiation of a request to its completion. This represents the total sample size for multidimensional data access.

[0219] The formula for calculating the latency of cache consistency maintenance is shown in the following formula (41):

[0220] (41)

[0221] in, This represents the latency of cache consistency maintenance, which is the average number of clock cycles consumed to ensure that multiple cache replicas of the same data remain consistent each time. This represents the total number of clock cycles consumed during the process of cache consistency maintenance from initiation to completion. This represents the total number of samples maintained for cache consistency.

[0222] The formula for calculating the delay of matrix reduction operation is shown in the following formula (42):

[0223] (42)

[0224] in, This represents the latency of a matrix reduction operation, i.e., the average number of clock cycles consumed per matrix reduction operation (such as matrix summation). This represents the total number of clock cycles consumed during the matrix reduction operation from initiation to completion. This represents the total number of samples in the matrix reduction operation.

[0225] The formula for calculating the delay of synchronous operations between different computing units is shown in the following formula (43):

[0226] (43)

[0227] in, This represents the latency of synchronous operations between different computing units, i.e., the average number of clock cycles consumed each time an operation between different computing units is executed. This represents the total number of clock cycles consumed during the process of an operation between different computing units from initiation to completion. This represents the total number of samples for synchronous operations between different computing units.

[0228] The formula for calculating the latency of cache block indexing operations is shown in the following formula (44):

[0229] (44)

[0230] in, This indicates the latency of cache block indexing operations, specifically the average number of clock cycles consumed per cache block indexing operation. This represents the total number of clock cycles consumed from the initiation to the completion of a cache block indexing operation. This represents the total number of samples for cache block indexing operations.

[0231] Furthermore, memory barriers are special synchronization instructions in GPUs that enforce the "sequentiality" and "visibility" of memory operations—preventing the compiler or hardware from rearranging memory access instructions before and after the barrier, ensuring that all memory operations initiated before the barrier are "completed and perceived by other cores" before executing memory operations after the barrier.

[0232] The calculation formula for the waiting delay of the vector core memory barrier is shown in the following formula (45), where the vector core (Vcore) refers to the computing unit in the loading storage cache that is oriented towards general parallel computing.

[0233] (45)

[0234] in, This represents the memory barrier wait latency of the vector core, i.e., the number of clock cycles required to wait for the vector core each time the memory barrier is executed. This represents the total number of clock cycles consumed by the vector core due to memory barrier waits. The total number of samples that the memory barrier waits for in the vector core.

[0235] The calculation formula for the waiting delay of the tensor core memory barrier is shown in the following formula (46), where the tensor core (Tcore) refers to the computational unit in the load storage cache that is oriented towards dedicated matrix operations.

[0236] (46)

[0237] in, This represents the memory barrier wait latency of the tensor core, which is the number of clock cycles required to wait for the tensor core to execute the memory barrier each time. This represents the total number of clock cycles consumed by the tensor core due to memory barrier waits; This represents the total number of samples waiting for memory barriers in the tensor core.

[0238] The proportion of memory barrier wait execution time exceeding the hardware preset threshold can also be calculated as the memory barrier wait timeout rate. The specific calculation formula is shown in the following formula (47):

[0239] (47)

[0240] in, Indicates the timeout rate for memory barrier waits; Indicates the number of timeouts for waiting through the memory barrier; This represents the total number of samples waiting for memory barriers.

[0241] In some embodiments, the throughput of the shared cache and the first-level cache is detected from the data dimension and the request dimension, respectively.

[0242] Specifically, the formula for calculating the shared cache throughput is shown in the following formula (48):

[0243] (48)

[0244] in, This represents the shared cache throughput, which is the number of requests made to the shared cache during each active period of loading the storage cache. Indicates the number of write operations performed on the shared cache; This indicates the number of read operations performed on the shared cache. Indicates the number of atomic operations performed on the shared cache; This indicates the number of active cycles for loading the storage cache.

[0245] The formula for calculating the throughput of the first-level cache is shown in the following formula (49):

[0246] (49)

[0247] in, This represents the throughput of the first-level cache, which is the number of requests made to the first-level cache during each active period of loading the cache. This indicates the number of write operations that bypassed the first-level cache due to write conflicts; This indicates the number of read operations that bypassed the L1 cache due to read-after-write conflicts; This indicates the number of atomic operations performed on the first-level cache. This indicates the number of times multiple separate read operations are combined into a single read operation; This indicates the number of times multiple separate write operations are combined into a single write operation; This indicates the number of active cycles for loading the storage cache.

[0248] Furthermore, from the overall perspective of loading the storage cache, the amount of data and bandwidth interacting between the loading storage cache and the data engine and computing units can be measured, thereby detecting the throughput from the overall perspective of loading the storage cache.

[0249] Specifically, the total amount of data from the computing unit to the loading storage cache is shown in the following formula (50):

[0250] (50)

[0251] in, This represents the total amount of data from the computing unit to the loaded storage cache; This represents the total number of ticks of matrix data transferred from the computation unit to the matrix computation unit in the loading and storage cache; This represents the total number of ticks in which the computing unit transfers data to the thread's local memory; This represents the total number of ticks by which the computing unit transfers data to the shared buffer; This represents the total number of ticks in which the computing unit transfers data to global memory; This represents the total number of ticks by which the computation unit transfers data to the matrix reduction units in the loading and storage cache; This represents the total number of ticks by which computing units transfer data to the cluster's shared cache; This represents the total number of ticks transferred by the computing unit to the general buffer in the load memory cache; This represents the total number of ticks by which the computation unit transfers image data to the tensor indexing units in the loading and storage cache; This represents the total number of ticks transferred from the computation unit to the thread bundle shuffling unit in the load storage cache.

[0252] The total amount of data loaded from the storage cache to the data engine is calculated. The data engine is the core hardware component in the GPU that performs complex data handling, transmission scheduling, and preprocessing. The specific calculation formula is shown in the following formula (51):

[0253] (51)

[0254] in, This indicates the total amount of data loaded from the storage cache into the data engine; This represents the total number of ticks from loading the matrix calculation unit in the storage cache to loading the matrix type data into the data engine; This represents the total number of ticks of texture data transferred from thread-local memory to the data engine; This represents the total number of ticks from loading data from the shared cache in the storage cache to the data engine; This represents the total number of ticks of data transferred from global memory to the data engine; This represents the total number of ticks in loading data from matrix reduction units in the storage cache to the data engine; This represents the total number of ticks in the cluster's shared cache to the data engine; This represents the total number of ticks in the load storage cache from the general buffer to the data engine; This represents the total number of ticks in loading image data from tensor index units in the storage cache to the data engine; This represents the total number of ticks from loading the thread bundle shuffle unit in the storage cache to exiting the data engine.

[0255] Furthermore, the total bandwidth from the computing unit to the loading storage cache is calculated as shown in the following formula (52):

[0256] (52)

[0257] in, This represents the total bandwidth from the computing unit to the load storage cache; This represents the total amount of data from the computing unit to the loaded storage cache; This indicates the number of active cycles for loading the storage cache.

[0258] The total bandwidth from loading the storage cache to the data engine is calculated as shown in the following formula (53):

[0259] (53)

[0260] in, This represents the total bandwidth used to load storage cache into the data engine; This indicates the total amount of data loaded from the storage cache into the data engine; This indicates the number of active cycles for loading the storage cache.

[0261] In addition, in this embodiment of the application, besides the activity rate of the loading storage cache, the traffic of the first-level cache and the shared cache can also be detected. The traffic calculation formula of the shared cache is as shown in the following formula (54):

[0262] (54)

[0263] in, This indicates traffic from the shared cache; This represents the total number of bytes read by the tensor core from the shared cache; This represents the total number of bytes written from the tensor core to the shared cache; This represents the total number of bytes read from the shared buffer by all stream processors; This represents the total number of bytes written to the shared buffer by all stream processors.

[0264] The traffic calculation formula for the first-level cache is shown in the following formula (55):

[0265] (55)

[0266] in, This represents the traffic to the first-level cache; This represents the total number of bytes read from the L1 cache by all stream processors; This represents the total number of bytes written to the L1 cache by all stream processors.

[0267] For the L1 cache, its instruction flow can also be calculated, and the specific calculation formula is shown in the following formula (56):

[0268] (56)

[0269] in, Indicates the instruction flow in the L1 cache; This represents the number of instruction ticks transferred from the cache block interface to the L1 cache.

[0270] Based on the same technical concept, this application provides a schematic diagram of the structure of a cache performance testing device, such as... Figure 8 As shown, the cache performance testing device 800 includes:

[0271] The active module 801 is used to obtain the activity rate of the loading storage cache based on the number of active cycles of the loading storage cache and the number of active cycles of the application running on the artificial intelligence chip.

[0272] Interface module 802 is used to obtain the blocking status of the interface component based on the blocking rate of the interface component in the loading storage cache in the request channel dimension and the data channel dimension, respectively.

[0273] The communication module 803 is used to obtain the blocking status of the communication component based on the blocking rate of the communication component in the data receiving channel dimension and the return channel dimension, respectively, based on the loading storage cache.

[0274] The detection module 804 is used to obtain the performance detection result of loading the storage cache based on the activity rate, the blocking status of the interface component, and the blocking status of the communication component.

[0275] The detection module 804 is also used for:

[0276] The full blocking rate of the first-level cache is obtained based on the number of stalled cycles when the first-level cache is full and the number of active cycles when loading the storage cache.

[0277] The full blocking rate of the shared cache is obtained based on the number of stalled cycles when the shared cache is full and the number of active cycles when loading the storage cache.

[0278] The detection module 804 is also used for:

[0279] Get the number of standby cycles in the L1 cache caused by read-write conflicts;

[0280] The blocking rate caused by the read-write conflict is obtained based on the number of stalled cycles of the first-level cache and the number of active cycles of the load storage cache.

[0281] The detection module 804 is also used for:

[0282] Get the number of standby cycles in the L1 cache caused by read-write conflicts;

[0283] The blocking rate caused by the read-write conflict is obtained based on the number of stalled cycles of the first-level cache and the number of active cycles of the load storage cache.

[0284] Based on the cache hit rate, cache reuse rate and cache full load rate of the first-level cache within a preset time period, the overall turbulence score of the first-level cache is obtained.

[0285] The cache efficiency of the first-level cache is obtained based on the cache hit rate, the blocking rate caused by the read-write conflict, and the turbulence comprehensive score.

[0286] The detection module 804 is specifically used for:

[0287] Determine the number of hits and misses from the total number of operations on the first-level cache, and obtain the number of cache blocks that are allowed to be evicted from the first-level cache;

[0288] The cache hit rate is obtained based on the number of hits and the total number of operations; and the cache reuse rate is obtained based on the number of misses and the number of cache blocks.

[0289] The number of full-load cycles when the amount of data in the first-level cache is lower than the full-load baseline is obtained, and the cache full-load rate is determined based on the number of full-load cycles and the number of active cycles of loading the storage cache.

[0290] The detection module 804 is also used for:

[0291] Based on the cache full load rate and the cache hit rate, the efficiency of the first-level cache under pressure is determined.

[0292] The detection module 804 is also used for:

[0293] The number of the first cycles in which write conflicts occurred in the shared cache within a preset time period is obtained, as well as the number of the first bytes written to the shared cache within the preset time period;

[0294] The write conflict rate is obtained based on the first cycle number and the first byte number;

[0295] Obtain the second number of cycles in which read conflicts occur in the shared cache within the preset time period, and the second number of bytes read from the shared cache within the preset time period;

[0296] The read conflict rate is obtained based on the second cycle number and the second byte number;

[0297] Based on the write conflict rate and the read conflict rate, the total conflict rate of the shared cache is obtained;

[0298] The caching efficiency of the shared cache is obtained based on the total conflict rate of the shared cache.

[0299] The detection module 804 is specifically used for:

[0300] The third cycle number of atomic operation conflicts that occurred in the shared cache within the preset time period is obtained, as well as the third byte number of the atomic operation executed within the preset time period; the atomic operation is the smallest uninterruptible execution unit in a concurrent scenario.

[0301] The atomic collision rate is obtained based on the third period number and the third byte number.

[0302] The total conflict rate of the shared cache is obtained by weighted summing of the write conflict rate, the read conflict rate, and the atomic conflict rate.

[0303] In this embodiment, the activity rate of the load storage cache is obtained based on the number of active cycles of the load storage cache and the number of active cycles of the application running on the artificial intelligence chip, enabling overall detection of the load storage cache. Based on the interface components in the load storage cache, the blocking rate of the interface components is obtained in the request channel dimension and the data channel dimension, respectively. Based on the communication components in the load storage cache, the blocking rate of the communication components is obtained in the data receiving channel dimension and the return channel dimension, respectively. This allows for in-depth detection of the blocking status of multiple components under the load storage cache in different dimensions, improving the coverage of performance detection of multiple components. Thus, based on the activity rate, the blocking status of interface components, and the blocking status of communication components, the performance detection results of the load storage cache are obtained, improving the accuracy of the performance detection of the load storage cache.

[0304] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 9 As shown, it includes at least one artificial intelligence chip 100 and a memory 901 connected to at least one artificial intelligence chip 100. In this embodiment, the specific connection medium between the artificial intelligence chip 100 and the memory 901 is not limited. Figure 9 Taking the connection between the AI ​​chip 100 and the memory 901 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0305] In this embodiment, the memory 901 stores instructions that can be executed by at least one artificial intelligence chip 100. By executing the instructions stored in the memory 901, the at least one artificial intelligence chip 100 can perform the steps of the above-described cache performance detection method.

[0306] The artificial intelligence chip 100 serves as the control center of the computer device. It connects to various parts of the computer device via various interfaces and lines, and performs cache performance testing by running or executing instructions stored in the memory 901 and accessing data stored in the memory 901. Optionally, the artificial intelligence chip 100 may include one or more processing units. The artificial intelligence chip 100 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 901 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0307] The artificial intelligence chip 100 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0308] Memory 901, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 901 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 901 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. Memory 901 in the embodiments of this application may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0309] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described cache performance detection method.

[0310] Based on the same inventive concept, this application provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described cache performance detection method.

[0311] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0312] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0313] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0314] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0315] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method of detecting cache performance, characterized by, The method comprises the following steps: obtaining an activity rate of the load-store cache based on the number of active cycles of the load-store cache and the number of active cycles of an application running on an artificial intelligence chip; obtaining the blocking condition of the interface component in the load-store cache based on the blocking rate of the interface component in the request channel dimension and the data channel dimension respectively; obtaining the blocking condition of the communication component in the load-store cache based on the blocking rate of the communication component in the data receiving channel dimension and the return channel dimension respectively; obtaining the performance detection result of the load-store cache based on the activity rate, the blocking condition of the interface component and the blocking condition of the communication component.

2. The method of claim 1, wherein, The load-store cache further comprises a level one cache and a shared cache; and the performance detection result further comprises a full blocking rate of the level one cache and a full blocking rate of the shared cache; The full blocking rate of the level one cache and the full blocking rate of the shared cache are obtained in the following manner: obtaining the full blocking rate of the level one cache based on the number of idle cycles of the level one cache in the full state and the number of active cycles of the load-store cache; obtaining the full blocking rate of the shared cache based on the number of idle cycles of the shared cache in the full state and the number of active cycles of the load-store cache.

3. The method of claim 1, wherein, The load-store cache further comprises a level one cache; and the performance detection result further comprises a blocking rate caused by a read-after-write conflict of the level one cache; the blocking rate caused by the read-after-write conflict is obtained in the following manner: obtaining the number of idle cycles of the level one cache caused by the read-after-write conflict; obtaining the blocking rate caused by the read-after-write conflict based on the number of idle cycles of the level one cache and the number of active cycles of the load-store cache.

4. The method of claim 1, wherein, The load-store cache further comprises a level one cache; and the performance detection result further comprises a cache efficiency of the level one cache; the cache efficiency of the level one cache is obtained in the following manner: obtaining the number of idle cycles of the level one cache caused by the read-after-write conflict; obtaining the blocking rate caused by the read-after-write conflict based on the number of idle cycles of the level one cache and the number of active cycles of the load-store cache; obtaining a comprehensive score of the level one cache based on a cache hit rate, a cache reuse rate and a cache full load rate of the level one cache within a preset time period; obtaining the cache efficiency of the level one cache based on the cache hit rate, the blocking rate caused by the read-after-write conflict and the comprehensive score of the level one cache.

5. The method of claim 4, wherein, The cache hit rate, the cache reuse rate and the cache full load rate are obtained in the following manner: determining a hit number and a miss number in a total operation number for the level one cache, and obtaining a number of cache blocks allowed to be evicted in the level one cache; obtaining the cache hit rate based on the hit number and the total operation number, and obtaining the cache reuse rate based on the miss number and the number of cache blocks; obtaining a full load cycle number of the level one cache in which a data amount is lower than a full load baseline, and determining the cache full load rate based on the full load cycle number and the number of active cycles of the load-store cache.

6. The method of claim 4, wherein, The performance detection result further includes: the efficiency of the level one cache under stress; the efficiency of the level one cache under stress is obtained in the following way: Based on the cache fullness rate and the cache hit rate, the efficiency of the level one cache under stress is determined.

7. The method of claim 1, wherein, The load-store cache further includes: a shared cache; the performance detection result further includes: the cache efficiency of the shared cache; the cache efficiency of the shared cache is obtained in the following way: The first number of cycles of write conflict of the shared cache in a preset period of time is obtained, and the first number of bytes written to the shared cache in the preset period of time is obtained; Based on the first number of cycles and the first number of bytes, a write conflict rate is obtained; The second number of cycles of read conflict of the shared cache in the preset period of time is obtained, and the second number of bytes read from the shared cache in the preset period of time is obtained; Based on the second number of cycles and the second number of bytes, a read conflict rate is obtained; Based on the write conflict rate and the read conflict rate, a total conflict rate of the shared cache is obtained; Based on the total conflict rate of the shared cache, the cache efficiency of the shared cache is obtained.

8. The method of claim 7, wherein, The total conflict rate of the shared cache is obtained based on the write conflict rate and the read conflict rate, including: The third number of cycles of atomic operation conflict of the shared cache in the preset period of time is obtained, and the third number of bytes of the atomic operation executed in the preset period of time is obtained; the atomic operation is the smallest execution unit that cannot be interrupted in a concurrent scenario; Based on the third number of cycles and the third number of bytes, an atomic conflict rate is obtained; The write conflict rate, the read conflict rate and the atomic conflict rate are weighted and summed to obtain the total conflict rate of the shared cache. 9.A computer device, comprising a memory, an artificial intelligence chip, and a computer program stored on the memory and running on the artificial intelligence chip, wherein, The artificial intelligence chip executes the computer program to realize the steps of the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It stores a computer program executed by a computer device, and when the computer program runs on the computer device, it makes the computer device execute the steps of the method of any one of claims 1-8.

Images