Secondary cache performance analysis method, device and medium

Abstract

Embodiments of the present application provide a secondary cache performance analysis method, device and medium, to accurately and comprehensively analyze the performance of the secondary cache. The method comprises: obtaining statistical characteristic data corresponding to access requests received by the secondary cache in a sampling period; determining performance indicators of the secondary cache according to the statistical characteristic data; the performance indicators of the secondary cache include at least one of the following: a first type of indicator representing request type distribution characteristics and data transmission characteristics, a second type of indicator representing request hit rate and response efficiency of the secondary cache, a third type of indicator representing cache utilization rate, and a fourth type of indicator representing cache jolt degree; and performing performance analysis on the secondary cache based on the performance indicators of the secondary cache. By determining the performance indicator values corresponding to different performance indicators of the secondary cache, the performance of the secondary cache can be accurately and comprehensively analyzed from multiple dimensions.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to methods, devices and media for analyzing the performance of secondary cache. Background Technology

[0002] With the widespread application of Graphics Processing Units (GPUs) in high-performance computing, artificial intelligence, and other fields, GPU performance optimization has become increasingly important. As a key component of the GPU memory subsystem, the performance of the L2 cache directly impacts the overall computing efficiency and memory access bandwidth of the GPU. Therefore, accurate and comprehensive performance analysis of the L2 cache is crucial. Current technologies that focus solely on single performance metrics such as L2 cache hit rate cannot fully reflect the performance status of the L2 cache system. Summary of the Invention

[0003] This application provides a method, device, and medium for performing performance analysis of the secondary cache, which can be used to perform accurate and comprehensive performance analysis of the secondary cache.

[0004] In a first aspect, embodiments of this application provide a method for performance analysis of a second-level cache, comprising: acquiring statistical characteristic data corresponding to access requests received by the second-level cache within a sampling period; determining performance indicators of the second-level cache based on the statistical characteristic data; the performance indicators of the second-level cache include at least one of the following: a first type of indicator for characterizing the request type distribution characteristics and data transmission characteristics of the second-level cache, a second type of indicator for characterizing the request hit rate and response efficiency of the second-level cache, a third type of indicator for characterizing the cache utilization rate of the second-level cache, and a fourth type of indicator for characterizing the cache thrashing degree of the second-level cache; and performing performance analysis on the second-level cache based on the performance indicators of the second-level cache.

[0005] In the above method, by determining the performance index values ​​corresponding to different performance indicators of the second-level cache based on the statistical feature data corresponding to the access requests received by the second-level cache within the sampling period, it is helpful to conduct accurate and comprehensive performance analysis of the second-level cache from multiple dimensions.

[0006] Secondly, embodiments of this application provide a performance analysis apparatus for a secondary cache, comprising:

[0007] The acquisition unit is used to acquire statistical feature data corresponding to the access requests received by the second-level cache within the sampling period;

[0008] The determination unit is used to determine the performance indicators of the second-level cache based on statistical characteristic data. The performance indicators of the second-level cache include at least one of the following: a first type of indicator used to characterize the request type distribution characteristics and data transmission characteristics of the second-level cache; a second type of indicator used to characterize the request hit rate and response efficiency of the second-level cache; a third type of indicator used to characterize the cache utilization rate of the second-level cache; and a fourth type of indicator used to characterize the cache thrashing degree of the second-level cache.

[0009] The analysis unit is used to perform performance analysis on the L2 cache based on its performance metrics.

[0010] Thirdly, this application provides a computer device, including a memory, a processor chip, and a computer program stored in the memory and executable on the processor chip, wherein the processor chip executes the computer program to implement the method in the first aspect or any possible implementation of the first aspect.

[0011] Fourthly, this application provides a computer-readable storage medium including computer-executable instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect or any possible implementation thereof.

[0012] Fifthly, this application provides a computer program product storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method in the first aspect or any possible implementation thereof.

[0013] The technical effects that can be achieved by any of the second to fifth aspects mentioned above can be referred to the description of the beneficial effects in the first aspect or any possible implementation of the first aspect, and will not be repeated here. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of an artificial intelligence chip provided in an embodiment of this application;

[0015] Figure 2 A flowchart illustrating a secondary cache performance analysis method provided in this application embodiment;

[0016] Figure 3 A schematic diagram of the performance analysis device provided in the embodiments of this application;

[0017] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and beneficial effects of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] Figure 1 This is a schematic diagram of the structure of an artificial intelligence chip applicable to the embodiments of this application.

[0020] like Figure 1 As shown, the artificial intelligence chip 100 includes at least multiple computing units 101, multiple level 1 caches (L1 cache, also known as L1 cache) 102, level 2 caches (L2 cache, also known as L2 cache) 103, and video memory 104.

[0021] The computing unit 101 is, for example, a streaming processing cluster (SPC). Each computing unit 101 corresponds to a private level 1 cache 102, and the computing unit 101 can access its own private level 1 cache 102. The level 2 cache 103 can be a shared cache for multiple computing units, and multiple computing units can access the level 2 cache 103.

[0022] The video memory 104 can be high-bandwidth memory (HBM) or other types of memory.

[0023] The access speed of the computing unit 101 to the above-mentioned storage structures, from fastest to slowest, is as follows: Level 1 cache 102, Level 2 cache 103, and video memory 104.

[0024] The computing unit 101 can read and write data on any of the above-mentioned storage structures.

[0025] Taking the computation unit 101 reading data from the storage structure for computation as an example, the computation unit 101 can send a read request to the private first-level cache 102. The first-level cache 102 checks whether there is a hit. If the check result of the first-level cache 102 is a hit, that is, the first-level cache 102 has the data corresponding to the read request, then the first-level cache 102 returns the data to the computation unit 101; if the check result of the first-level cache 102 is a miss, that is, the first-level cache 102 does not have the data corresponding to the read request, then the first-level cache 102 sends the read request to the second-level cache 103.

[0026] After receiving the read request, the L2 cache 103 checks whether it has a hit. If the L2 cache 103 checks and finds a hit, it sends the data corresponding to the read request back to the L1 cache 102, and the L1 cache 102 sends the data obtained from the L2 cache 103 to the computing unit 101. If the L2 cache 103 checks and finds a miss, it sends the read request to the video memory 104.

[0027] If the data corresponding to the read request exists in the video memory 104, the video memory 104 responds to the read request by returning the data corresponding to the read request to the secondary cache 103. Then, the secondary cache 103 sends the read data back to the primary cache 102, and the primary cache 102 sends the data obtained from the secondary cache 103 to the computing unit 101. Optionally, the primary cache 102 and / or the secondary cache 103 may also write the read data locally to respond to the next received read request.

[0028] It should be understood that the artificial intelligence chip in this application, in addition to containing Figure 1 In addition to the structure shown, other structures may also be included. For example, an artificial intelligence chip may also be a multi-die structure, that is, an artificial intelligence chip includes multiple dies, each die including multiple computing units, a private L1 cache for each computing unit, a shared L2 cache for multiple computing units within the die, and video memory.

[0029] This application does not impose any limitations on this. Nor does it limit the number of computing units included in the artificial intelligence chip.

[0030] In this embodiment of the application, the artificial intelligence chip can be a graphics processing unit (GPU), a general-purpose computing on graphics processing unit (GPGPU), etc.

[0031] As a key component of the GPU memory subsystem, the performance of the L2 cache directly impacts the overall computing efficiency and memory access bandwidth of the GPU. Therefore, accurate and comprehensive performance analysis of the L2 cache is crucial. However, current techniques that focus solely on single performance metrics such as L2 cache hit rate cannot fully reflect the overall performance of the L2 cache system.

[0032] In view of this, this application provides a method for secondary cache performance analysis, which can be executed by a performance analysis device, such as... Figure 2 As shown, the secondary cache performance analysis method provided in this application includes the following steps:

[0033] Step 201: Obtain statistical feature data corresponding to the access requests received by the second-level cache within the sampling period.

[0034] The secondary cache can receive access requests of various types within a sampling period. For example, based on the operation type of the request, the access request type can include, but is not limited to, at least one of the following: read request, write request, atomic operation request, reduction request, and bundle reduction request. Alternatively, based on access attributes, the access request type can include, but is not limited to, normal read request, streaming read request, and persistent read request. This application does not limit the specific duration of a sampling period.

[0035] Statistical characteristics include, but are not limited to, at least one of the following: the number of access requests of different types, the number of ticks of data transmitted in response to access requests, the number of request hits, the number of latency cycles for requests, the number of cycles in which the incomplete queue for different types of requests is full, the number of active cycles of the secondary cache, the number of blocking cycles of the HBM interface AR channel, the number of blocking cycles of the HBM interface R channel, the number of blocking cycles of the HBM interface W channel; the number of blocking cycles of the NIU replication R channel; the number of starvation cycles of the HBM interface R channel, the number of starvation cycles of the HBM interface W channel; the number of starvation cycles of the NIU replication R channel; the number of access requests for each sector; and the number of cache line accesses for each set.

[0036] Step 202: Determine the performance metrics of the second-level cache based on the statistical feature data; wherein the performance metrics of the second-level cache include at least one of the following: a first type of metric used to characterize the request type distribution characteristics and data transmission characteristics of the second-level cache, a second type of metric used to characterize the request hit rate and response efficiency of the second-level cache, a third type of metric used to characterize the cache utilization rate of the second-level cache, and a fourth type of metric used to characterize the cache thrashing degree of the second-level cache.

[0037] Step 203: Perform performance analysis on the second-level cache based on its performance metrics.

[0038] The specific implementation of steps 202 and 203 above will be explained below using different statistical characteristic data.

[0039] To evaluate the request type distribution characteristics and data transmission characteristics of the second-level cache, the first type of indicators mentioned above can be determined based on statistical characteristic data. The first type of indicators includes at least one or more of the following: special operation rate indicator, traffic indicator, and average burst length indicator.

[0040] The aforementioned special operation rate metric is used to represent the percentage of access requests of different operation types received by the second-level cache out of the total number of access requests received by the second-level cache. The special operation rate metric includes at least one of the following: atomic operation request rate, reduction request rate, and in-bundle reduction (WRED) request rate. Based on at least one of the atomic operation request rate, reduction request rate, and in-bundle reduction request rate included in the special operation rate metric, the distribution characteristics of requests of at least one operation type received by the second-level cache are evaluated.

[0041] The atomic operation request rate is the percentage of all atomic operation requests received by the second-level cache to the total number of requests received by the second-level cache. The atomic operation request rate L2_atomic_request_rate can be obtained by the following formula:

[0042]

[0043] Here, L2_req_atomic_spc represents the number of atomic operation requests received by the L2 cache from the Stream Processor Cluster (SPC), L2_req_atomics_cp represents the number of atomic operation requests received by the L2 cache from the Compute Processor (CP), L2_req_atomics_sdma represents the number of atomic operation requests received by the L2 cache from the System Direct Memory Access (SDMA), and L2_total_requests represents the total number of requests received by the L2 cache.

[0044] The total number of requests received by the second-level cache (L2_total_requests) satisfies the following formula:

[0045] L2_total_requests=L2_req_read_hit_test+L2_req_write_hit_test+

[0046] L2_atomic_hit_test+L2_wred_hit_test

[0047] Where L2_req_read_hit_test is the number of read requests received by the second-level cache, L2_req_write_hit_test is the number of write requests received by the second-level cache, L2_atomic_hit_test is the number of atomic operation requests received by the second-level cache, and L2_wred_hit_test is the number of WRED operation requests received by the second-level cache.

[0048] The performance analysis device can determine the atomic operation request rate based on statistical feature data such as L2_req_atomic_spc, L2_req_atomics_cp, L2_req_atomics_sdma, L2_req_read_hit_test, L2_req_write_hit_test, L2_atomic_hit_test, and L2_wred_hit_test. Then, it evaluates the distribution characteristics of atomic operation requests in the total requests received in the L2 cache based on the atomic operation request rate.

[0049] The reduction request rate is the percentage of reduction requests received by the second-level cache out of the total number of requests received by the second-level cache. The reduction request rate satisfies the following formula:

[0050]

[0051] Where L2_reduce_request_rate is the reduction request rate, L2_req_reduce_cp is the number of reduction operation requests received by the second-level cache and initiated by CP, and L2_total_requests is the total number of requests received by the second-level cache.

[0052] The performance analysis device can determine the reduction request rate based on statistical characteristic data such as L2_req_reduce_cp, L2_req_read_hit_test, L2_req_write_hit_test, L2_atomic_hit_test, and L2_wred_hit_test. Then, it can evaluate the distribution of reduction requests in the total requests received in the L2 cache based on the reduction request rate.

[0053] The WRED request rate is the percentage of WRED requests received by the second-level cache out of the total number of requests received by the second-level cache. The WRED request rate satisfies the following formula:

[0054]

[0055] Where L2_wred_request_rate is the WRED request rate, L2_req_wred_spc is the number of intra-bundle reduction operation requests received by the L2 cache and initiated by SPC, L2_req_wred_p2p is the number of intra-bundle reduction operation requests received by the L2 cache that occurred in peer-to-peer communication, peer-to-peer (p2p) communication refers to the ability of GPUs to transfer data directly without going through host memory, and L2_total_requests is the total number of requests received by the L2 cache.

[0056] The performance analysis device can determine the WRED request rate based on statistical characteristic data such as L2_req_wred_spc, L2_req_wred_p2p, and L2_total_requests. Then, it can evaluate the distribution characteristics of WRED requests in the total requests received in the second-level cache based on the WRED request rate.

[0057] The aforementioned traffic metrics include at least one of the following: Network Interface Unit (NIU) read traffic, NIU write traffic, HBM read traffic, HBM write traffic, and NIU copy traffic. Based on at least one of the aforementioned traffic metrics (NIU read traffic, NIU write traffic, HBM read traffic, HBM write traffic, and NIU copy traffic), the traffic characteristics generated by the secondary cache are evaluated.

[0058] NIU read traffic is the total number of bytes read by the compute unit from the L2 cache through the NIU interface. NIU read traffic satisfies the following formula:

[0059]

[0060] Where L2_niu_read_bytes represents the NIU read traffic, and L2_beats_niu_read represents the number of NIU read beats. A beat is the basic unit of memory or interconnect bus transmission. One beat corresponds to one data block transmitted on the data bus, and its size is, for example, 64 or 128 bytes, depending on the specific bus protocol and configuration. In this embodiment, a beat of 128 bytes is used as an example for illustration, and will not be elaborated further below.

[0061] The performance analysis device can determine the NIU read traffic based on the acquired statistical characteristic data L2_beats_niu_read, and then assess the read load on the L2 cache based on the NIU read traffic.

[0062] NIU write traffic is the total number of bytes written by the compute unit to the L2 cache through the NIU interface. NIU write traffic satisfies the following formula:

[0063] L2_niu_write_bytes= L2_beats_niu_write 128

[0064] Where L2_niu_write_bytes is the NIU write traffic and L2_beats_niu_write is the NIU write beats.

[0065] The performance analysis device can determine the NIU write traffic based on the acquired statistical characteristic data L2_beats_niu_write, and then assess the write load on the L2 cache based on the NIU write traffic.

[0066] HBM read traffic is the total number of bytes read from HBM by main port 1 (mst1) of the L2 cache. HBM read traffic satisfies the following formula:

[0067] L2_hbm_mst1_read_bytes = L2_beats_hbm_mst1_read 64

[0068] Where L2_hbm_mst1_read_bytes is the HBM read traffic, and L2_beats_hbm_mst1_read is the number of beats read from HBM by main port 1 of the L2 cache.

[0069] The performance analysis device can determine the HBM read traffic based on the acquired statistical characteristic data L2_beats_hbm_mst1_read. Then, based on the HBM read traffic, it can evaluate the read traffic generated when the GPU reads data from HBM memory through a specific main port of the L2 cache, which can be used to analyze memory read bandwidth.

[0070] HBM write traffic is the total number of bytes written from main port 1 (mst1) of the L2 cache to HBM. HBM write traffic satisfies the following formula:

[0071] L2_hbm_mst1_write_bytes= L2_beats_hbm_mst1_write 64

[0072] Where L2_hbm_mst1_write_bytes is the HBM write traffic, and L2_beats_hbm_mst1_write is the number of beats written from main port 1 of the L2 cache to HBM.

[0073] The performance analysis device can determine the HBM write traffic based on the acquired statistical characteristic data L2_beats_hbm_mst1_write. Then, based on the HBM write traffic, it can evaluate the write traffic generated when the GPU writes data to the HBM memory through a specific main port of the L2 cache, which can be used to analyze memory write bandwidth.

[0074] NIU copy traffic is the total number of bytes generated by the NIU through master port 0 (mst0) during copy operations. NIU copy traffic satisfies the following formula:

[0075] L2_niu_copy_bytes = L2_beats_niu_copy_mst0 64

[0076] Where L2_niu_copy_bytes is the NIU replication traffic, and L2_beats_niu_copy_mst0 is the number of beats for the replication operation through master port 0 (mst0).

[0077] The performance analysis device can determine NIU copy traffic based on the acquired statistical characteristic data L2_beats_niu_copy_mst0, and then, based on the NIU copy traffic, it can evaluate the traffic characteristics generated at the L2 cache by direct memory access (DMA) copy operations performed through NIU (e.g., data copying between GPUs or between GPU and system memory).

[0078] The aforementioned average burst length metric is used to characterize the amount of data continuously transmitted in a single bus transaction or access request. The average burst length metric includes at least one of the following: NIU read average burst length, NIU write average burst length, NIU copy average burst length, HBM read average burst length, and HBM write average burst length. Based on at least one of the NIU read average burst length, NIU write average burst length, NIU copy average burst length, HBM read average burst length, and HBM write average burst length included in the average burst length metric, the data transmission efficiency corresponding to requests initiated through NIU or HBM is evaluated.

[0079] The NIU read average burst length is the average number of data ticks transmitted by the computing unit in each read request sent to the L2 cache through the network interface unit. The NIU read average burst length satisfies the following formula:

[0080]

[0081] Where L2_niu_read_avg_bursts is the average burst length of NIU reads, L2_beats_niu_read is the total number of beats read through NIU, and L2_req_niu_read is the total number of requests read through NIU.

[0082] The performance analysis device can determine the average burst length of NIU reads based on the acquired statistical characteristic data L2_beats_niu_read and L2_req_niu_read, and then evaluate the transmission efficiency when reading data through the network interface based on the average burst length of NIU reads.

[0083] The NIU write average burst length is the average number of data ticks transmitted by the compute unit in each write request sent to the L2 cache through the network interface unit. The NIU write average burst length satisfies the following formula:

[0084]

[0085] Wherein, L2_niu_write_avg_bursts is the average burst length of NIU writes, L2_beats_niu_write is the total number of beats written through NIU, and L2_req_niu_write is the total number of write requests through NIU. The performance analysis device can determine the average burst length of NIU writes based on statistical characteristic data such as L2_beats_niu_write and L2_req_niu_write.

[0086] The NIU replication average burst length is the average number of data ticks transmitted during each replication operation via the main port 0 of the network interface unit. The NIU replication average burst length satisfies the following formula:

[0087]

[0088] Wherein, L2_niu_copy_avg_bursts is the average burst length of NIU replication, L2_beats_niu_copy_mst0 is the total number of replication cycles performed through the main port 0 of the network interface unit, and L2_req_niu_copy_mst_0 is the total number of replication requests. The performance analysis device can determine the average burst length of NIU writes based on statistical characteristic data such as L2_beats_niu_copy_mst0 and L2_req_niu_copy_mst_0.

[0089] The average burst length of an HBM read is the average number of data ticks transmitted per read from main port 1 of the L2 cache. The average burst length of an HBM read satisfies the following formula:

[0090]

[0091] Here, L2_hbm_mst1_read_avg_bursts represents the average burst length of HBM reads, L2_beats_hbm_mst1_read represents the total number of beats read from HBM via main port 1 of the L2 cache, and L2_req_hbm_mst1_read represents the total number of read requests. The performance analysis device can determine the average burst length of NIU writes based on statistical characteristic data such as L2_beats_hbm_mst1_read and L2_req_hbm_mst1_read.

[0092] The average burst length of an HBM write is the average number of data ticks transmitted per write request initiated to HBM via main port 1 of the L2 cache. The formula for its calculation is as follows:

[0093]

[0094] Among them, L2_hbm_mst1_write_avg_bursts is the average burst length of HBM writes, L2_beats_hbm_mst1_write is the total number of write cycles corresponding to data written to HBM through main port 1 of the L2 cache, and L2_req_hbm_mst1_write is the total number of write requests. The performance analysis device can determine the average burst length of NIU writes based on statistical characteristic data such as L2_beats_hbm_mst1_write and L2_req_hbm_mst1_write.

[0095] Among the secondary cache performance metrics in step 202 above, the second category of metrics includes at least one or more of the following: at least one cache hit rate metric, at least one latency analysis metric, at least one blocking analysis metric, and at least one starvation analysis metric.

[0096] At least one cache hit rate metric includes at least one of the following: read hit rate, normal read hit rate, streaming read hit rate, persistent read hit rate, write hit rate, actual write hit rate, normal write hit rate, streaming write hit rate, persistent write hit rate, WRED hit rate, atomic hit rate, and copy hit rate. The following describes how the performance analysis device determines the cache hit rate metrics for the secondary cache based on statistical characteristic data.

[0097] The read hit rate is the percentage of read requests of different types that are hit in the total number of read requests received by the second-level cache. The read hit rate satisfies the following formula:

[0098]

[0099] Here, L2_read_hit_rate is the read hit rate, L2_req_read_hit is the number of read request hits, and L2_req_read_hit_test is the number of read requests received by the second-level cache. For example, if the read requests received by the second-level cache include regular read requests, streaming read requests, and persistent read requests, then L2_req_read_hit is the sum of the number of hits (or hit counts) of regular read requests, streaming read requests, and persistent read requests received by the second-level cache, and L2_req_read_hit_test is the sum of the total number of regular read requests, streaming read requests, and persistent read requests received by the second-level cache.

[0100] It should be noted that data read / written to the second-level cache by streaming read / write requests can be replaced when a cacheline conflict is encountered for the first time; regular read / write requests use the default replacement strategy; and data currently read / written to the second-level cache by persistent read / write requests is not easily replaced.

[0101] The normal read hit rate is the percentage of normal read requests that are hit in the second-level cache. The read hit rate satisfies the following formula:

[0102]

[0103] Where L2_normal_read_hit_rate is the normal read hit rate, L2_req_normal_read_hit is the number of normal read requests that are hit, and L2_req_normal_read_hit_test is the total number of normal read requests.

[0104] The streaming read hit rate is the percentage of streaming read requests that are hit out of the total number of streaming read requests received by the second-level cache. The read hit rate satisfies the following formula:

[0105]

[0106] Where L2_streaming_read_hit_rate is the streaming read hit rate, L2_req_streaming_read_hit is the number of hits for streaming read requests, and L2_req_streaming_read_hit_test is the total number of accesses for streaming read requests.

[0107] The persistent read hit rate is the percentage of persistent read requests that are hit out of the total number of persistent read requests received by the second-level cache. The persistent read hit rate satisfies the following formula:

[0108]

[0109] Where L2_persist_read_hit_rate is the persistent read hit rate, L2_req_persist_read_hit is the number of persistent read requests that are hit, and L2_req_persist_read_hit_test is the total number of persistent read requests.

[0110] The performance analysis device can determine the performance indicators of the secondary cache based on the statistical characteristic data corresponding to each type of read request among the different types acquired. Then, based on the performance indicators and performance evaluation strategies, it can determine whether there are any abnormal performance problems. Step 203 above can be implemented as follows: For any type of read request received by the secondary cache, based on the read hit rate corresponding to that type of read request and the performance evaluation strategy corresponding to the read hit rate, it can determine whether the secondary cache has a high miss rate problem. The performance evaluation strategy corresponding to the read hit rate includes the condition that if the read hit rate corresponding to that type of read request is less than a first threshold, then the secondary cache has a high miss rate problem of that type. For example, different types of read requests include all read requests, ordinary read requests, and streaming read requests. Taking one type of read request as an example, such as a streaming read request, its corresponding statistical characteristic data includes the number of hits of the streaming read request and the total number of accesses of the streaming read request received by the secondary cache. The streaming read hit rate is determined, and then based on the streaming read hit rate and its corresponding performance evaluation strategy, it can be evaluated whether there is a high miss rate problem for streaming read requests. For example, the performance evaluation strategy corresponding to the streaming read hit rate includes the following: if the streaming read hit rate is less than a first threshold, then there is a high streaming read request miss rate problem. The value of the first threshold is not limited, for example, it can be 0.7.

[0111] In some embodiments, the performance evaluation strategy for different types of read requests can be set to be the same. For example, if the hit rate for any type of read request is less than a first threshold, there is a high miss rate problem.

[0112] In other embodiments, the performance evaluation strategies for different types of read requests can also be set differently. For example, the performance evaluation strategy for streaming read requests includes: if the streaming read hit rate is less than a first threshold, then there is a high miss rate problem for streaming read requests; if the streaming read hit rate is greater than or equal to the first threshold, then there is no high miss rate problem for streaming read requests. The performance evaluation strategy for ordinary read requests includes: if the ordinary read hit rate is less than a second threshold, then there is a high miss rate problem for ordinary read requests; if the ordinary read hit rate is greater than or equal to the second threshold, then there is no high miss rate problem for ordinary read requests.

[0113] The write hit rate is the percentage of write requests that hit different types of write requests out of the total number of write requests received by the second-level cache. The write hit rate satisfies the following formula:

[0114]

[0115] Where L2_write_hit_rate is the write hit rate, L2_req_write_hit is the number of write hits, and L2_req_write_hit_test is the total number of write request accesses.

[0116] The actual write hit rate is the percentage of actual write requests that are hit out of the total number of write requests received by the second-level cache. The actual write hit rate satisfies the following formula:

[0117]

[0118] Where L2_write_real_hit_rate is the actual write hit rate, L2_req_write_real_hit is the actual number of write hits, and L2_req_write_hit_test is the total number of actual write request accesses.

[0119] The difference between the actual write hit rate (L2_write_real_hit_rate) and the aforementioned write hit rate (L2_write_hit_rate) lies in the fact that L2_write_real_hit_rate considers the AXI bus write mask (wstrb) field. If the write mask field is all zeros and the AXI requested address hits the cache line, l2_write_hit_rate will increment, while l2_write_real_hit_rate will not. The AXI read channel does not have a mask field, so there is no distinction between "real" and "real," meaning there is no actual read hit rate metric.

[0120] The normal write hit rate is the percentage of normal write requests that are hit in the second-level cache. The normal write hit rate satisfies the following formula:

[0121]

[0122] Where L2_normal_write_hit_rate is the normal write hit rate, L2_req_normal_write_hit is the number of normal write hits, and L2_req_normal_write_hit_test is the total number of normal write requests.

[0123] The streaming write hit rate is the percentage of successful streaming write requests out of the total number of streaming write requests received by the second-level cache. The streaming write hit rate satisfies the following formula:

[0124]

[0125] Where L2_streaming_write_hit_rate is the streaming write hit rate, L2_req_streaming_write_hit is the number of streaming write hits, and L2_req_streaming_write_hit_test is the total number of streaming write request accesses.

[0126] The persistent write hit rate is the percentage of persistent write requests that are hit out of the total number of persistent write requests received by the second-level cache. The persistent write hit rate satisfies the following formula:

[0127]

[0128] Where L2_persist_write_hit_rate is the persistence write hit rate, L2_req_persist_write_hit is the number of persistence write hits, and L2_req_persist_write_hit_test is the total number of persistence write request accesses.

[0129] The WRED hit rate is the percentage of WRED requests that hit out of the total number of WRED requests received in the second-level cache. The WRED hit rate follows the formula:

[0130]

[0131] Where L2_wred_hit_rate is the WRED hit rate, L2_wred_hit is the number of WRED requests that were hit, and L2_wred_hit_test is the total number of WRED requests received by the second-level cache.

[0132] The atomic hit rate is the percentage of atomic operation requests that hit out of the total number of atomic operation requests received in the second-level cache. The atomic hit rate satisfies the following formula:

[0133]

[0134] Where L2_atomic_hit_rate is the atomic hit rate, L2_atomic_hit is the number of atomic operation requests that were hit, and L2_atomic_hit_test is the total number of atomic operation requests.

[0135] The replication hit rate is the percentage of the total number of successful replication operations across all domains (domain1, domain2, domain3) out of the total number of replication accesses across all domains. The replication hit rate satisfies the following formula:

[0136]

[0137] Where L2_copy_hit_rate is the copy hit rate, L2_req_copy_hit_domain1 is the total number of copy operation hits in domain 1, L2_req_copy_hit_domain2 is the total number of copy operation hits in domain 2, and L2_req_copy_hit_domain3 is the total number of copy operation hits in domain 3.

[0138] L2_req_copy_hit_domain1_test represents the total number of replication accesses for domain 1.

[0139] L2_req_copy_hit_domain2_test represents the total number of replication accesses for domain 2.

[0140] L2_req_copy_hit_domain3_test represents the total number of replication accesses for domain 3.

[0141] At least one latency analysis metric includes at least one of the following: average read latency, average write latency, average copy latency, HBM average read latency, and HBM average write latency. The following describes how the performance analysis device determines the latency analysis metrics for the L2 cache based on statistical characteristic data.

[0142] Average read latency is the average number of clock cycles elapsed from when the compute unit sends a read request to the L2 cache until the data is ready. Average read latency satisfies the following formula:

[0143]

[0144] Where L2_avg_read_latency is the average read latency, L2_cycles_read_latencies is the total number of latency cycles for all sampled read requests, and L2_samples_read is the total number of all sampled read requests.

[0145] Average write latency is the average number of clock cycles elapsed from when the compute unit sends a write request to the L2 cache to when the write operation is completed. Average write latency satisfies the following formula:

[0146]

[0147] Where L2_avg_write_latency is the average write latency, L2_cycles_write_latencies is the total number of latency cycles for all sampled write requests, and L2_samples_write is the total number of sampled write requests.

[0148] Average replication latency is the average latency of data transfer operations performed through a replication engine (such as DMA). Average replication latency satisfies the following formula:

[0149]

[0150] Where L2_avg_copy_latency is the average copy latency, L2_cycles_copy_latencies is the total number of latency cycles for all sampled copy operations, and L2_samples_copy is the total number of sampled copy operations.

[0151] The average read latency of HBM is the average number of clock cycles from initiating a read request from the L2 cache (or through L2) to the return of data from the HBM. The average read latency of HBM satisfies the following formula:

[0152]

[0153] Where L2_avg_hbm_mst1_read_latency is the average read latency of HBM, L2_cycles_hbm_mst1_read_latencies is the total number of latency cycles for all read requests initiated to HBM through the main port 1 of the L2 cache within the sampling period, and L2_samples_hbm_mst1_read is the total number of read requests initiated to HBM through the main port 1 of the L2 cache within the sampling period.

[0154] The average write latency of HBM is the average number of clock cycles required to write data from the L2 cache to the HBM. The average write latency of HBM satisfies the following formula:

[0155]

[0156] Where L2_avg_hbm_mst1_write_latency is the average write latency of HBM, L2_cycles_hbm_mst1_write_latencies is the total number of latency cycles for all write requests initiated to HBM through the main port 1 of the L2 cache within the sampling period, and L2_samples_hbm_mst1_write is the total number of write requests initiated to HBM through the main port 1 of the L2 cache within the sampling period.

[0157] Further analysis of the blocking situation can be conducted by identifying at least one blocking analysis indicator based on statistical characteristic data. This is to analyze the blocking situation of the L2 cache. The blocking situation usually refers to the situation where the data receiver is unable to receive data due to insufficient resources (such as a full buffer), causing the data sender to be forced to wait.

[0158] At least one blocking analysis metric includes at least one of the following: First-In-First-Out (FIFO) full blocking rate, Write FIFO full blocking rate, Copy FIFO full blocking rate, HBM interface AR channel blocking rate, HBM interface R channel blocking rate, HBM interface W channel blocking rate, NIU Copy AR channel blocking rate, and NIU Copy R channel blocking rate. The following describes how the performance analysis device determines the various blocking analysis metrics for the L2 cache based on statistical characteristic data.

[0159] The read FIFO full blocking rate is the proportion of time that the L2 cache's internal FIFO (FIFO for tracking incomplete read requests) is full, causing new read requests to be blocked and unable to be received. It satisfies the following formula:

[0160]

[0161] Among them, L2_read_outstanding_full_stall_rate is the full blocking rate of the read FIFO, L2_cycles_read_outstanding_full is the number of cycles when the read FIFO is full, and L2_cycles_active is the total number of active cycles of the L2 cache.

[0162] The write FIFO full blocking rate is the proportion of time the L2 cache's internal FIFO (FIFO for tracking incomplete write requests) is full, causing new write requests to be blocked and unable to be received. It satisfies the following formula:

[0163]

[0164] Among them, L2_write_outstanding_full_stall_rate is the write FIFO full blocking rate, L2_cycles_write_outstanding_full is the number of cycles when the write FIFO is full, and L2_cycles_active is the total number of active cycles of the L2 cache.

[0165] The replication FIFO full blocking rate is the proportion of time that new replication operations are blocked because the queue of incomplete requests for processing replication operations (such as DMA) is full, preventing them from being scheduled. It satisfies the following formula:

[0166]

[0167] Among them, L2_copy_outstanding_full_stall_rate is the full blocking rate of the copy FIFO, L2_cycles_copy_outstanding_full is the number of cycles in which the copy FIFO is full, and L2_cycles_active is the total number of active cycles in the L2 cache.

[0168] The HBM interface AR channel blocking rate is the percentage of time that the read address cannot be sent due to channel (AR) blocking when sending read request addresses to HBM.

[0169]

[0170] Where L2_hbm_mst1_ar_stall_rate is the blocking rate of the AR channel of the HBM interface, L2_cycles_stall_by_ar_hbm_mst1 is the number of blocking cycles of the AR channel, and L2_cycles_active is the total number of active cycles of the L2 cache.

[0171] The HBM interface R channel blocking rate is the proportion of time that data cannot be received in a timely manner due to blockage of the channel (R) for returning read data from HBM. It satisfies the following formula:

[0172]

[0173] Where L2_hbm_mst1_r_stall_rate is the blocking rate of the HBM interface R channel, L2_cycles_stall_r_hbm_mst1 is the number of blocking cycles of the HBM interface R channel, and L2_cycles_active is the total number of active cycles of the L2 cache.

[0174] The HBM interface W channel blocking rate is the proportion of time that write operations cannot be completed due to channel (W) blockage in sending write data and addresses to the HBM. It satisfies the following formula:

[0175]

[0176] Where L2_hbm_mst1_w_stall_rate is the blocking rate of the W channel of the HBM interface, L2_cycles_stall_by_w_hbm_mst1 is the number of blocking cycles of the W channel of the HBM interface, and L2_cycles_active is the total number of active cycles of the L2 cache.

[0177] The NIU Copy AR Channel Blocking Rate characterizes the proportion of time that the channel sending read request addresses to other GPUs is blocked during copy operations via NIU. It satisfies the following formula:

[0178]

[0179] Where L2_niu_copy_mst0_ar_stall_rate is the NIU copy AR channel blocking rate, L2_cycles_stall_by_ar_niu_copy_mst0 is the number of NIU copy AR channel blocking cycles, and L2_cycles_active is the total number of L2 cache active cycles.

[0180] The NIU replication R channel blocking rate is the proportion of time that channel R, which receives read data from the remote source, is blocked during replication operations via NIU master port 0. It satisfies the following formula:

[0181]

[0182] Where L2_niu_copy_mst0_r_stall_rate is the NIU copy R channel blocking rate, L2_cycles_stall_r_niucopy_mst0 is the number of R channel blocking cycles, and L2_cycles_active is the total number of L2 cache active cycles.

[0183] In one implementation of step 203, the performance analysis device can determine whether a blocking problem exists in the secondary cache based on at least one blocking analysis indicator and a performance evaluation strategy corresponding to the at least one blocking analysis indicator. The performance evaluation strategy corresponding to the at least one blocking analysis indicator includes at least one of the following:

[0184] If any of the read FIFO full blocking rate, write FIFO full blocking rate, or copy FIFO full blocking rate is greater than the third threshold, then the L2 cache has a high FIFO blocking rate problem; if the read FIFO full blocking rate, write FIFO full blocking rate, or copy FIFO full blocking rate are all less than the third threshold, then the L2 cache does not have a high FIFO blocking rate problem; the third threshold is, for example, 0.05, and this application does not impose any restrictions on it.

[0185] If the HBM interface AR channel blocking rate or the HBM interface W channel blocking rate is greater than the fourth threshold, then the L2 cache has a high HBM interface blocking rate problem; or, if the HBM interface AR channel blocking rate or the HBM interface W channel blocking rate is less than or equal to the fourth threshold, then the L2 cache does not have a high HBM interface blocking rate problem; the fourth threshold is, for example, 0.1, and this application does not limit it.

[0186] If the NIU copy AR channel blocking rate or the NIU copy R channel blocking rate is greater than the fifth threshold, then the L2 cache has a high NIU interface blocking rate. Alternatively, if the NIU copy AR channel blocking rate or the NIU copy R channel blocking rate is less than or equal to the fifth threshold, then the L2 cache does not have a high NIU interface blocking rate problem. The fifth threshold is, for example, 0.1, and this application does not impose any restrictions on it.

[0187] To further analyze the hunger situation, at least one hunger analysis indicator can be determined based on statistical characteristic data. Hunger typically refers to a situation where the data requester is ready to receive data, but the data provider is unable to supply the data in a timely manner. At least one hunger analysis indicator includes at least one of the following: HBM R channel hunger rate, HBM W channel hunger rate, and NIU replication R channel hunger rate.

[0188] HBM R-channel starvation rate characterizes the proportion of time when the L2 buffer is ready to receive data, but the HBM R-channel has no data to return (starvation). It satisfies the following formula:

[0189]

[0190] Where L2__hbm_mst1_r_starvation_rate is the HBM R channel starvation rate, L2_cycles_starve_by_r_hbm_mst1 is the number of HBM R channel starvation cycles, and L2_cycles_active is the total number of L2 cache active cycles.

[0191] HBM W channel hunger rate characterizes the proportion of time when the L2 cache is ready to send write data, but the HBM W channel is unable to receive it (e.g., the write buffer is full or the target bank is busy). It satisfies the following formula:

[0192]

[0193] Where L2_hbm_mst1_w_starvation_rate is the HBM W channel starvation rate, L2_cycles_starve_w_hbm_mst1 is the number of HBM W channel starvation cycles, and L2_cycles_active is the total number of L2 cache active cycles.

[0194] The NIU Copy (R) channel hunger rate characterizes the proportion of time during an NIU copy operation when the L2 buffer is ready to receive data, but the NIU master port 0 data read (R) channel does not return any data. It satisfies the following formula:

[0195]

[0196] Where L2_niu_copy_mst0_r_starvation_rate is the starvation rate of the NIU copy R channel, L2_cycles_starve_by_r_niu_copy_mst0 is the number of starvation cycles of the NIU copy R channel, and L2_cycles_active is the total number of active cycles of the L2 cache.

[0197] In one implementation of step 203, the performance analysis device can determine whether a starvation problem exists in the secondary cache based on at least one starvation analysis indicator and a performance evaluation strategy corresponding to the at least one starvation analysis indicator. The performance evaluation strategy corresponding to the at least one starvation analysis indicator includes at least one of the following:

[0198] If the HBM R channel starvation rate is greater than the sixth threshold, then the L2 cache has a high HBM starvation rate problem; the sixth threshold is, for example, 0.1, and this application does not impose any restrictions on it.

[0199] If the NIU replication R channel starvation rate is greater than the seventh threshold, then the L2 cache has a high NIU starvation rate problem; the seventh threshold is, for example, 0.1, and this application does not impose any restrictions on it.

[0200] If the HBM R channel starvation rate is less than or equal to the sixth threshold or the NIU replication R channel starvation rate is less than or equal to the seventh threshold, then the secondary cache does not have a starvation problem.

[0201] Among the secondary cache performance metrics in step 202 above, the third category of metrics includes one or more of the following: at least one sector balance metric, at least one cache allocation distribution metric, at least one cache eviction distribution metric, at least one cache line access distribution metric, and at least one cache line replacement distribution metric.

[0202] Among them, at least one sector balance indicator includes at least one of the following: sector read balance and sector write balance.

[0203] Sector read balancing is used to characterize the distribution of read requests between two adjacent sectors. A sector is typically 32 bytes, representing a finer granularity within a cache line. Sector read balancing satisfies the following formula:

[0204]

[0205] Here, L2_sector_balance_reads represents sector read balance, L2_req_read_sector0 represents the number of read requests for sector 0, and L2_req_read_sector1 represents the number of read requests for sector 1. An ideal sector read balance value is close to 0.5, indicating even access. A sector read balance value significantly deviating from 0.5 indicates that read accesses are concentrated in a particular sector, which may affect the parallel efficiency of the memory subsystem.

[0206] Sector write balance characterizes the distribution of write requests between two adjacent sectors. A uniform access distribution helps improve write bandwidth and reduce conflicts, while an imbalance may indicate room for optimization in data layout or access patterns. Write balance satisfies the following formula:

[0207]

[0208] Where L2_sector_balance_writes represents sector write balancing, L2_req_write_sector0 represents the number of write requests to sector 0, and L2_req_write_sector1 represents the number of write requests to sector 1.

[0209] In one implementation of step 203, the performance analysis device can determine whether a sector imbalance problem exists in the L2 cache based on at least one sector balance index and the performance evaluation strategy corresponding to the sector balance index. The performance evaluation strategy corresponding to the sector balance index includes:

[0210] If the difference between the sector write balance and 0.5 is greater than the eighth threshold, or the difference between the sector read balance and 0.5 is greater than the eighth threshold, then there is a sector imbalance problem in the L2 cache; the eighth threshold is, for example, 0.2, and this application does not impose any restrictions on it.

[0211] If the difference between sector write balance and 0.5 is less than or equal to the eighth threshold, or the difference between sector read balance and 0.5 is less than or equal to the eighth threshold, then there is no sector imbalance problem in the second-level cache.

[0212] At least one cache allocation distribution metric includes at least one of the following: the total number of set allocations and the allocation ratio of each set.

[0213] The total number of set allocations includes the sum of the number of cache line allocations for all sets (Set 0 to Set 7). For example, if the L2 cache is configured with 8 sets, the total number of set allocations for each set satisfies the following formula:

[0214] Where L2_alloc_set_total is the total number of allocations for the set, sum() represents the summation operation, and L2_allocate_count_set0, L2_allocate_count_set1, ..., L2_allocate_count_set7 are the number of cache line allocations for Set 0 to Set 7, respectively.

[0215] The set allocation ratio is used to measure the distribution of new cache lines among the sets in the L2 cache. Ideally, it should be evenly distributed, that is, the set allocation ratios of each set should be relatively close. If the allocation ratio of a certain set is significantly higher, it indicates that there is a set conflict, and multiple frequently accessed addresses are mapped to the same set, causing frequent swapping in and out of that set, which reduces cache efficiency.

[0216] Taking set 0 as an example, the set allocation ratio L2_alloc_set0_pct of set 0 can be calculated using the following formula:

[0217]

[0218] Where L2_allocate_count_set0 is the number of cache lines allocated to set 0, and L2_alloc_set_total is the total number of sets allocated. The set allocation ratio for any of the other sets Set1 to set7 can be calculated by referring to the set allocation ratio L2_alloc_set0_pct for set 0, which will not be repeated here.

[0219] In one implementation of step 203, the performance analysis device can determine whether there is a set usage imbalance problem in the secondary cache based on cache allocation distribution indicators and the corresponding performance evaluation strategies. The performance evaluation strategies corresponding to the cache allocation distribution indicators include:

[0220] If there is a significant difference between the set allocation ratio of any one or more sets among the multiple sets to be monitored and the set allocation ratio of other sets, such as the minimum difference being greater than the ninth threshold, or the set allocation ratio of any one or more sets among the multiple sets to be monitored being greater than the tenth threshold, then there is a set usage imbalance problem; otherwise, there is no set usage imbalance problem. The ninth threshold is, for example, 0.2, and this application does not impose any restrictions on it.

[0221] For example, the difference between the set allocation ratios of any two sets from set0 to set7 is calculated. For instance, the difference between the set allocation ratio of set0 and set6 is the smallest, i.e., the minimum difference. If the minimum difference is greater than the ninth threshold, it indicates that there is an imbalance in set usage in the second-level cache.

[0222] At least one cached eviction distribution metric includes at least one of the following: total evictions, normal eviction rate, dynamic eviction rate, streaming eviction rate, and persistent eviction rate.

[0223] The total eviction count is the sum of the counts of all types of eviction events. For example, eviction events include normal, dynamic, streaming, and persistent eviction events. The total eviction count can be calculated using the following formula:

[0224] L2_evict_total=L2_normal_evict_count+L2_dyn_evict_count+

[0225] L2_streaming_evict_count + L2_persist_evict_count

[0226] Where L2_normal_evict_count is the number of normal evicts, L2_dyn_evict_count is the number of dynamic evicts, L2_streaming_evict_count is the number of streaming evicts, and L2_persist_evict_count is the number of persistent evicts.

[0227] The normal eviction rate measures the proportion of cache lines evicted using the normal replacement strategy. The normal eviction rate satisfies the following formula:

[0228]

[0229] Where L2_normal_evict_pct is the normal eviction rate, L2_normal_evict_count is the number of normal evictions, and L2_evict_total is the total number of evictions.

[0230] Dynamic eviction ratio measures the proportion of cache lines evicted after their priority is dynamically adjusted based on runtime access patterns. The dynamic eviction ratio satisfies the following formula:

[0231]

[0232] Where L2_dyn_evict_pct is the dynamic eviction rate, L2_dyn_evict_count is the number of dynamic evictions, and L2_evict_total is the total number of evictions.

[0233] The streaming eviction ratio measures the proportion of cache lines marked as having streaming access attributes that are evicted, and it satisfies the following formula:

[0234]

[0235] Where L2_streaming_evict_pct is the streaming evict ratio, L2_streaming_evict_count is the number of streaming evicts, and L2_evict_total is the total number of evicts.

[0236] The persistent eviction ratio measures the proportion of cache lines marked as having persistent access attributes that are evicted, and it satisfies the following formula:

[0237]

[0238] Where L2_persist_evict_pct is the persistent eviction rate, L2_persist_evict_count is the number of persistent evictions, and L2_evict_total is the total number of evictions.

[0239] In one implementation of step 203, the performance analysis device can determine whether a high streaming eviction rate problem exists in the L2 cache based on at least one cache eviction distribution metric and a performance evaluation strategy corresponding to the cache line replacement distribution metric. The performance evaluation strategy corresponding to the cache line replacement distribution metric includes:

[0240] If the streaming eviction rate is greater than the eleventh threshold, then the L2 cache has a high streaming eviction rate problem; otherwise, the L2 cache does not have a high streaming eviction rate problem.

[0241] At least one cache line access distribution metric includes at least one of the following: total number of cache line accesses, and access ratio of each cache line.

[0242] The total number of cache line accesses is the sum of the access counts for all monitored cache lines. Taking cache lines 0 to 15 as an example, the total number of cache line accesses for cache lines 0 to 15 satisfies the following formula:

[0243] L2_cacheline_access_total = sum(L2_req_cacheline_0, L2_req_cacheline_1, ..., L2_req_cacheline_15)

[0244] Where L2_cacheline_access_total is the total number of cache line accesses, and L2_req_cacheline_0 to L2_req_cacheline_15 are the number of accesses for cache lines 0 to 15, respectively.

[0245] The access ratio of each cache line measures the degree to which access concentration (hot spots) is concentrated in a few specific cache lines. Taking cacheline0 as an example, the access ratio of cacheline0 can be calculated using the following formula:

[0246]

[0247] Here, L2_req_cacheline_0 represents the number of accesses to cache line 0, and L2_cacheline_access_total represents the total number of cache line accesses. The cache access ratios for other cache lines 1 through 15 can be referenced from the cache line access ratio for cache line 0.

[0248] In one implementation of step 203, the performance analysis device can determine whether a cache line usage imbalance problem exists based on the cache line access distribution index and the corresponding performance evaluation strategy. The performance evaluation strategy corresponding to the cache line access distribution index includes:

[0249] If the access ratio of one or more cache lines in a set of cache lines differs significantly from the access ratio of cache lines in other sets—for example, if the minimum difference is greater than the twelfth threshold, or if the access ratio of one or more cache lines is greater than the thirteenth threshold—then the second-level cache has a cache line usage imbalance problem. Conversely, if the access ratios of cache lines are not significantly different, the second-level cache does not have a cache line usage imbalance problem. Based on the cache line access ratio, cache lines with high access ratios can be identified as having serious hotspot access issues, which may lead to bandwidth bottlenecks and resource contention, providing a basis for optimizing data layout.

[0250] At least one cache line replacement distribution metric includes at least one of the following: total number of cache line replacements, and replacement ratio of each cache line.

[0251] The total number of cache line replacements is used to count the total number of times a set of cache lines are overwritten (replaced) by new cache lines due to conflicts or other reasons, reflecting the intensity of contention at these locations. The total number of cache line replacements, L2_cacheline_replace_total, can be calculated using the following formula:

[0252] L2_cacheline_replace_total=sum(L2_replace_cacheline_0,L2_replace_cacheline_1, ..., L2_replace_cacheline_15)

[0253] Where L2_replace_cacheline_0 to L2_replace_cacheline_15 are the number of replacements that occurred on cache lines 0 to 15, respectively.

[0254] The replacement ratio for each cache line is the proportion of replacements on each cache line within a set of cache lines to the total number of cache line replacements. Taking cache line 0 as an example, the cache line replacement ratio L2_cacheline0_replace_pct for cache line 0 can be calculated using the following formula:

[0255]

[0256] Where L2_replace_cacheline_0 is the number of replacements on cache line 0, and L2_cacheline_replace_total is the total number of cache line replacements.

[0257] In one implementation of step 203, the performance analysis device can determine whether a cache line usage imbalance problem exists based on the cache line replacement distribution index and the performance evaluation strategy corresponding to the cache line replacement distribution index. The performance evaluation strategy corresponding to the cache line replacement distribution index includes:

[0258] If the replacement ratio of any one or more cache lines in a set of cache lines differs significantly from the replacement ratio of the other set—for example, if the minimum difference is greater than the fourteenth threshold, or if the replacement ratio of any one or more cache lines is greater than the fifteenth threshold—then a cache line usage imbalance problem exists. Conversely, if the replacement ratios are not significantly different, then a cache line usage imbalance problem does not exist. Based on the replacement ratios of each cache line, cache line locations with high replacement ratios can be identified as conflict hotspots, i.e., cache line locations where replacements occur frequently.

[0259] Among the secondary cache performance metrics in step 202 above, the fourth category of metrics includes at least one or more of the following metrics: cache turnover rate, read cache thrashing index, write cache thrashing index, and the correlation between cache thrashing and misses.

[0260] Cache turnover rate measures the average frequency of cache line evictions in the L2 cache per unit of time (each active cycle). The cache turnover rate metric L2_cache_turnover_rate can be calculated as follows:

[0261]

[0262] Where L2_evict_total is the total number of evictions, and L2_cycles_active is the total number of active L2 cache cycles. The higher the value of the cache turnover rate metric, the more frequently data is swapped in and out of the cache, and the more severe the cache thrashing.

[0263] The read thrashing index is a composite metric that comprehensively assesses the severity of cache thrashing caused by read requests. The read thrashing index (L2_read_thrashing_index) can be calculated as follows:

[0264]

[0265] Here, L2_read_outstanding_full_stall_rate is the read FIFO full blocking rate, and L2_read_hit_rate is the read hit rate. The calculation formula for L2_read_thrashing_index considers both resource contention (read FIFO full blocking rate) and low cache efficiency (read miss rate). The higher the exponent value of L2_read_thrashing_index, the more severe the thrashing caused by read operations. A large number of read misses not only cause frequent replacement of cache lines, but also cause new read requests to be blocked due to a full queue, forming a vicious cycle.

[0266] The write thrashing index is a composite metric that comprehensively assesses the severity of cache thrashing caused by write requests. The write thrashing index L2_write_thrashing_index can be calculated as follows:

[0267]

[0268] Where L2_write_outstanding_full_stall_rate is the write FIFO full blocking rate, L2_write_hit_rate is the write hit rate, and (1 - L2_write_hit_rate) is the write miss rate. The formula for calculating the write cache thrashing index reflects both the backlog of write requests (write FIFO full blocking rate) and the caching efficiency of write requests (write miss rate). The higher the exponent value of L2_write_thrashing_index, the more severe the thrashing caused by write requests. A large number of write request misses lead to the continuous allocation and eviction of new cache lines, while the write request queue becomes saturated due to insufficient processing, resulting in a significant decrease in system throughput.

[0269] The cache thrashing-miss correlation is used to quantify the proportion of blocking caused by cache misses / replacements (i.e., thrashing-related blocking) in the total blocking. The cache thrashing-miss correlation metric L2_thrashing_miss_correlation satisfies the following formula:

[0270]

[0271] Among them, L2_read_outstanding_full_stall_rate is the full blocking rate of the read FIFO, L2_write_outstanding_full_stall_rate is the full blocking rate of the write FIFO, and L2_cache_turnover_rate is the cache turnover rate.

[0272] In one implementation of step 203, the performance analysis device can determine whether the L2 cache has a cache thrashing problem based on the fourth type of indicator and the corresponding performance evaluation strategy. The performance evaluation strategy corresponding to the fourth type of indicator includes:

[0273] If the cache turnover rate is greater than the sixteenth threshold, or the read cache turbulence index or write cache turbulence index is greater than the seventeenth threshold, or the cache turbulence-miss correlation index is greater than the eighteenth threshold, then there is a cache turbulence problem; otherwise, there is no cache turbulence problem.

[0274] It should be understood that any of the above methods for implementing step 203 can be used alone or in combination, and this application will not elaborate further on this.

[0275] In some embodiments, the performance analysis device may also combine any of the above-mentioned performance indicators to determine the overall performance score of the secondary cache.

[0276] For example, by using a multi-dimensional blocking comprehensive analysis method, various blocking situations of the L2 cache can be evaluated, specifically including the following steps: (1) FIFO blocking analysis: analyzing the outstanding full blocking situation of read requests, write requests, and copy requests; (2) Interface blocking analysis: analyzing the blocking situation of requests and data channels of HBM interface and NIU interface; (3) Starvation analysis: analyzing the starvation situation of HBM and NIU interfaces and evaluating data transmission wait; (4) Blocking correlation analysis: analyzing the correlation between various types of blocking and identifying the root bottleneck. The relevant descriptions of each step can be found in the relevant content of the aforementioned embodiments, and will not be repeated here. By comprehensively identifying system bottlenecks through this method, it is helpful to provide targeted optimization suggestions.

[0277] For example, the thrashing of the L2 cache can be quantitatively evaluated using multi-dimensional indicators, specifically including the following steps: (1) Calculate the cache turnover rate: L2_cache_turnover_rate, to evaluate the frequency of cache line replacement; (2) Calculate the read cache thrashing index: L2_read_thrashing_index, to evaluate the degree of read operation thrashing; (3) Calculate the write cache thrashing index: L2_write_thrashing_index, to evaluate the degree of write operation thrashing; (4) Calculate the correlation between cache thrashing and misses: L2_thrashing_miss_correlation, to evaluate the correlation between thrashing and misses. Through the comprehensive evaluation method of cache thrashing, cache thrashing problems can be accurately identified, and the cache stability can be evaluated through quantitative indicators, which helps to provide a precise basis for optimization.

[0278] Based on the same technical concept, embodiments of this application provide a performance analysis device for secondary cache, such as... Figure 3 As shown, the performance analysis device 300 includes an acquisition unit 301, a determination unit 302, and an analysis unit 303. Wherein:

[0279] The acquisition unit 301 is used to acquire statistical feature data corresponding to the access requests received by the secondary cache within the sampling period;

[0280] The determining unit 302 is used to determine the performance indicators of the second-level cache based on statistical characteristic data. The performance indicators of the second-level cache include a first type of indicator used to characterize the request type distribution characteristics and data transmission characteristics of the second-level cache, a second type of indicator used to characterize the request hit rate and response efficiency of the second-level cache, a third type of indicator used to characterize the cache utilization rate of the second-level cache, and a fourth type of indicator used to characterize the cache thrashing degree of the second-level cache.

[0281] Analysis unit 303 is used to perform performance analysis on the secondary cache based on the performance metrics of the secondary cache.

[0282] Optionally, the first type of indicators includes one or more of the following: special operation rate indicators; traffic indicators; average burst length indicators; and the analysis unit 303 is specifically used to: evaluate the distribution characteristics of requests of at least one type of operation received by the secondary cache based on the special operation rate indicators, including atomic operation request rate, reduction request rate, and bundle-level reduction request rate; evaluate the traffic characteristics generated by the secondary cache based on at least one of the traffic indicators, including network interface unit read traffic, NIU write traffic, HBM read traffic, HBM write traffic, and NIU copy traffic; and evaluate the data transmission efficiency corresponding to requests initiated through NIU or HBM based on at least one of the average burst length indicators, including NIU read average burst length, NIU write average burst length, NIU copy average burst length, HBM read average burst length, and HBM write average burst length.

[0283] Optionally, the second type of metric includes the cache hit rate metric; the analysis unit 303 is specifically used to: for any type of read request received by the second-level cache, determine whether the second-level cache has a high miss rate problem based on the read hit rate corresponding to the read request of that type and the performance evaluation strategy corresponding to the read hit rate; wherein, the performance evaluation strategy corresponding to the read hit rate includes that if the read hit rate corresponding to the read request of that type is less than a first threshold, then the second-level cache has a high miss rate problem of that type.

[0284] Optionally, the second type of indicator includes at least one blocking analysis indicator; the analysis unit 303 is specifically used to: determine whether there is a blocking problem in the secondary cache based on at least one blocking analysis indicator and the performance evaluation strategy corresponding to at least one blocking analysis indicator; wherein, the performance evaluation strategy corresponding to at least one blocking analysis indicator includes at least one of the following: if any one of the read FIFO full blocking rate, write FIFO full blocking rate, and copy FIFO full blocking rate is greater than a third threshold, then the secondary cache has a high FIFO blocking rate problem; or, if the read FIFO full blocking rate, write FIFO full blocking rate, and copy FIFO full blocking rate are all less than or equal to the third threshold, then the secondary cache does not have a high FIFO blocking rate problem; or, if the HBM interface AR channel blocking rate or HBM interface W channel blocking rate is greater than a fourth threshold, then the secondary cache has a high HBM interface blocking rate problem; or, if the HBM interface AR channel blocking rate or HBM interface W channel blocking rate is less than or equal to the fourth threshold, then the secondary cache does not have a high HBM interface blocking rate problem; or, if the NIU copy AR channel blocking rate or NIU copy R channel blocking rate is greater than a fifth threshold, then the secondary cache has a high NIU interface blocking rate problem; if the NIU... If the AR channel blocking rate or the NIU R channel blocking rate is less than or equal to the fifth threshold, then the L2 cache does not have a high NIU interface blocking rate problem.

[0285] Optionally, the second type of indicator includes at least one hunger analysis indicator; the analysis unit 303 is specifically used to: determine whether there is a hunger problem in the secondary cache based on at least one hunger analysis indicator and the performance evaluation strategy corresponding to at least one hunger analysis indicator; wherein, the performance evaluation strategy corresponding to at least one hunger analysis indicator includes at least one of the following: if the HBM R channel hunger rate is greater than the sixth threshold, then the secondary cache has a high HBM hunger rate problem; or, if the NIU replication R channel hunger rate is greater than the seventh threshold, then the secondary cache has a high NIU hunger rate problem; or, if the HBM R channel hunger rate is less than or equal to the sixth threshold or the NIU replication R channel hunger rate is less than or equal to the seventh threshold, then the secondary cache does not have a hunger problem.

[0286] Optionally, the third type of indicator includes at least one sector balance indicator; the analysis unit 303 is specifically used to: determine whether there is a sector imbalance problem in the secondary cache based on at least one sector balance indicator and the performance evaluation strategy corresponding to the sector balance indicator; wherein, the performance evaluation strategy corresponding to the sector balance indicator includes: if the difference between the sector write balance and 0.5 is greater than the eighth threshold, or the difference between the sector read balance and 0.5 is greater than the eighth threshold, then there is a sector imbalance problem in the secondary cache; or, if the difference between the sector write balance and 0.5 is less than the eighth threshold, or the difference between the sector read balance and 0.5 is less than or equal to the eighth threshold, then there is no sector imbalance problem in the secondary cache.

[0287] Optionally, the third type of indicator includes at least one cache allocation distribution indicator; the analysis unit 303 is specifically used to: determine whether there is a set usage imbalance problem in the secondary cache based on the cache allocation distribution indicator and the performance evaluation strategy corresponding to the cache allocation distribution indicator; wherein, the performance evaluation strategy corresponding to the cache allocation distribution indicator includes: if the minimum difference between the set allocation ratio of any one or more sets among the monitored sets and the allocation ratio of other sets is greater than a ninth threshold, or if the set allocation ratio of any one or more sets among the monitored sets is greater than a tenth threshold, then there is a set usage imbalance problem. Alternatively, if the minimum difference between the set allocation ratio of any one or more sets among the monitored sets and the allocation ratio of other sets is less than or equal to the ninth threshold, or if the set allocation ratio of any one or more sets among the monitored sets is less than or equal to the tenth threshold, then there is no set usage imbalance problem in the secondary cache.

[0288] Optionally, the third type of indicator includes at least one cache eviction distribution indicator; the analysis unit 303 is specifically used to: determine whether there is a high streaming eviction rate problem based on at least one cache eviction distribution indicator and the performance evaluation strategy corresponding to the cache eviction distribution indicator; wherein, the performance evaluation strategy corresponding to the cache eviction distribution indicator includes: if the streaming eviction ratio is greater than the eleventh threshold, then the secondary cache has a high streaming eviction rate problem; or, if the streaming eviction ratio is less than or equal to the eleventh threshold, then the secondary cache does not have a high streaming eviction rate problem.

[0289] Optionally, the third type of indicators includes cache line access distribution indicators and cache line replacement distribution indicators; the analysis unit 303 is specifically used to: determine whether there is a cache line usage imbalance problem in the second-level cache based on the cache line access distribution indicators and the performance evaluation strategy corresponding to the cache line access distribution indicators, or based on the cache line replacement distribution indicators and the performance evaluation strategy corresponding to the cache line replacement indicators.

[0290] The performance evaluation strategy corresponding to the cache line access distribution index includes: if the minimum difference between the cache line access ratio of any one or more cache lines in a set of cache lines and the cache line access ratio of other sets is greater than the twelfth threshold, or if the cache line access ratio of any one or more cache lines in a set of cache lines is greater than the thirteenth threshold, then the second-level cache has a cache line usage imbalance problem; or if the minimum difference between the cache line access ratio of any one or more cache lines in a set of cache lines and the cache line access ratio of other sets is less than or equal to the twelfth threshold, or if the cache line access ratio of any one or more cache lines in a set of cache lines is less than or equal to the thirteenth threshold, then the second-level cache does not have a cache line usage imbalance problem.

[0291] The performance evaluation strategy corresponding to the cache line replacement distribution index includes: if the minimum difference between the cache line replacement ratio of any one or more cache lines in a set of cache lines and the cache line replacement ratio of other sets is greater than the fourteenth threshold, or if the cache line replacement ratio of any one or more cache lines is greater than the fifteenth threshold, then the second-level cache has a cache line usage imbalance problem; or if the minimum difference between the cache line replacement ratio of any one or more cache lines in a set of cache lines and the cache line replacement ratio of other sets is less than or equal to the fourteenth threshold, or if the cache line replacement ratio of any one or more cache lines is less than or equal to the fifteenth threshold, then the second-level cache does not have a cache line usage imbalance problem.

[0292] Optionally, the fourth category of indicators includes at least one of the following: cache turnover rate, read cache thrashing index, write cache thrashing index, and correlation between cache thrashing and misses; the analysis unit 303 is specifically used to: determine whether a cache thrashing problem exists based on the fourth category of indicators and the performance evaluation strategy corresponding to the fourth category of indicators; wherein, the performance evaluation strategy corresponding to the fourth category of indicators includes: if the cache turnover rate is greater than the sixteenth threshold, or the read cache thrashing index or the write cache thrashing index is greater than the seventeenth threshold, or the correlation between cache thrashing and misses is greater than the eighteenth threshold, then the second-level cache has a cache thrashing problem; if the cache turnover rate is less than or equal to the sixteenth threshold, or the read cache thrashing index or the write cache thrashing index is less than or equal to the seventeenth threshold, or the correlation between cache thrashing and misses is less than or equal to the eighteenth threshold, then the second-level cache does not have a cache thrashing problem.

[0293] It should be noted that the division of units in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The functional units in the embodiments of this application can be integrated into one verification unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0294] The above methods can be implemented wholly or partially through software, hardware, firmware, or any other combination thereof. When implemented using software, the above methods can be implemented wholly or partially in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive (SSD).

[0295] In a simplified embodiment, those skilled in the art will recognize that the performance analysis device used for the secondary cache in the embodiment can be an electronic device, which may employ... Figure 4 As shown in the figure.

[0296] like Figure 4 The electronic device 400 shown includes at least one processor 401 and a memory 402, and optionally, may also include a communication interface 403.

[0297] Memory 402 may be volatile memory, such as random access memory; memory may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 402 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 402 may be a combination of the above-described memories.

[0298] The specific connection medium between the processor 401 and the memory 402 described above is not limited in the embodiments of this application.

[0299] Processor 401 can be a GPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, artificial intelligence chips, on-chip devices, etc. General-purpose processors can be microprocessors or any conventional processor. (The last sentence appears to be incomplete and possibly refers to a different context.) Figure 4 In electronic devices, an independent data transceiver module, such as communication interface 403, can also be set up for sending and receiving data; when the processor 401 communicates with other devices, it can transmit data through communication interface 403.

[0300] In one possible application scenario, electronic devices adopt... Figure 4 The form shown, Figure 4 The processor 401 can call computer execution instructions stored in the memory 402, enabling the electronic device to execute the methods in any of the above method embodiments.

[0301] Based on the same technical concept, embodiments of this application provide a computer-readable storage medium including computer-executable instructions, which are used to cause a computer to perform the methods in any of the above method embodiments.

[0302] Based on the same technical concept, this application provides a computer program product that stores a computer program, which includes program instructions. When the program instructions are executed by a computer, the computer performs the method in any of the above method embodiments.

[0303] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), systems, chips, computer-readable storage media, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects, all of which are collectively referred to herein as "modules" or "systems".

[0304] This application is described with reference to at least one of the following diagrams of the method, apparatus (device) or system of this application: flowchart, block diagram. It should be understood that at least one of the following can be implemented by computer program instructions: each step in the flowchart, each block in the block diagram, and a combination of steps in the flowchart and blocks in the block diagram.

[0305] These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing at least one of the following: in the process Figure 1 Devices or blocks that specify functions in one or more processes Figure 1 A device that provides the functions specified in one or more boxes.

[0306] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means that performs at least one of the following: in the process Figure 1 The specified function or box in one or more processes Figure 1 The function specified in one or more boxes.

[0307] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing at least one of the following: steps for a function specified in one or more processes in a flowchart, or steps for a function specified in one or more blocks in a block diagram.

[0308] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A method for analyzing the performance of a secondary cache, characterized in that, include: Obtain statistical feature data corresponding to access requests received by the secondary cache within the sampling period. The secondary cache is a shared cache of multiple computing units within an artificial intelligence chip, and the artificial intelligence chip includes a GPU or a GPGPU. Based on the statistical feature data, the performance indicators of the second-level cache are determined. These performance indicators include: a first type of indicator characterizing the request type distribution characteristics and data transmission characteristics of the second-level cache; a second type of indicator characterizing the request hit rate and response efficiency of the second-level cache; a third type of indicator characterizing the cache utilization rate of the second-level cache; and a fourth type of indicator characterizing the cache thrashing degree of the second-level cache. The first type of indicator includes a traffic indicator, representing the total number of bytes read from or written to the HBM by the second-level cache, or the total number of bytes read from or written to the second-level cache by the computing unit within the AI ​​chip. The second type of indicator includes at least one of the following: at least one cache hit rate indicator and at least one blocking analysis indicator. The at least one cache hit rate indicator represents the percentage of hits of different types of memory access requests within the corresponding type of memory access requests received by the second-level cache. The first category of indicators includes at least one of the following: a blockage analysis metric representing the proportion of the number of periods when the FIFO is full for different types of memory access requests within the L2 cache, or the proportion of the number of periods when the channel between the L2 cache and HBM is blocked, relative to the total number of active periods in the L2 cache; the third category of indicators includes at least one of the following: at least one sector balance metric and at least one cache line access distribution metric; the at least one sector balance metric represents the proportion of the number of one type of memory access requests received by a sector in the L2 cache relative to the number of the same type of memory access requests received by two adjacent sectors; the at least one cache line access distribution metric represents the proportion of the number of accesses to each monitored cache line relative to the total number of accesses to each cache line; the fourth category of indicators includes at least one of the following: a read cache thrashing index and a write cache thrashing index; the read cache thrashing index represents the product of the read FIFO full blockage rate and the read miss rate; the write cache thrashing index represents the product of the write FIFO full blockage rate and the write miss rate; Based on the performance metrics of the L2 cache, a performance analysis of the L2 cache is performed.

2. The method as described in claim 1, characterized in that, The first category of indicators also includes one or more of the following: special operation rate; average burst length; The performance analysis of the secondary cache based on its performance metrics includes: Based on at least one of the special operation rate indicators, namely atomic operation request rate, reduction request rate, and in-bundle reduction request rate, evaluate the distribution characteristics of at least one type of operation requests received by the second-level cache. The traffic characteristics generated by the secondary cache are evaluated based on at least one of the following traffic metrics: network interface unit read traffic, NIU write traffic, HBM read traffic, HBM write traffic, and NIU copy traffic. Based on at least one of the average burst length metrics, including NIU read average burst length, NIU write average burst length, NIU copy average burst length, HBM read average burst length, and HBM write average burst length, the data transmission efficiency corresponding to requests initiated through NIU or HBM is evaluated.

3. The method as described in claim 1, characterized in that, The performance analysis of the secondary cache based on its performance metrics includes: For any type of read request received by the second-level cache, determine whether the second-level cache has a high miss rate problem based on the read hit rate corresponding to the read request of that type and the performance evaluation strategy corresponding to the read hit rate; wherein, the performance evaluation strategy corresponding to the read hit rate includes that if the read hit rate corresponding to the read request of that type is less than a first threshold, then the second-level cache has a high miss rate problem of that type.

4. The method as described in claim 1, characterized in that, The performance analysis of the secondary cache based on its performance metrics includes: Based on at least one blocking analysis metric and the corresponding performance evaluation strategy, determine whether the secondary cache has a blocking problem; wherein the performance evaluation strategy corresponding to at least one blocking analysis metric includes at least one of the following: If any one of the read FIFO full blocking rate, write FIFO full blocking rate, or copy FIFO full blocking rate is greater than the third threshold, then the L2 cache has a high FIFO blocking rate problem; or, if the read FIFO full blocking rate, write FIFO full blocking rate, and copy FIFO full blocking rate are all less than or equal to the third threshold, then the L2 cache does not have a high FIFO blocking rate problem; or... If the HBM interface AR channel blocking rate or the HBM interface W channel blocking rate is greater than the fourth threshold, then the L2 cache has a high HBM interface blocking rate problem; or, if the HBM interface AR channel blocking rate or the HBM interface W channel blocking rate is less than or equal to the fourth threshold, then the L2 cache does not have a high HBM interface blocking rate problem; or... If the NIU copy AR channel blocking rate or the NIU copy R channel blocking rate is greater than the fifth threshold, then the secondary cache has a high NIU interface blocking rate problem; or, if the NIU copy AR channel blocking rate or the NIU copy R channel blocking rate is less than or equal to the fifth threshold, then the secondary cache does not have a high NIU interface blocking rate problem.

5. The method as described in claim 1, characterized in that, The second category of indicators also includes at least one hunger analysis indicator; The performance analysis of the secondary cache based on its performance metrics includes: Based on at least one hunger analysis metric and the corresponding performance evaluation strategy, determine whether the secondary cache has a hunger problem; wherein the performance evaluation strategy corresponding to at least one hunger analysis metric includes at least one of the following: If the HBM R channel starvation rate is greater than the sixth threshold, then the secondary cache has a high HBM starvation rate problem; or, If the NIU replication R channel starvation rate is greater than the seventh threshold, then the L2 cache has a high NIU starvation rate problem; or, If the HBM R channel starvation rate is less than or equal to the sixth threshold or the NIU replication R channel starvation rate is less than or equal to the seventh threshold, then the secondary cache does not have a starvation problem.

6. The method according to any one of claims 1-5, characterized in that, The performance analysis of the secondary cache based on its performance metrics includes: Based on at least one sector balance metric and the corresponding performance evaluation strategy, determine whether the L2 cache has a sector imbalance problem; wherein, the performance evaluation strategy corresponding to the sector balance metric includes: If the difference between the sector write balance and 0.5 is greater than the eighth threshold, or the difference between the sector read balance and 0.5 is greater than the eighth threshold, then the L2 cache has a sector imbalance problem; or, If the difference between sector write balance and 0.5 is less than or equal to the eighth threshold, or the difference between sector read balance and 0.5 is less than or equal to the eighth threshold, then there is no sector imbalance problem.

7. The method according to any one of claims 1-5, characterized in that, The third category of indicators also includes at least one cache allocation distribution indicator; The performance analysis of the secondary cache based on its performance metrics includes: Based on the cache allocation distribution metrics and the corresponding performance evaluation strategies, determine whether the secondary cache has a set usage imbalance problem; wherein, the performance evaluation strategies corresponding to the cache allocation distribution metrics include: If the minimum difference between the allocation ratio of any one or more sets among the monitored sets and the allocation ratio of other sets is greater than the ninth threshold, or if the allocation ratio of any one or more sets among the monitored sets is greater than the tenth threshold, then the second-level cache has a set usage imbalance problem; or... If the minimum difference between the set allocation ratio of any one or more sets among the monitored sets and the allocation ratio of other sets is less than or equal to the ninth threshold, or if the set allocation ratio of any one or more sets among the monitored sets is less than or equal to the tenth threshold, then the second-level cache does not have a set usage imbalance problem.

8. The method according to any one of claims 1-5, characterized in that, The third category of indicators also includes at least one cache eviction distribution indicator; The performance analysis of the secondary cache based on its performance metrics includes: Based on at least one cache eviction distribution metric and the corresponding performance evaluation strategy, determine whether a high streaming eviction rate problem exists; wherein, the performance evaluation strategy corresponding to the cache eviction distribution metric includes: If the streaming eviction rate is greater than the eleventh threshold, then the secondary cache has a high streaming eviction rate problem; or, if the streaming eviction rate is less than or equal to the eleventh threshold, then the secondary cache does not have a high streaming eviction rate problem.

9. The method according to any one of claims 1-5, characterized in that, The third category of indicators also includes cache line replacement distribution indicators; The performance analysis of the secondary cache based on its performance metrics includes: Based on the cache line access distribution index and the corresponding performance evaluation strategy, and / or based on the cache line replacement distribution index and the corresponding performance evaluation strategy, determine whether the second-level cache has a cache line usage imbalance problem. The performance evaluation strategies corresponding to the cache line access distribution metric include: If the minimum difference between the cache line access ratio of any one or more cache lines in a set of cache lines and the cache line access ratio of the other set is greater than the twelfth threshold, or if the cache line access ratio of any one or more cache lines in a set of cache lines is greater than the thirteenth threshold, then the second-level cache has a cache line usage imbalance problem; or... If the minimum difference between the cache line access ratio of any one or more cache lines in a set of cache lines and the cache line access ratio of other sets is less than or equal to the twelfth threshold, or if the cache line access ratio of any one or more cache lines in a set of cache lines is less than or equal to the thirteenth threshold, then the second-level cache does not have a cache line usage imbalance problem. Performance evaluation strategies for cache line replacement distribution metrics include: If the minimum difference between the cache line replacement ratio of any one or more cache lines in a set and the cache line replacement ratio of the other set is greater than the fourteenth threshold, or if the cache line replacement ratio of any one or more cache lines is greater than the fifteenth threshold, then the second-level cache has a cache line usage imbalance problem, or... If the minimum difference between the cache line replacement ratio of any one or more cache lines in a set of cache lines and the cache line replacement ratio of other sets is less than or equal to the fourteenth threshold, or if the cache line replacement ratio of any one or more cache lines is less than or equal to the fifteenth threshold, then the second-level cache does not have a cache line usage imbalance problem.

10. The method according to any one of claims 1-5, characterized in that, The fourth category of metrics also includes at least one of the following: cache turnover rate, and the correlation between cache thrashing and misses; The performance analysis of the secondary cache based on its performance metrics includes: Based on the fourth type of metric and the corresponding performance evaluation strategy, determine whether the secondary cache has a cache thrashing problem; wherein, the performance evaluation strategy corresponding to the fourth type of metric includes: If the cache turnover rate is greater than the sixteenth threshold, or the read cache turbulence index or write cache turbulence index is greater than the seventeenth threshold, or the correlation between cache turbulence and misses is greater than the eighteenth threshold, then the second-level cache has a cache turbulence problem. If the cache turnover rate is less than or equal to the sixteenth threshold, or the read cache turbulence index or write cache turbulence index is less than or equal to the seventeenth threshold, or the correlation between cache turbulence and misses is less than or equal to the eighteenth threshold, then the secondary cache does not have a cache turbulence problem.

11. A computer device, comprising a memory, a processor chip, and a computer program stored in the memory and executable on the processor chip, characterized in that, When the processor chip executes the computer program, it implements the steps of the method according to any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, It includes computer-executable instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 10.

Images