Cache line replacement method and device, chip and electronic equipment

Abstract

The invention relates to the technical field of artificial intelligence chips and storage management, in particular to a cache line replacement method and device, a chip and electronic equipment. The method is applied to a chip comprising a data cache, the data cache comprises N cache lines in which data are stored, an access path corresponding to each cache line is managed based on a binary tree structure comprising N-1 nodes in a PLRU algorithm, and a data replacement instruction is responded, obtaining a path status bit from a root node to a leaf node on an access path corresponding to each cache line; for a specified cache line, calculating a PLRU path value according to the path status bit, and calculating a replacement score in combination with the importance level and the corresponding weight of the stored data; determining a replaceable cache line according to the replacement score; and storing the to-be-cached data and the importance level thereof into a replaceable cache line to replace the original data and the original level. Therefore, the cache line replacement priority is evaluated by combining the PLRU path value and the importance level of the stored data, the cache hit rate is effectively improved, and the system performance is optimized.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence chips and storage management, in particular to a cache line replacement method and device, a chip and an electronic equipment. BACKGROUND

[0002] At present, with the rapid development of artificial intelligence technology, the performance of neural network processor (NPU) as the core hardware for accelerating deep learning tasks plays a crucial role in the running efficiency of various intelligent applications. Deep learning tasks have the characteristics of high parallelism, strong data reuse and complex access mode, which requires NPU to efficiently manage and utilize data when processing these tasks to fully exert its computing power.

[0003] In the architecture of NPU, data cache (Cache) is a key component to improve computing efficiency. Cache sets a smaller capacity but faster access speed storage level between the processor and the main memory, caches the data frequently accessed by the processor, thereby reducing the number of times the processor accesses the main memory, reducing access delay and improving overall computing performance.

[0004] However, the traditional Cache design usually adopts simple replacement algorithms such as the least recently used algorithm (LRU), the first-in-first-out algorithm (FIFO) and the like. These algorithms may have certain applicability in general computing scenarios, but they have many problems in the specific workloads faced by NPU.

[0005] For the LRU algorithm, its core idea is to replace the data block with the least recent usage. When NPU processes deep learning tasks, due to the complex access mode and strong data reuse, only relying on the recent usage to determine the replacement priority of the data block often cannot accurately predict the subsequent data access demand. In deep learning algorithms, the access of data may exhibit periodicity or locality characteristics, but this characteristic cannot be fully covered by simple time sequence. For example, in some neural network layers, certain weight data may be repeatedly used in multiple computing cycles, but these data may not be frequently accessed in the recent period of time. According to the LRU algorithm, these important data may be incorrectly replaced out of Cache, resulting in the need to read from the main memory for subsequent access, increasing the access delay and reducing the cache hit rate.

[0006] The FIFO algorithm replaces the data blocks according to the order in which the data blocks enter the cache, and the data blocks that enter the cache first are replaced first. This algorithm does not consider the frequency of use and importance of the data blocks at all, and is too mechanical and blind under the high parallelism and complex access mode of the NPU. The data access in the deep learning task has high dynamicity and uncertainty, and only the time sequence of the data entering the cache is used to determine the replacement, which cannot adapt to the changes, so that a large amount of data that is no longer frequently used may be retained in the cache, and the data that is really needed is replaced out, further reducing the utilization and hit rate of the cache.

[0007] In addition, with the continuous expansion of artificial intelligence applications to embedded devices and edge computing scenarios, lightweight NPUs have emerged. Such NPUs have strict limitations on power consumption and hardware resources, and need to implement high-performance computing under limited resources. However, the existing simple cache replacement algorithms not only cannot fully utilize the cache resources, resulting in low cache hit rate and high access delay, but also consume certain hardware resources and power consumption in implementing these algorithms. For example, the LRU algorithm needs to maintain a complex access history record structure to track the use of each data block, and the least frequently used algorithm (LFU) also needs to count and record the use frequency of the data blocks. This increases the additional overhead in hardware implementation, including storage space and computing resources. For resource-constrained lightweight NPUs, such additional overhead is difficult to bear, which seriously affects their performance and energy efficiency.

[0008] In summary, the existing cache replacement algorithms have many shortcomings under the workload of the NPU, and cannot meet the needs of the NPU for high performance and low overhead.

[0009] How to improve the cache hit rate in the lightweight NPU application scenario and improve the overall performance of the system is a problem to be solved. SUMMARY

[0010] To solve the problems in the related art, the embodiments of the present disclosure provide a cache line replacement method and device, a chip and an electronic equipment.

[0011] In a first aspect, a cache line replacement method is provided in the embodiments of the present disclosure. The method is applied to a chip, and the chip includes a data cache including N cache lines of stored data. A binary tree structure including N-1 nodes in a pseudo least recently used (PLRU) algorithm is used to manage an access path corresponding to each cache line, wherein each node corresponds to a path state bit, and the path state bit is used to dynamically record an access direction preference. The method includes the following steps. In response to the received data replacement instruction, path state bits from the root node to the leaf node on the access path corresponding to each cache line are obtained; For a specified cache line, a PLRU path value corresponding to the specified cache line is calculated according to the path state bits from the root node to the leaf node on the access path corresponding to the specified cache line; an importance level of the stored data in the specified cache line is obtained; and a replacement score of the specified cache line is calculated according to the PLRU path value corresponding to the specified cache line and the importance level of the stored data in the specified cache line, and a specified PLRU path value weight and a specified importance level weight. A replaceable cache line in the data cache is determined according to the replacement score of each cache line. The data to be cached and an importance level corresponding to the data to be cached are obtained, and the data to be cached and the importance level corresponding to the data to be cached are stored in the replaceable cache line to replace the stored data and the importance level of the stored data in the replaceable cache line.

[0012] According to an embodiment of the present disclosure, the binary tree structure containing N-1 nodes in the pseudo least recently used (PLRU) algorithm is used to manage the access path corresponding to each cache line, including: After the data to be cached and the importance level corresponding to the data to be cached are stored in the replaceable cache line, the path state bits of the nodes in the binary tree structure are updated based on the PLRU algorithm.

[0013] According to an embodiment of the present disclosure, the cache line includes a data area and a Tag area; and the data to be cached and the importance level corresponding to the data to be cached are stored in the replaceable cache line, including: The data to be cached is stored in the data area of the replaceable cache line, and the importance level corresponding to the data to be cached is stored in the Tag area of the replaceable cache line. The importance level of the stored data in the specified cache line is obtained from the Tag area of the specified cache line.

[0014] According to an embodiment of the present disclosure, the PLRU path value corresponding to the specified cache line is calculated according to the path state bits from the root node to the leaf node on the access path corresponding to the specified cache line, including: The path state bits from the root node to the leaf node on the access path corresponding to the specified cache line are added to obtain the PLRU path value corresponding to the specified cache line.

[0015] According to an embodiment of the present disclosure, the determining the replaceable cache line in the data cache according to the replacement score of each cache line comprises: obtaining the lowest replacement score in the replacement scores of each cache line; if the number of the lowest replacement scores is one, taking the cache line corresponding to the lowest replacement score as the replaceable cache line; if the number of the lowest replacement scores is more than one, obtaining the replaceable cache line in the data cache based on the PLRU algorithm.

[0016] According to an embodiment of the present disclosure, when obtaining the lowest replacement score in the replacement scores of each cache line, a multi-stage comparison tree structure is implemented, wherein each comparator in the multi-stage comparison tree structure is a binary comparator, and the binary comparator is configured to receive the replacement scores of any two cache lines as input and output the smaller value and the cache line index corresponding to the smaller value. The implementation based on the multi-stage comparison tree structure comprises: a first-stage comparator compares all cache lines in pairs to obtain a first-stage candidate minimum value set; each subsequent-stage comparator continues to compare the candidate minimum values output by the previous stage in groups until the cache line index corresponding to the final lowest replacement score is output.

[0017] According to an embodiment of the present disclosure, the method further comprises: obtaining at least one parameter in the historical hit rate, the future predicted hit rate and the future predicted access pattern of the data cache; dynamically adjusting the specified PLRU path value weight and the specified importance level weight according to at least one parameter in the historical hit rate, the future predicted hit rate and the future predicted access pattern of the data cache.

[0018] According to an embodiment of the present disclosure, the replacement score RS of the specified cache line is calculated by the following formula: ; wherein, represents the PLRU path value corresponding to the specified cache line, represents the importance level of the data stored in the specified cache line, represents the specified PLRU path value weight, represents the specified importance level weight, and are integers not less than 1.

[0019] According to an embodiment of the present disclosure, when the replacement score of the specified cache line is calculated by the formula, the multiplication operation in the formula is implemented by binary decomposition and weighted accumulation based on a shift-add circuit, and then the product results after the multiplication operation are added by an adder to obtain the replacement score of the specified cache line.

[0020] According to an embodiment of the present disclosure, the shift add circuit comprises a shift register and an accumulator; the multiplication operation in the formula is implemented by binary decomposition and weighted accumulation based on the shift add circuit, comprising: determining whether the importance level of the stored data in the specified cache line is a preset low importance level; If not, a full-precision calculation mode is started, comprising: decomposing the multiplier by binary bits, detecting all bits from right to left in bit order, if the i-th bit is 1, then the multiplicand is left-shifted by i bits through the shift register to obtain the effective shift result corresponding to the i-th bit; if it is 0, then skip, until all bit orders are detected; the effective shift results of all bits of the multiplicand are added through the accumulator to obtain the final product result; If yes, an approximate calculation optimization mode is started, comprising: decomposing the multiplier by binary bits, selecting the corresponding effective bits in the multiplier according to a preset truncation bit width configuration parameter, detecting the effective bits from right to left in bit order, if the i-th bit is 1, then the multiplicand is left-shifted by i bits through the shift register to obtain the effective shift result corresponding to the i-th bit; if it is 0, then skip, until all bit orders in the effective bits are detected; the effective shift results of all bits of the multiplicand and the truncation error compensation value are added through the accumulator to obtain the final product result; wherein the truncation error compensation value is obtained based on the average truncation error compensation and dynamic weight calculated offline.

[0021] In a second aspect, the present disclosure provides a cache line replacement device, which is applied to a chip, the chip comprising a data cache including N cache lines of stored data, the device comprising a PLRU algorithm implementation module, a PLRU path value calculation module, a replacement score calculation module, a replaceable cache line determination module and a replacement operation module, wherein: The PLRU algorithm implementation module is configured to manage the access path corresponding to each cache line based on the binary tree structure containing N-1 nodes in the pseudo least recently used PLRU algorithm, wherein each node corresponds to a path state bit, and the path state bit is used to dynamically record the access direction preference, The PLRU path value calculation module is configured to obtain the path state bits from the root node to the leaf node on the access path corresponding to each cache line in response to the received data replacement instruction; and calculate the PLRU path value corresponding to the specified cache line according to the path state bits from the root node to the leaf node on the access path corresponding to the specified cache line. ​The replacement score calculation module is configured to: obtain an importance level of the stored data in the specified cache line; and calculate a replacement score of the specified cache line according to the PLRU path value corresponding to the specified cache line, the importance level of the stored data in the specified cache line, a specified PLRU path value weight, and a specified importance level weight. The replaceable cache line determination module is configured to: determine a replaceable cache line in the data cache according to the replacement score of each cache line. The replacement operation module is configured to: obtain to-be-cached data and an importance level corresponding to the to-be-cached data; and store the to-be-cached data and the importance level corresponding to the to-be-cached data in the replaceable cache line to replace stored data in the replaceable cache line and an importance level of the stored data.

[0022] According to an embodiment of the present disclosure, the replaceable cache line in the data cache is determined according to the replacement score of each cache line, including: obtaining a lowest replacement score in the replacement score of each cache line; if the number of the lowest replacement scores is one, taking the cache line corresponding to the lowest replacement score as the replaceable cache line; if the number of the lowest replacement scores is multiple, finding the replaceable cache line in the data cache based on a PLRU algorithm.

[0023] According to an embodiment of the present disclosure, when the lowest replacement score in the replacement score of each cache line is obtained, a multi-stage comparison tree structure is implemented, wherein each comparator in the multi-stage comparison tree structure is a binary comparator, and the binary comparator is configured to receive the replacement scores of any two cache lines as input and output a smaller value and a cache line index corresponding to the smaller value. The implementation based on the multi-stage comparison tree structure includes: a first-stage comparator compares all cache lines in pairs to obtain a first-stage candidate minimum value set; and each subsequent-stage comparator continues to compare the candidate minimum values output by the previous stage in groups until the cache line index corresponding to the final lowest replacement score is output.

[0024] According to an embodiment of the present disclosure, the apparatus further includes: The weight dynamic adjustment module is configured to: obtain at least one parameter in a historical hit rate, a future predicted hit rate, and a future predicted access mode of the data cache; and dynamically adjust the specified PLRU path value weight and the specified importance level weight according to the at least one parameter in the historical hit rate, the future predicted hit rate, and the future predicted access mode of the data cache.

[0025] In a third aspect, the chip provided in the embodiments of the present disclosure includes the device of any one of the second aspect; or includes a memory and a processor; the memory is configured to store one or more computer instructions; the one or more computer instructions are executed by the processor to implement the method of any one of the first aspect.

[0026] In a fourth aspect, the electronic device provided in the embodiments of the present disclosure includes a memory and a processor; the memory is configured to store one or more computer instructions; the one or more computer instructions are executed by the processor to implement the method of any one of the first aspect.

[0027] According to the technical scheme provided in the embodiments of the present disclosure, a high-performance cache line replacement method based on multi-factor evaluation is provided. The method is applied to a chip, the chip includes a data cache, the data cache includes N cache lines of stored data, a binary tree structure including N-1 nodes in a PLRU algorithm is used to manage an access path corresponding to each cache line, wherein each node corresponds to a path state bit, the path state bit is used to dynamically record an access direction preference, in response to a received data replacement instruction, path state bits from a root node to a leaf node on an access path corresponding to each cache line are obtained; for a specified cache line, a PLRU path value corresponding to the specified cache line is calculated according to the path state bits from the root node to the leaf node on the access path corresponding to the specified cache line; an importance level of the stored data in the specified cache line is obtained; a replacement score of the specified cache line is calculated according to the PLRU path value corresponding to the specified cache line and the importance level of the stored data in the specified cache line, and a specified PLRU path value weight and a specified importance level weight; a replaceable cache line in the data cache is determined according to the replacement score of each cache line; the to-be-cached data and the importance level corresponding to the to-be-cached data are stored in the replaceable cache line to replace the stored data in the replaceable cache line and the importance level of the stored data. Thus, the cache line replacement priority is evaluated by combining the PLRU path value and the importance level of the stored data, which effectively improves the cache hit rate and optimizes the system performance.

[0028] It should be understood that the general description above and the detailed description below are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0029] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of the non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings: Figure 1 A flowchart of a cache line replacement method according to an embodiment of the present disclosure is shown; Figure 2 A schematic diagram showing the correspondence between the binary tree structure containing N-1 nodes and N cache lines in the PLRU algorithm; Figure 3 A schematic diagram showing the determined path of the replaceable cache line when all node initial path state bits are 0 in the PLRU algorithm; Figure 4 A schematic diagram showing the change of each node state bit after the first cache line is replaced and the determined path of the next replaceable cache line in the PLRU algorithm; Figure 5 A flowchart showing a method for determining the replaceable cache line according to the replacement score of each cache line; Figure 6 A schematic diagram showing the determined path of the replaceable cache line based on the replacement score according to an embodiment of the present disclosure; Figure 7 A schematic diagram showing the determined path of the replaceable cache line based on the replacement score according to an embodiment of the present disclosure; Figure 6 A schematic diagram showing the change of each node state bit after the sixth cache line is replaced as the replaceable cache line according to the node state bit condition shown in the table; Figure 8 A flowchart showing another cache line replacement method according to an embodiment of the present disclosure; Figure 9 A block diagram showing the structure of a cache line replacement device according to an embodiment of the present disclosure; Figure 10 A specific structure example of a cache line replacement device according to an embodiment of the present disclosure is shown; Figure 11 A block diagram showing another cache line replacement device according to an embodiment of the present disclosure is shown; Figure 12 A block diagram showing the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0030] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so as to make those skilled in the art readily implement them. Also, portions irrelevant to the description of the exemplary embodiments are omitted in the accompanying drawings for the sake of clarity.

[0031] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate that there are features, numbers, steps, actions, components, parts or combinations thereof disclosed in the specification, and do not exclude the possibility that one or more other features, numbers, steps, actions, components, parts or combinations thereof exist or are added.

[0032] In addition, it should be further noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0033] As described above, the traditional replacement algorithm (such as FIFO) cannot adapt to the complex data access mode of NPU, resulting in low cache hit rate; in addition, complex cache replacement algorithms such as LRU algorithm and LFU (Least Frequently Used) algorithm require more hardware resources, which is not suitable for lightweight NPU, and at present, the Cache in NPU generally only supports one replacement algorithm, such as PLRU (Pseudo Least Recently Used) algorithm, although this algorithm has lower hardware overhead than LRU algorithm, but it has poor dynamic adaptability and limited accuracy.

[0034] In order to solve the above problems in the prior art, the present inventors have conducted in-depth research on the current Cache replacement algorithm, and innovatively proposed a high-performance cache line replacement method based on multi-factor evaluation. This method introduces a multi-factor dynamic evaluation mechanism based on the traditional PLRU algorithm, calculates the replacement score of each cache line by combining the PLRU path value corresponding to the cache line and the importance level of the stored data in the cache line and their respective weights, and then determines the replaceable cache line according to the replacement score of each cache line. The PLRU path value is a new concept proposed by the present disclosure, which reflects the local access time of the cache line, that is, the access frequency of the cache line and the priority of the access path (the smaller the path value, the longer the cache line has not been accessed and the easier it is to be replaced), and the importance level is the business value of the data block marked by software or hardware (such as the weight data in the AI model marked as critical), which can reflect the importance of the data. The existing PLRU algorithm usually only relies on the binary tree path state bit to determine the replaced cache line, while the scheme in the present disclosure introduces the PLRU path value, the data importance level and their respective dynamic weights, and then obtains the replacement score based on these parameters. Therefore, through the quantitative scoring method, the value of each cache line can be more comprehensively evaluated, so that the truly "unimportant" and "infrequently used" cache line can be more accurately selected for replacement, avoiding the unreasonable replacement caused by the qualitative judgment method of the traditional PLRU relying only on the binary tree path state bit, reducing the situation of useful data being replaced by mistake, and thus improving the cache hit rate. Moreover, while significantly improving the cache hit rate (compared with PLRU), the hardware overhead is also lower than that of the LRU algorithm.

[0035] Figure 1A flow chart of a cache line replacement method according to an embodiment of the present disclosure is shown. The method is applied to a chip, which includes a data cache including N cache lines of stored data, and a binary tree structure including N-1 nodes in a pseudo least recently used (PLRU) algorithm is used to manage a corresponding access path of each cache line, wherein each node corresponds to a path state bit, which is used to dynamically record an access direction preference, .

[0036] In a specific example of the present disclosure, the data cache includes a cache memory (Cache), and a data mapping manner between the Cache and a main memory uses a set-associative mapping, that is, the Cache is divided into a plurality of sets, each set including N Cache lines (i.e., cache lines), and a data block in the main memory is first mapped to a set through a set index, and then a full-associative manner is used in the set, that is, the data block can be stored in any cache line in the set.

[0037] The cache line includes a data area and a Tag area, the data area is a part of the cache line for storing actual data. It contains specific data content read from the main memory, and the Tag area is a part of the cache line for storing metadata related to the cache line, including address information, validity bits, dirty bits, etc., for managing and identifying the cache line to ensure that the data in the cache line is correct and up-to-date. For N cache lines of stored data, the stored data is stored in the data area of the cache line.

[0038] In the PLRU algorithm, a binary tree structure is used to efficiently manage the access state of the cache line, and its core role is to approximate simulate the LRU behavior through the bit mask (state bit) of the node. For an N-way set-associative Cache, N-1 binary state bits (nodes) are required, and each node corresponds to a hardware register or SRAM bit.

[0039] Figure 2 A corresponding relationship diagram of a binary tree structure including N-1 nodes in the PLRU algorithm and N cache lines is shown. Taking an example of a data cache including 8 cache lines (first cache line to eighth cache line), as shown in Figure 2As shown, the binary tree structure contains a total of 7 nodes (Bit0-Bit6). Among them, the root node (Bit0) is at the top of the binary tree and determines the global replacement direction. The intermediate nodes (Bit1, Bit2) make branching decisions and point to subtrees. For example, Bit1 points to the left subtree (Bit1 branch), and Bit2 points to the right subtree (Bit2 branch). The leaf nodes are directly associated with specific cache lines, such as Bit3 associated with the first and second cache lines, Bit4 associated with the third and fourth cache lines, Bit5 associated with the fifth and sixth cache lines, and Bit6 associated with the seventh and eighth cache lines. The path from the root node to the leaf node of the binary tree is mapped to each cache line. For example, the access path corresponding to the first cache line is Bit0→Bit1→Bit3, and the access path corresponding to the sixth cache line is Bit0→Bit2→Bit5.

[0040] Among them, the path status bit (0 or 1) of each node indicates the priority replacement direction. In a specific example, 0 indicates that the left subtree is replaced first, and 1 indicates that the right subtree is replaced first. For example: if Bit0=0, the left subtree is replaced first (Bit1 branch); if Bit0=1, the right subtree is replaced first (Bit2 branch).

[0041] Figure 3 Figure 2 shows the determined path for replacing cache lines in the PLRU algorithm when the initial path status bits of all nodes are 0. Figure 3 As shown, in the initial state, that is, when the initial path state bits of all nodes are 0, according to the traditional PLRU algorithm rules, specifically as shown by the red path, the first cache line is eventually replaced, for example: data Data0 is stored.

[0042] Figure 4 The following figure shows the change of each node status bit after the first cache line is replaced in the PLRU algorithm and the determination path of the next replaceable cache line. Figure 4 As shown in the figure, after the first cache line is replaced, trace back from the leaf node to the root node, flip all the status bits on the path, and set the node status on the access path to the non-current direction (that is, the other side will be replaced first next time), that is: set Bit0, Bit1 and Bit3 to 1. In this way, according to the traditional PLRU algorithm rules, as shown in the red path, the next replaceable cache line is the fifth cache line.

[0043] Depend on Figure 3 and Figure 4 As shown, in the traditional PLRU algorithm, the direction preference (0 / 1) of the path status bit directly determines the replacement path, and the candidate line pointed to by the path status bit combination logic is the replaceable cache line.

[0044] In the present disclosure, the method for determining the replaceable cache line is essentially different from the conventional PLRU algorithm. As shown in Figure 1 The cache line replacement method involved in the present disclosure includes the following steps S110-S140: In step S110, in response to the received data replacement instruction, the path state bits from the root node to the leaf node on the access path corresponding to each cache line are obtained.

[0045] The data replacement instruction is usually triggered when there is a cache miss (such as a write miss or a read miss) and no free cache line (the cache is full). When the processor needs to write new data but there is a write miss, the data to be cached includes the data to be written. When the processor reads data but there is a read miss, new data needs to be loaded from the main memory, and the data to be cached includes the data to be loaded. After triggering the data replacement instruction, the execution of the cache line replacement method of the present disclosure is triggered.

[0046] Unlike the conventional PLRU algorithm which selects the replaceable cache line according to the combination logic of the path state bits (0 / 1), the cache line replacement method in the present disclosure first needs to obtain the path state bits from the root node to the leaf node on the access path corresponding to each cache line. For example, as shown in Figure 4 The path state bits from the root node to the leaf node on the access path corresponding to the first cache line and the second cache line are both 1 (Bit0), 1 (Bit1), and 1 (Bit3). The path state bits from the root node to the leaf node on the access path corresponding to the third cache line and the fourth cache line are both 1 (Bit0), 1 (Bit1), and 0 (Bit4). The path state bits from the root node to the leaf node on the access path corresponding to the fifth cache line and the sixth cache line are both 1 (Bit0), 0 (Bit2), and 0 (Bit5). The path state bits from the root node to the leaf node on the access path corresponding to the seventh cache line and the eighth cache line are both 1 (Bit0), 0 (Bit2), and 0 (Bit6).

[0047] In step S120, for a specified cache line, the PLRU path value corresponding to the specified cache line is calculated according to the path state bits from the root node to the leaf node on the access path corresponding to the specified cache line. The importance level of the data stored in the specified cache line is obtained. According to the PLRU path value corresponding to the specified cache line and the importance level of the data stored in the specified cache line, as well as the specified PLRU path value weight and the specified importance level weight, the replacement score of the specified cache line is calculated.

[0048] According to an embodiment of the present disclosure, the PLRU path value corresponding to the specified cache line is calculated according to the path state bits from the root node to the leaf node on the access path corresponding to the specified cache line, which includes: The path state bits from the root node to the leaf node on the corresponding access path of the specified cache line are added to obtain a PLRU path value corresponding to the specified cache line.

[0049] Also take the state of each node as an example, for each cache line in the first cache line to the eighth cache line, after obtaining the path state bits from the root node to the leaf node on the corresponding access path, the PLRU path value corresponding to the first cache line and the second cache line is 3, the PLRU path value corresponding to the third cache line and the fourth cache line is 2, and the PLRU path value corresponding to the fifth cache line to the eighth cache line is 1. Figure 4

[0050] In the present disclosure, the essence of the invention principle of introducing the concept of PLRU path value is to convert the discrete logic of PLRU binary tree state bits into continuous priority score. By quantifying the access history, the "cold data" can be more accurately identified, thereby improving the hit rate. Specifically, the PLRU path value can reflect the access frequency of the cache line and the priority of the access path, thereby indirectly reflecting the access locality of the data. Taking the PLRU path value as a factor for calculating the replacement score can scientifically use the principle of access locality to evaluate the importance of the cache line, which is consistent with the internal law of program running. The smaller the PLRU path value, the lower the priority of the cache line in the access path, i.e. the longer time it has not been accessed; the larger the PLRU path value, the higher the priority of the cache line in the access path, i.e. the greater the possibility of recent access. This quantitative method can provide an objective and quantifiable index for the replacement score, making the replacement decision more scientific and accurate.

[0051] ​In the traditional PLRU algorithm, a replacement path is directly selected according to the current value of the binary tree state bit, and then a cache line that can be replaced is determined. This method only reflects the direction of the last access, and may ignore the historical access pattern. In the present disclosure, the PLRU path value obtained by adding the current value of the binary tree state bit on each cache line corresponding access path reflects the long-term access tendency, which not only considers the last access, but also comprehensively considers the previous access history, and can more comprehensively reflect the usage of the cache line. This comprehensive consideration of the access history enables the PLRU path value to better reflect the actual use value of the data, providing a depth and breadth evaluation basis for replacement scoring. Compared with the algorithm that only considers the last access, the evaluation based on the PLRU path value can more scientifically avoid making incorrect judgments about the importance of the cache line due to accidental access. In addition, when some cache lines are frequently alternately accessed, the state bit of the traditional PLRU will constantly flip, causing the replacement target to fluctuate. In this case, the advantage of the path value addition is that it smooths short-term fluctuations by adding values, and preferentially replaces cache lines that have not been paid attention to for a long time (such as cache lines with the lowest path value). In addition, the goal of cache replacement is to retain as much data as possible that may be accessed again in the future in a limited cache space, in order to improve cache hit rate. The PLRU path value, as an indicator reflecting the access priority of the cache line, is highly consistent with the goal of cache replacement. By incorporating the PLRU path value into the calculation of the replacement score, the replacement decision can be made more in line with the goal of cache replacement, i.e., preferentially replacing data with low access priority and long time of non-access, thereby improving the utilization of cache space and cache hit rate.

[0052] Although the PLRU path value itself is an important evaluation indicator, the present disclosure does not use it as the only decision basis, but combines it with other factors (such as the importance level of data) for comprehensive evaluation. This comprehensive evaluation method can fully utilize the advantages of various factors and avoid decision errors caused by a single factor. The PLRU path value plays a role in reflecting access history and access priority in the comprehensive evaluation, and is complementary to other factors, making the replacement score more comprehensive and accurate, thereby improving the rationality of the cache replacement decision.

[0053] In the present disclosure, the importance level can be represented using 2 bits or more, and when represented using 2 bits or more (such as 3 bits), a more fine-grained importance level (such as differences in weights of different layers of an AI model) is supported. The number of bits used can be selected according to the specific application scenario. Taking 2 bits as an example, 4 levels of importance level are supported, which can be set as: 00 represents low importance, 01 represents medium importance, 10 represents high importance, and 11 represents critical data block.

[0054] In addition, the storage of the importance level of the data stored in the cache line can be in various ways. For example, a region for storing the importance level can be specially arranged in the cache line structure (in parallel with the Tag region and the Data region), a separate mapping table can be maintained in the Cache controller to query the corresponding importance level through the cache line index, or the importance level can be encoded into the header of the data block or the ECC check bit (e.g., using the redundant bits of the ECC).

[0055] In a specific example, the present disclosure stores the importance level of the data stored in the specified cache line in the free bit or the reserved bit of the Tag region of the specified cache line. Thus, when the importance level of the data stored in the specified cache line is obtained, the importance level of the data stored in the specified cache line can be obtained from the Tag region in the specified cache line.

[0056] Storing the importance level in the Tag region of the cache line has the following beneficial effects compared to other storage methods mentioned above: 1. Hardware cost is saved, and no additional storage unit is needed. Storing the importance level in the Tag region does not need to open a new storage space in the chip to store the importance level separately, and the free or reserved bit is used to store the importance level, which does not increase the cost of additional storage units.

[0057] 2. The control circuit design is reduced. If a separate storage structure is used to store the importance level, additional control circuits are needed to manage the read and write operations of the storage structure, including address decoding, data transmission, etc. Storing the importance level in the Tag region can use the existing cache control circuit to manage, without the need for additional complex control circuits, thereby further reducing the hardware cost.

[0058] 3. The program processing flow is simplified. When the importance level is stored in the Tag region, the program can directly access the corresponding importance level while accessing the cache line data, without the need for additional access steps. For example, when reading the cache line data, the program can obtain the data and its corresponding importance level from the cache line at one time, without the need to first access the cache line to obtain the data and then access another separate storage structure to obtain the importance level, thereby simplifying the data access flow.

[0059] Compared with other methods, if the data is stored together, although this method does not require an additional storage structure, it will increase the complexity of data processing. For example, when reading data, it is necessary to first parse the high or low bits of the data to obtain the importance level, and then process the data normally. This will increase the number of instructions and execution time of the program. Moreover, if the data format changes, the program may need to be modified to adapt to the new importance level storage method, increasing the maintenance cost of the program. If a separate storage structure is used for storage, this method will increase hardware costs and complicate the program processing flow. The program requires additional instructions to access the independent storage structure and needs to handle the mapping relationship between data and importance levels, which increases the complexity of the program and the possibility of errors.

[0060] After obtaining the PLRU path value corresponding to the specified cache line and the importance level of the data stored in the specified cache line, the replacement score of the specified cache line is calculated by the following formula according to the PLRU path value corresponding to the specified cache line and the importance level of the data stored in the specified cache line, as well as the specified PLRU path value weight and the specified importance level weight. : ; in, Indicates the PLRU path value corresponding to the specified cache line, Indicates the importance level of the data stored in the specified cache line, Indicates the specified PLRU path value weight, represents the specified importance level weight, and are integers not less than 1.

[0061] Specifically, the PLRU path value weight and importance level weight can be specified by an external program, and for different levels of cache (such as L1 / L2 / L3 cache), the PLRU path value weight and importance level weight can be configured independently.

[0062] Also Figure 4 Assume that the status of each node is as follows. and are 1 respectively, then the PLRU path value corresponding to each cache line and the importance level and replacement score RS of the stored data are shown in Table 1 below.

[0063] Table 1 PLRU path value corresponding to each cache line and the importance level and replacement score RS first comparison table of stored data

[0064] According to an embodiment of the present disclosure, in the calculation of the replacement score of the specified cache line by the formula, the multiplication operation in the formula is realized by binary decomposition and weighted accumulation based on a shift-add circuit, and then the product results after the multiplication operation are added by an adder to obtain the replacement score of the specified cache line.

[0065] Unlike the practice of setting the weight as a decimal number, in the present disclosure, the and are set as integers no less than 1, and the weight multiplication is realized by a shift-add operation, which has the beneficial effects of: on the one hand, simplifying the hardware implementation, for example, if =0.3 needs to be converted into a fixed-point number (such as Q1.15 format), compared with =3, the area of the multiplier needs to be increased by 3-5 times; on the other hand, improving the energy efficiency, respectively taking the calculation of 0. and as examples, if the former hardware unit uses a 16-bit fixed-point multiplier, the number of gate circuits is about 200 gates, and the dynamic power consumption is about 2mW / MHz, while the latter hardware unit uses a shifter + adder, the number of gate circuits can be reduced to about 30 gates, and the dynamic power consumption is only about 0.5mW / MHz. In addition, the integer multiplication-addition can be completed in a single cycle, while the decimal multiplication needs multi-stage pipelining (such as 3-cycle delay), so that under the same process, the integer scheme can improve the frequency by 10-15%, and the integer operation can also avoid floating-point truncation error, so that the precision is lossless.

[0066] The shift-add circuit includes a shift register and an accumulator; the multiplication operation in the formula is realized by binary decomposition and weighted accumulation based on the shift-add circuit, including: determining whether the importance level of the stored data in the specified cache line is a preset low importance level. For example, when the importance level parameter is represented by 2-bit digits, the preset low importance level can be set as 00.

[0067] If not, a full-precision calculation mode is started, including: decomposing the multiplier by binary digits, detecting all bits in the order from right to left, if the i-th bit is 1, then the multiplicand is left-shifted by i bits by the shift register to obtain the effective shift result of the corresponding i-th bit; if it is 0, it is skipped, until all bit orders are detected; the effective shift results of all bits of the multiplicand are added by the accumulator to obtain the final product result.

[0068] If yes, the approximate calculation optimization mode is started, including: decomposing the multiplier according to binary bits, selecting the corresponding valid bits in the multiplier according to the preset truncation bit width configuration parameters, and detecting the valid bits one by one from right to left in bit order. If the i-th bit is 1, the multiplicand is shifted left by i bits through the shift register to obtain the valid shift result corresponding to the i-th bit; if it is 0, it is skipped until all bit orders in the valid bits are detected; the valid shift results of all bits of the multiplicand and the truncation error compensation value are added through the accumulator to obtain the final product result; wherein, the truncation error compensation value is obtained based on the average truncation error compensation of offline statistics and dynamic weight calculation, .

[0069] The present disclosure provides two calculation modes in cache line replacement score calculation. For high-importance data, full-precision calculation mode is enabled, while for low-importance data (such as =00) enables approximate computing, reducing power consumption by truncating low-order accumulations, while maintaining stable system performance through error compensation. This solution, through the collaborative design of hardware dynamic truncation and software error compensation, significantly reduces power consumption while maintaining system performance. It is particularly suitable for cache management with hierarchical importance in edge AI devices.

[0070] In step S130 , replaceable cache lines in the data cache are determined according to the replacement score of each cache line.

[0071] like Figure 5 As shown, determining the replaceable cache lines in the data cache according to the replacement score of each cache line includes the following steps S131 to S134: In step S131 , the lowest replacement score among the replacement scores of each cache line is obtained.

[0072] In step S132 , it is determined whether the number of the lowest replacement scores is one.

[0073] If the number of the lowest replacement scores is one, step S133 is executed; if the number of the lowest replacement scores is multiple, step S134 is executed.

[0074] In step S133 , the cache line corresponding to the lowest replacement score is used as a replaceable cache line.

[0075] In step S134, a replaceable cache line in the data cache is found based on the PLRU algorithm.

[0076] Also Figure 4 Take the status of each node as an example. Figure 6 FIG. 4 shows a determination path for selecting a replaceable cache line based on a replacement score according to an embodiment of the present disclosure. Figure 6As shown in the red path, for the RS results calculated in Table 1 above, if the number of lowest replacement scores is one, the sixth cache line with the lowest replacement score will be selected as the replaceable cache line. Figure 4 Based on the traditional PLRU algorithm, the fifth cache line is considered a replaceable cache line. Although the PLRU path values ​​for the fifth and sixth cache lines are identical (both 1), the fifth cache line has a higher importance than the data stored in the sixth cache line, so the sixth cache line is replaced first. Thus, determining the replaceable cache line based on the cache line replacement scheme disclosed herein reduces the risk of important data being mistakenly replaced, thereby improving the cache hit rate.

[0077] The present disclosure also considers the case where there are multiple lowest replacement scores. For this case, the processing method is to fall back to the traditional PLRU algorithm, that is, to find and obtain replaceable cache lines in the data cache based on the PLRU algorithm.

[0078] Figure 7 Shown with Figure 6 As an example, the status bits of each node are shown in FIG. 6 , and the status bits of each node are changed after the sixth cache line is replaced as a replaceable cache line. Assuming that the data stored is If the value is 01, the PLRU path value corresponding to each cache line and the importance level and replacement score RS of the stored data are shown in Table 2. According to the RS results calculated in Table 2, there are two with the lowest RS of 2, which meets the situation that there are multiple lowest replacement scores. At this time, the replaceable cache line in the data cache is found based on the PLRU algorithm. As a result, the next replaceable cache line is the third cache line, as shown in Figure 7 Shown in red.

[0079] Table 2 PLRU path value corresponding to each cache line and the importance level and replacement score RS second comparison table of stored data

[0080] According to an embodiment of the present disclosure, when obtaining the lowest replacement score among the replacement scores of each cache line, it is implemented based on a multi-level comparison tree structure, wherein each level of comparator in the multi-level comparison tree structure is a binary comparator, and the binary comparator is used to receive the replacement scores of any two cache lines as input, and output the smaller value and its corresponding cache line index.

[0081] The implementation based on the multi-level comparison tree structure includes: the first-level comparator performs group comparison on all cache lines in pairs to obtain a first-level candidate minimum value set; the subsequent comparators in each level continue to group and compare the candidate minimum values ​​output by the previous level until the cache line index corresponding to the final lowest replacement score is output.

[0082] In step S140, the data to be cached and the importance level corresponding to the data to be cached are obtained, and the data to be cached and the importance level corresponding to the data to be cached are stored in the replaceable cache line to replace the stored data and the importance level of the stored data in the replaceable cache line.

[0083] The data to be cached may be obtained from a CPU or a main memory, and the importance level corresponding to the data to be cached may be specified by software or hardware.

[0084] According to an embodiment of the present disclosure, storing the to-be-cached data and the importance level corresponding to the to-be-cached data in the replaceable cache line includes: The data to be cached is stored in the data area of ​​the replaceable cache line, and the importance level corresponding to the data to be cached is stored in the tag area of ​​the replaceable cache line, for example, in an idle bit or a reserved bit of the tag area of ​​the replaceable cache line.

[0085] In addition, after the data to be cached and the importance level corresponding to the data to be cached are stored in the replaceable cache line, the path status bits of the nodes in the binary tree structure are updated based on the PLRU algorithm.

[0086] by Figure 6 Taking the status bits of each node shown in the figure as an example, after the sixth cache line is replaced with the cached data and its corresponding importance level, the status bits of the access path from the leaf node to the root node corresponding to the sixth cache line are flipped to the non-current direction, that is, Bit 5 is set to 1, Bit 2 is set to 1, and Bit 0 is set to 0. When triggered again, steps S110 to S140 are repeated to perform the next cache line replacement.

[0087] Figure 8 FIG. 1 is a flow chart showing another cache line replacement method according to an embodiment of the present disclosure. Figure 8 In addition to the above steps S110 to S140, the method further includes the following step S150: In step S150, obtain the historical hit rate, future predicted hit rate and at least one parameter of the future predicted access pattern of the data cache; dynamically adjust the specified PLRU path value weight and the specified importance level weight according to the historical hit rate, future predicted hit rate and at least one parameter of the future predicted access pattern of the data cache.

[0088] The data cache records the number of cache hits and the total number of accesses through two counters, and the ratio of the number of cache hits to the total number of accesses is the cache hit rate. The historical hit rate refers to the cache hit rate of a certain time period or a certain number of accesses in the past, such as the last 1K / 10K / 100K accesses.

[0089] The future prediction hit rate refers to predicting the cache hit rate in a certain time period in the future based on the current access pattern and historical hit rate data. Machine learning algorithms (such as linear regression, time series analysis, etc.) or simple statistical methods can be used to establish a hit rate prediction model, and historical hit rate and access pattern are used as input data to train the prediction model (such as lightweight LSTM), and then use the trained model to predict the hit rate in a certain time period in the future based on the current access pattern and historical hit rate data. The predicted hit rate is stored in a variable or data structure for subsequent use.

[0090] The access pattern refers to the characteristic description of the access behavior to the cache or storage system, which reflects the regularity and predictability of data access, including but not limited to access frequency (indicating the number of times a certain data or address is accessed), access time interval (indicating the time interval between two consecutive accesses), access type (indicating the type of access, such as read or write), access range (access range can reflect the use range of data, larger access range may indicate the importance and complexity of data) and the like.

[0091] When the specified PLRU path value weight and the specified importance level weight are dynamically adjusted according to at least one of the historical hit rate of the data cache, the future prediction hit rate, and the future prediction access pattern, the adjustment strategy and rules can be defined according to specific application scenarios and actual needs, such as: if the historical hit rate and / or future prediction hit rate is higher than a preset first reference threshold, it means that the current cache management strategy is effective, and the current PLRU path value weight and importance level weight do not need to be adjusted; if the historical hit rate and / or future prediction hit rate is lower than a preset second reference threshold, it means that the current cache management strategy may need to be adjusted, and the PLRU path value weight can be appropriately increased and the importance level weight can be appropriately reduced; if the future prediction access pattern shows that some data will be frequently accessed, the importance of these data is higher, therefore the importance level weight of these data can be appropriately increased to ensure that these data will not be replaced; if the future prediction access pattern shows that some data will be rarely accessed, the importance of these data is lower, therefore the importance level weight of these data can be appropriately reduced to avoid retaining unnecessary data and improve the utilization of the cache.

[0092] The disclosure makes the weight adjustment rule dynamically adapt to different running environments and data access patterns by comprehensively considering the historical hit rate, the future predicted hit rate and the future predicted access pattern. The dynamic adjustment mechanism enables the cache management strategy to be optimized according to the actual situation, thereby improving the cache hit rate and system performance. In addition, the PLRU path value mainly reflects the locality of access, and the importance level mainly reflects the importance of data. By dynamically adjusting the weights of the two, a balance can be achieved between locality and importance, making the cache management strategy more comprehensive and effective. Moreover, the introduction of the future predicted hit rate and the access pattern enables the cache management strategy to have a certain prediction ability, which can adapt to future changes in advance, thereby improving the adaptability and performance of the cache.

[0093] Figure 9 A structural block diagram of a cache line replacement device according to an embodiment of the disclosure is shown. The device is applied to a chip, the chip including a data cache including N cache lines of stored data, the cache line replacement device including a PLRU algorithm implementation module, a PLRU path value calculation module, a replacement score calculation module, a replaceable cache line determination module and a replacement operation module. As shown in Figure 9 , wherein: The PLRU algorithm implementation module is configured to manage each cache line corresponding access path based on a binary tree structure including N-1 nodes in a pseudo least recently used PLRU algorithm, wherein each node corresponds to a path state bit, and the path state bit is used to dynamically record the access direction preference, The PLRU path value calculation module is configured to, in response to a received data replacement instruction, acquire path state bits from a root node to a leaf node on each cache line corresponding access path; calculate a PLRU path value corresponding to a specified cache line according to path state bits from a root node to a leaf node on the specified cache line corresponding access path; the replacement score calculation module is configured to acquire an importance level of stored data in the specified cache line; calculate a replacement score of the specified cache line according to the PLRU path value corresponding to the specified cache line and the importance level of the stored data in the specified cache line, and a specified PLRU path value weight and a specified importance level weight; the replaceable cache line determination module is configured to determine a replaceable cache line in the data cache according to the replacement score of each cache line; and the replacement operation module is configured to acquire to-be-cached data and an importance level corresponding to the to-be-cached data, and store the to-be-cached data and the importance level corresponding to the to-be-cached data in the replaceable cache line to replace stored data in the replaceable cache line and an importance level of the stored data.

[0094] According to an embodiment of the present disclosure, the replacement score of the specified cache line is calculated by a formula as follows : ; wherein, represents a PLRU path value corresponding to the specified cache line, represents an importance level of the data stored in the specified cache line, represents a specified PLRU path value weight, represents a specified importance level weight, and are integers not less than 1.

[0095] According to an embodiment of the present disclosure, when the replacement score of the specified cache line is calculated by the formula, the multiplication operation in the formula is implemented by binary decomposition and weighted accumulation based on a shift-add circuit, and then the product results after multiplication are added by an adder to obtain the replacement score of the specified cache line.

[0096] According to an embodiment of the present disclosure, the shift-add circuit includes a shift register and an accumulator; and the multiplication operation in the formula is implemented by binary decomposition and weighted accumulation based on the shift-add circuit, including: determining whether the importance level of the data stored in the specified cache line is a preset low importance level.

[0097] If no, a full-precision calculation mode is started, including: decomposing the multiplier by binary bits, detecting all bits in bit order from right to left, if the i-th bit is 1, then the multiplicand is left shifted by i bits by the shift register to obtain the effective shift result corresponding to the i-th bit; if it is 0, then it is skipped, until all bit orders are detected; the effective shift results of all bits of the multiplicand are added by the accumulator to obtain the final product result.

[0098] If yes, an approximate calculation optimization mode is started, including: decomposing the multiplier by binary bits, selecting the effective bits in the multiplier according to a preset truncation bit width configuration parameter, detecting the effective bits in bit order from right to left, if the i-th bit is 1, then the multiplicand is left shifted by i bits by the shift register to obtain the effective shift result corresponding to the i-th bit; if it is 0, then it is skipped, until all bit orders in the effective bits are detected; the effective shift results of all bits of the multiplicand and a truncation error compensation value are added by the accumulator to obtain the final product result; wherein the truncation error compensation value is obtained based on an offline statistical average truncation error compensation and a dynamic weight, .

[0099] According to an embodiment of the present disclosure, the determining the replaceable cache line in the data cache according to the replacement score of each cache line comprises: Obtaining the lowest replacement score in the replacement scores of each cache line; if the number of the lowest replacement scores is one, taking the cache line corresponding to the lowest replacement score as the replaceable cache line; if the number of the lowest replacement scores is multiple, obtaining the replaceable cache line in the data cache based on the PLRU algorithm.

[0100] According to an embodiment of the present disclosure, when the lowest replacement score in the replacement scores of each cache line is obtained, the implementation is based on a multi-stage comparison tree structure, wherein each comparator in the multi-stage comparison tree structure is a binary comparator, the binary comparator is used to receive the replacement scores of any two cache lines as input, and output the smaller value and the cache line index corresponding thereto; the implementation based on the multi-stage comparison tree structure comprises: the first-stage comparator compares all cache lines in pairs to obtain a first-stage candidate minimum value set; each subsequent-stage comparator continues to compare the candidate minimum values output by the previous stage, until the cache line index corresponding to the final lowest replacement score, i.e. the index of the replaceable cache line, is output; when the comparator identifies that the number of the lowest replacement scores is multiple, the PLRU_en signal is output to the PLRU algorithm implementation module to enable the PLRU algorithm implementation module to select the final replaceable cache line based on the PLRU algorithm.

[0101] Figure 10 A specific structure example of a cache line replacement device according to an embodiment of the present disclosure is shown. Taking an example that the data cache contains 8 cache lines, each module in the cache line replacement device is implemented in hardware circuit, wherein the PLRU algorithm implementation circuit corresponds to the PLRU algorithm implementation module, the PLRU path value calculation module is implemented in the form of an adder, the replacement score calculation module comprises a shift-add circuit and an adder, and the replaceable cache line determination module is implemented in the form of a comparison tree structure circuit. As shown in Figure 10As shown, the PLRU algorithm implementation circuit outputs the path state bits (Path_status) from the root node to the leaf node on each cache line corresponding access path to the adder, the adder adds the path state bits of the same cache line to obtain the PLRU path value corresponding to each cache line (such as the PLRU path value corresponding to the first cache line is Line1_path_value), and outputs the PLRU path value corresponding to each cache line to the shift add circuit. The shift add circuit calculates the replacement score corresponding to each cache line (such as the replacement score corresponding to the first cache line is Line1_RS) according to the received replacement score corresponding to each cache line, the importance level of the data stored in each cache line (such as the importance level of the data stored in the first cache line is Line1_importance), and the specified PLRU path value weight w1 and the specified importance level weight w2, and outputs the replacement score corresponding to each cache line to the comparison tree structure circuit. The comparison tree structure circuit obtains the lowest replacement score from the replacement scores of each cache line. If the number of the lowest replacement score is one, the cache line corresponding to the lowest replacement score is taken as the replaceable cache line, and the cache line index corresponding to the lowest replacement score is output. If the number of the lowest replacement score is more than one, the PLRU_en signal is output to the PLRU algorithm implementation circuit to enable the PLRU algorithm implementation circuit to select the final replaceable cache line based on the PLRU algorithm, and the cache line index corresponding to the lowest replacement score is finally output by the PLRU algorithm implementation circuit.

[0102] Figure 11 A structural block diagram of another cache line replacement device according to an embodiment of the present disclosure is shown. As shown, in addition to the modules shown in the above structural block diagram of the cache line replacement device, the cache line replacement device according to an embodiment of the present disclosure further comprises a weight dynamic adjustment module configured to: obtain at least one parameter in the historical hit rate, the future predicted hit rate and the future predicted access pattern of the data cache; and dynamically adjust the specified PLRU path value weight and the specified importance level weight according to at least one parameter in the historical hit rate, the future predicted hit rate and the future predicted access pattern of the data cache. Figure 11 Figure 10

[0103] The present disclosure also provides a chip comprising the device according to any one of the device embodiments above; or comprising a memory and a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of the method embodiments above.

[0104] Figure 12 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. As shown, in addition to the modules shown in the above structural block diagram of the electronic device, the electronic device according to an embodiment of the present disclosure further comprises a weight dynamic adjustment module configured to: obtain at least one parameter in the historical hit rate, the future predicted hit rate and the future predicted access pattern of the data cache; and dynamically adjust the specified PLRU path value weight and the specified importance level weight according to at least one parameter in the historical hit rate, the future predicted hit rate and the future predicted access pattern of the data cache. Figure 12 ​​As shown, the electronic device includes a memory and a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any one of the above method embodiments.

[0105] According to the technical scheme provided by the embodiment of the present disclosure, a high-performance cache line replacement method based on multi-factor evaluation is provided. The method is applied to a chip, the chip includes a data cache, the data cache includes N cache lines of stored data, a binary tree structure including N-1 nodes in a PLRU algorithm is used to manage an access path corresponding to each cache line, wherein each node corresponds to a path state bit, the path state bit is used to dynamically record the access direction preference, in response to a received data replacement instruction, the path state bits from the root node to the leaf node on the access path corresponding to each cache line are obtained; for a specified cache line, the PLRU path value corresponding to the specified cache line is calculated according to the path state bits from the root node to the leaf node on the access path corresponding to the specified cache line; the importance level of the stored data in the specified cache line is obtained; the replacement score of the specified cache line is calculated according to the PLRU path value corresponding to the specified cache line and the importance level of the stored data in the specified cache line, and the specified PLRU path value weight and the specified importance level weight; the replaceable cache line in the data cache is determined according to the replacement score of each cache line; the to-be-cached data and the importance level corresponding to the to-be-cached data are stored in the replaceable cache line to replace the stored data in the replaceable cache line and the importance level of the stored data. Thus, by combining the PLRU path value and the importance level of the stored data to evaluate the cache line replacement priority, the cache hit rate is effectively improved, and the system performance is optimized.

[0106] The above description is merely preferred embodiments of the present disclosure and a description of the principles of the technology used. Those skilled in the art should understand that the scope of the application involved in the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features without departing from the inventive concept. For example, the above features are replaced with each other to form a technical solution with similar functions disclosed in the present disclosure (but not limited to).

Claims

1. A cache line replacement method, characterized in that: The method is applied to a chip, the chip including a data cache, the data cache including N cache lines of stored data, and managing the access path corresponding to each cache line based on a binary tree structure containing N-1 nodes in a pseudo least recently used (PLRU) algorithm, wherein each node corresponds to a path status bit, and the path status bit is used to dynamically record an access direction preference. The method comprises: In response to the received data replacement instruction, obtaining a path status bit from a root node to a leaf node on an access path corresponding to each cache line; For a specified cache line, calculating a PLRU path value corresponding to the specified cache line based on a path status bit from a root node to a leaf node on an access path corresponding to the specified cache line; obtaining an importance level of data stored in the specified cache line; and calculating a replacement score for the specified cache line based on the PLRU path value corresponding to the specified cache line and the importance level of the data stored in the specified cache line, as well as a specified PLRU path value weight and a specified importance level weight; determining replaceable cache lines in the data cache according to the replacement score of each cache line; Obtaining data to be cached and an importance level corresponding to the data to be cached, and storing the data to be cached and the importance level corresponding to the data to be cached in the replaceable cache line to replace the stored data and the importance level of the stored data in the replaceable cache line.

2. The method according to claim 1, characterized in that The binary tree structure including N-1 nodes in the pseudo least recently used (PLRU) algorithm manages the access path corresponding to each cache line, including: After the to-be-cached data and the importance level corresponding to the to-be-cached data are stored in the replaceable cache line, the path status bits of the nodes in the binary tree structure are updated based on the PLRU algorithm.

3. The method according to claim 1, characterized in that The cache line includes a data area and a tag area; The storing the to-be-cached data and the importance level corresponding to the to-be-cached data in the replaceable cache line includes: Storing the data to be cached in a data area of ​​the replaceable cache line, and storing the importance level corresponding to the data to be cached in a tag area of ​​the replaceable cache line; The acquiring the importance level of the data stored in the designated cache line includes: acquiring the importance level of the data stored in the designated cache line from a Tag area in the designated cache line.

4. The method according to claim 1, wherein The calculating the PLRU path value corresponding to the designated cache line according to the path status bit from the root node to the leaf node on the access path corresponding to the designated cache line includes: The path status bits from the root node to the leaf node on the access path corresponding to the designated cache line are added to obtain the PLRU path value corresponding to the designated cache line.

5. The method according to claim 1, wherein The determining of replaceable cache lines in the data cache according to the replacement score of each cache line comprises: Obtain the lowest replacement score among the replacement scores of each cache line; if the number of the lowest replacement scores is one, use the cache line corresponding to the lowest replacement score as a replaceable cache line; if the number of the lowest replacement scores is multiple, find the replaceable cache line in the data cache based on the PLRU algorithm.

6. The method according to claim 5, characterized in that When obtaining the lowest replacement score among the replacement scores of each cache line, it is implemented based on a multi-level comparison tree structure, wherein each level comparator in the multi-level comparison tree structure is a binary comparator, and the binary comparator is used to receive the replacement scores of any two cache lines as input and output the smaller value and its corresponding cache line index; The implementation based on the multi-level comparison tree structure includes: the first-level comparator performs group comparison on all cache lines in pairs to obtain a first-level candidate minimum value set; the subsequent comparators in each level continue to group and compare the candidate minimum values ​​output by the previous level until the cache line index corresponding to the final lowest replacement score is output.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining at least one parameter of a historical hit rate, a future predicted hit rate, and a future predicted access pattern of the data cache; The specified PLRU path value weight and the specified importance level weight are dynamically adjusted according to at least one parameter of a historical hit rate, a future predicted hit rate, and a future predicted access pattern of the data cache.

8. The method according to claim 1, characterized in that The replacement score of the specified cache line is calculated by the following formula: : ； in, Indicates the PLRU path value corresponding to the specified cache line, Indicates the importance level of the data stored in the specified cache line, Indicates the specified PLRU path value weight, represents the specified importance level weight, and are integers not less than 1.

9. The method according to claim 8, characterized in that When calculating the replacement score of the specified cache line using the formula, the multiplication operation in the formula is implemented through binary decomposition and weighted accumulation based on the shift-and-add circuit, and then the product results after the multiplication operation are added through an adder to obtain the replacement score of the specified cache line.

10. The method according to claim 9, characterized in that The shift-and-add circuit includes a shift register and an accumulator; the shift-and-add circuit implements the multiplication operation in the formula through binary decomposition and weighted accumulation, including: Determining whether the importance level of the data stored in the designated cache line is a preset low importance level; If not, start the full-precision calculation mode, including: decomposing the multiplier according to the binary bit, checking all the bits one by one from right to left in the bit order, if the i-th bit is 1, then shift the multiplicand left by i bits through the shift register to obtain the valid shift result corresponding to the i-th bit; if it is 0, skip it until all the bit orders are checked; add the valid shift results of all bits of the multiplicand through the accumulator to obtain the final product result; If so, the approximate calculation optimization mode is started, including: decomposing the multiplier according to binary bits, selecting the corresponding valid bits in the multiplier according to the preset truncation bit width configuration parameters, and detecting the valid bits one by one from right to left in bit order. If the i-th bit is 1, the multiplicand is shifted left by i bits through the shift register to obtain the valid shift result corresponding to the i-th bit; if it is 0, it is skipped until all bit sequences in the valid bits are detected; the valid shift results of all bits of the multiplicand and the truncation error compensation value are added through the accumulator to obtain the final product result; wherein, the truncation error compensation value is calculated based on the average truncation error compensation of offline statistics and the dynamic weight.

11. A cache line replacement device, characterized in that: The device is applied to a chip, the chip includes a data cache, the data cache includes N cache lines of stored data, the device includes a PLRU algorithm implementation module, a PLRU path value calculation module, a replacement score calculation module, a replaceable cache line determination module and a replacement operation module, wherein: The PLRU algorithm implementation module is configured to manage the access path corresponding to each cache line based on a binary tree structure containing N-1 nodes in the pseudo least recently used PLRU algorithm, wherein each node corresponds to a path status bit, and the path status bit is used to dynamically record the access direction preference. ; The PLRU path value calculation module is configured to, in response to a received data replacement instruction, obtain a path status bit from a root node to a leaf node on an access path corresponding to each cache line; and calculate, for a specified cache line, a PLRU path value corresponding to the specified cache line based on the path status bit from the root node to the leaf node on the access path corresponding to the specified cache line; The replacement score calculation module is configured to: obtain an importance level of the data stored in the specified cache line; calculate a replacement score for the specified cache line based on a PLRU path value corresponding to the specified cache line and the importance level of the data stored in the specified cache line, as well as a specified PLRU path value weight and a specified importance level weight; The replaceable cache line determination module is configured to: determine a replaceable cache line in the data cache according to a replacement score of each cache line; The replacement operation module is configured to: obtain the data to be cached and the importance level corresponding to the data to be cached, store the data to be cached and the importance level corresponding to the data to be cached in the replaceable cache line to replace the stored data and the importance level of the stored data in the replaceable cache line.

12. The device according to claim 11, characterized in that The determining of replaceable cache lines in the data cache according to the replacement score of each cache line comprises: Obtain the lowest replacement score among the replacement scores of each cache line; if the number of the lowest replacement scores is one, use the cache line corresponding to the lowest replacement score as a replaceable cache line; if the number of the lowest replacement scores is multiple, find the replaceable cache line in the data cache based on the PLRU algorithm.

13. The device according to claim 12, characterized in that When obtaining the lowest replacement score among the replacement scores of each cache line, it is implemented based on a multi-level comparison tree structure, wherein each level comparator in the multi-level comparison tree structure is a binary comparator, and the binary comparator is used to receive the replacement scores of any two cache lines as input and output the smaller value and its corresponding cache line index; The implementation based on the multi-level comparison tree structure includes: the first-level comparator performs group comparison on all cache lines in pairs to obtain a first-level candidate minimum value set; the subsequent comparators in each level continue to group and compare the candidate minimum values ​​output by the previous level until the cache line index corresponding to the final lowest replacement score is output.

14. The device according to claim 11, characterized in that The device further comprises: The weight dynamic adjustment module is configured to: obtain at least one parameter among the historical hit rate, future predicted hit rate and future predicted access pattern of the data cache; and dynamically adjust the specified PLRU path value weight and the specified importance level weight according to at least one parameter among the historical hit rate, future predicted hit rate and future predicted access pattern of the data cache.

15. A chip, characterized in that: The device comprises the apparatus of any one of claims 11 to 14; or comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any one of claims 1 to 10.

16. An electronic device, characterized in that: Comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1 to 10.

Images