Cache line replacement method and device, chip and electronic equipment

Abstract

The present disclosure relates to the technical field of artificial intelligence chips and storage management, in particular to a cache line replacement method and device, a chip and an electronic equipment. The method is applied to a chip containing a data cache, the data cache contains N cache lines of stored data, a binary tree structure containing N-1 nodes in the PLRU algorithm is used to manage the access path corresponding to each cache line, in response to a data replacement instruction, the path state bits from the root node to the leaf node on the access path corresponding to each cache line are obtained; for a specified cache line, the PLRU path value is calculated according to the path state bits, the importance level and the corresponding weight of the stored data are combined to calculate the replacement score; the replaceable cache line is determined according to the replacement score; the data to be cached and its importance level are stored in the replaceable cache line, and the original data and level are replaced. Thus, the cache line replacement priority is evaluated by combining the PLRU path value and the importance level of the stored data, which effectively improves the cache hit rate and optimizes the system performance.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence chips and storage management technology, specifically to a cache line replacement method, apparatus, chip, and electronic device. Background Technology

[0002] In the current era of rapid development in artificial intelligence technology, the Neural Processing Unit (NPU), as the core hardware for accelerating deep learning tasks, plays a crucial role in the operational efficiency of various intelligent applications. Deep learning tasks are characterized by high parallelism, strong data reusability, and complex access patterns. This requires the NPU to efficiently manage and utilize data when processing these tasks in order to fully leverage its computing power.

[0003] In the architecture of the NPU, the data cache is a key component for improving computing efficiency. The cache sets up a smaller but faster storage tier between the processor and main memory to cache data that the processor frequently accesses, thereby reducing the number of times the processor accesses main memory, reducing access latency, and improving overall computing performance.

[0004] However, traditional cache designs typically employ simple replacement algorithms, such as Least Recently Used (LRU) and First-In-First-Out (FIFO). While these algorithms may be suitable for general computing scenarios, they present numerous challenges under the specific workloads faced by NPUs.

[0005] The core idea of ​​the LRU algorithm is to replace the least recently used data block. When NPUs process deep learning tasks, due to the complexity of access patterns and the high reusability of data, simply relying on recent usage to determine the replacement priority of a data block often fails to accurately predict subsequent data access needs. In deep learning algorithms, data access may exhibit periodic or localized characteristics, but these characteristics cannot be fully encompassed by simple time order. For example, in some neural network layers, specific weight data may be repeatedly used over multiple computation cycles, but this data may not have been frequently accessed recently. According to the LRU algorithm, this important data may be incorrectly replaced from the cache, requiring subsequent accesses to reread from main memory, increasing access latency and reducing cache hit rate.

[0006] The FIFO algorithm replaces data blocks according to the order they enter the cache, with the first block to enter being replaced first. This algorithm completely disregards the frequency and importance of data blocks, making it overly mechanical and indiscriminate in the high parallelism and complex access patterns of NPUs. Data access in deep learning tasks is highly dynamic and uncertain. Relying solely on the order in which data enters the cache to determine replacement cannot adapt to this change. This can lead to the cache retaining a large amount of infrequently used data while the truly needed data is replaced, further reducing cache utilization and hit rate.

[0007] Furthermore, as artificial intelligence applications continue to expand into embedded devices and edge computing scenarios, lightweight NPUs have emerged. These NPUs have strict limitations on power consumption and hardware resources, requiring high-performance computing within limited resources. However, existing simple cache replacement algorithms not only fail to fully utilize cache resources, resulting in low cache hit rates and high access latency, but also consume significant hardware resources and power when implementing these algorithms. For example, the LRU algorithm requires maintaining a complex access history structure to track the usage of each data block, and the Least Frequently Used (LFU) algorithm also needs to statistically record the usage frequency of data blocks. This adds additional overhead to the hardware implementation, including storage space and computing resources. For resource-constrained lightweight NPUs, this additional overhead is unacceptable and severely impacts their performance and energy efficiency.

[0008] In summary, existing cache replacement algorithms have many shortcomings under NPU workloads and cannot meet the NPU's requirements for high performance and low overhead.

[0009] Improving cache hit rate in lightweight NPU application scenarios and enhancing overall system performance is an urgent problem to be solved. Summary of the Invention

[0010] To address the problems in the related technologies, embodiments of this disclosure provide a cache line replacement method, apparatus, chip, and electronic device.

[0011] In a first aspect, this disclosure provides a cache line replacement method applied to a chip. The chip includes a data cache, which comprises N cache lines containing stored data. The method manages the access path corresponding to each cache line based on a binary tree structure with N-1 nodes using a pseudo-Least Recently Used (PLRU) algorithm. Each node corresponds to a path status bit, which is used to dynamically record access direction preferences. The method includes:

[0012] In response to the received data replacement instruction, obtain the path status bit from the root node to the leaf node on the access path corresponding to each cache line;

[0013] For a given cache line, calculate the PLRU path value corresponding to the given cache line based on the path status bits from the root node to the leaf node on the access path corresponding to the given cache line; obtain the importance level of the data already stored in the given cache line; calculate the replacement score of the given cache line based on the PLRU path value corresponding to the given cache line, the importance level of the data already stored in the given cache line, the specified PLRU path value weight, and the specified importance level weight.

[0014] Replaceable cache lines in the data cache are determined based on the replacement score of each cache line;

[0015] Obtain the data to be cached and the importance level corresponding to the data to be cached, and store the data to be cached and the importance level corresponding to the data to be cached in the replaceable cache line to replace the already stored data and the importance level of the already stored data in the replaceable cache line.

[0016] According to embodiments of this disclosure, the management of the access path corresponding to each cache line using a binary tree structure containing N-1 nodes based on the pseudo Least Recently Used (PLRU) algorithm includes:

[0017] After storing the data to be cached and the importance level corresponding to the data to be cached in the replaceable cache line, the path status bits of the nodes in the binary tree structure are updated based on the PLRU algorithm.

[0018] According to embodiments of this disclosure, the cache line includes a data area and a tag area; storing the data to be cached and its corresponding importance level in the replaceable cache line includes:

[0019] The data to be cached is stored in the data area of ​​the replaceable cache line, and the importance level corresponding to the data to be cached is stored in the tag area of ​​the replaceable cache line;

[0020] The step of obtaining the importance level of the data already stored in the specified cache line includes: obtaining the importance level of the data already stored in the specified cache line from the Tag area in the specified cache line.

[0021] According to embodiments of this disclosure, calculating the PLRU path value corresponding to the specified cache line based on the path status bits from the root node to the leaf node on the access path corresponding to the specified cache line includes:

[0022] The PLRU path value corresponding to the specified cache line is obtained by adding the path status bits from the root node to the leaf node on the access path corresponding to the specified cache line.

[0023] According to embodiments of this disclosure, determining replaceable cache lines in the data cache based on the replacement score of each cache line includes:

[0024] Obtain the lowest replacement score among the replacement scores of each cache line; if there is only one lowest replacement score, then the cache line corresponding to the lowest replacement score is taken as a replaceable cache line; if there are multiple lowest replacement scores, then the replaceable cache lines in the data cache are found based on the PLRU algorithm.

[0025] According to an embodiment of this disclosure, when obtaining the lowest replacement score among the replacement scores of each cache line, it is implemented based on a multi-level comparison tree structure, wherein each level comparator in the multi-level comparison tree structure is a binary comparator, which is used to receive the replacement scores of any two cache lines as input and output the smaller value and its corresponding cache line index.

[0026] The implementation based on a multi-level comparison tree structure includes: the first-level comparator performs pairwise comparisons on all cache lines to obtain a first-level candidate minimum value set; each subsequent level comparator continues to group and compare the candidate minimum values ​​output by the previous level until the cache line index corresponding to the final lowest replacement score is output.

[0027] According to embodiments of this disclosure, the method further includes:

[0028] Obtain at least one parameter from the historical hit rate, future predicted hit rate, and future predicted access pattern of the data cache;

[0029] The weight of the specified PLRU path value and the weight of the specified importance level are dynamically adjusted based on at least one of the following parameters: historical hit rate of the data cache, future predicted hit rate, and future predicted access pattern.

[0030] According to embodiments of this disclosure, the replacement score RS for the specified cache line is calculated using the following formula:

[0031] ;

[0032] in, This indicates the PLRU path value corresponding to the specified cache line. This indicates the importance level of the data already stored in the specified cache line. This indicates the weight of the specified PLRU path value. This indicates the weight of the specified importance level. and All are integers not less than 1.

[0033] According to an embodiment of this disclosure, when calculating the replacement score of the specified cache line using the formula, the multiplication operation in the formula is implemented by binary decomposition and weighted accumulation based on a shift-add circuit, and then the product results after the multiplication operation are added together by an adder to obtain the replacement score of the specified cache line.

[0034] According to embodiments of this disclosure, the shift-add circuit includes a shift register and an accumulator; the multiplication operation in the formula based on the shift-add circuit, through binary decomposition and weighted accumulation, includes:

[0035] Determine whether the importance level of the data already stored in the specified cache line is a preset low importance level;

[0036] If not, then the full-precision calculation mode is activated, including: decomposing the multiplier into binary bits, checking each bit in order from right to left, if the i-th bit is 1, then shifting the multiplier left by i bits through a shift register to obtain the effective shift result of the i-th bit; if it is 0, skipping it, until all bits have been checked; and adding the effective shift results of all bits of the multiplier through an accumulator to obtain the final product result.

[0037] If so, the approximate calculation optimization mode is activated, including: decomposing the multiplier into binary bits, selecting the corresponding valid bits in the multiplier according to the preset truncation bit width configuration parameters, and detecting the valid bits bit by bit from right to left. If the i-th bit is 1, the multiplier is shifted left by i bits through a shift register to obtain the corresponding valid shift result of the i-th bit; if it is 0, it is skipped until all bits in the valid bits have been detected; the valid shift results of all bits of the multiplier and the truncation error compensation value are added together through an accumulator to obtain the final product result; wherein, the truncation error compensation value is calculated based on the offline statistical average truncation error compensation and dynamic weight.

[0038] Secondly, this disclosure provides a cache line replacement device applied to a chip. The chip includes a data cache, which includes N cache lines containing stored data. The device includes a PLRU algorithm implementation module, a PLRU path value calculation module, a replacement score calculation module, a replaceable cache line determination module, and a replacement operation module, wherein:

[0039] The PLRU algorithm implementation module is configured to manage the access path corresponding to each cache line based on a binary tree structure containing N-1 nodes in the pseudo-Least Recently Used PLRU algorithm. Each node corresponds to a path status bit, which is used to dynamically record access direction preferences. ;

[0040] The PLRU path value calculation module is configured to, in response to a received data replacement instruction, obtain the path status bit from the root node to the leaf node on the access path corresponding to each cache line; and, for a specified cache line, calculate the PLRU path value corresponding to the specified cache line based on the path status bit from the root node to the leaf node on the access path corresponding to the specified cache line.

[0041] The replacement score calculation module is configured to: obtain the importance level of the data already stored in the specified cache line; calculate the replacement score of the specified cache line based on the PLRU path value corresponding to the specified cache line, the importance level of the data already stored in the specified cache line, the specified PLRU path value weight, and the specified importance level weight;

[0042] The replaceable cache line determination module is configured to: determine replaceable cache lines in the data cache based on the replacement score of each cache line;

[0043] The replacement operation module is configured to: obtain the data to be cached and the importance level corresponding to the data to be cached, and store the data to be cached and the importance level corresponding to the data to be cached in the replaceable cache line to replace the stored data and the importance level of the stored data in the replaceable cache line.

[0044] According to embodiments of this disclosure, determining replaceable cache lines in the data cache based on the replacement score of each cache line includes:

[0045] Obtain the lowest replacement score among the replacement scores of each cache line; if there is only one lowest replacement score, then the cache line corresponding to the lowest replacement score is taken as a replaceable cache line; if there are multiple lowest replacement scores, then the replaceable cache lines in the data cache are found based on the PLRU algorithm.

[0046] According to an embodiment of this disclosure, when obtaining the lowest replacement score among the replacement scores of each cache line, it is implemented based on a multi-level comparison tree structure, wherein each level comparator in the multi-level comparison tree structure is a binary comparator, which is used to receive the replacement scores of any two cache lines as input and output the smaller value and its corresponding cache line index.

[0047] The implementation based on a multi-level comparison tree structure includes: the first-level comparator performs pairwise comparisons on all cache lines to obtain a first-level candidate minimum value set; each subsequent level comparator continues to group and compare the candidate minimum values ​​output by the previous level until the cache line index corresponding to the final lowest replacement score is output.

[0048] According to embodiments of this disclosure, the apparatus further includes:

[0049] The weight dynamic adjustment module is configured to: obtain at least one parameter among the historical hit rate, future predicted hit rate, and future predicted access pattern of the data cache; and dynamically adjust the weight of the specified PLRU path value and the weight of the specified importance level based on at least one parameter among the historical hit rate, future predicted hit rate, and future predicted access pattern of the data cache.

[0050] Thirdly, this disclosure provides a chip comprising the apparatus of any one of the second aspects; or comprising a memory and a processor; wherein the memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method of any one of the first aspects.

[0051] Fourthly, this disclosure provides an electronic device including a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method described in any one of the first aspects.

[0052] According to the technical solution provided in the embodiments of this disclosure, a high-performance cache line replacement method based on multi-factor evaluation is proposed. This method is applied to a chip, which includes a data cache comprising N cache lines containing stored data. Based on the PLRU algorithm, a binary tree structure with N-1 nodes is used to manage the access path corresponding to each cache line. Each node corresponds to a path status bit, which dynamically records access direction preferences. In response to a received data replacement instruction, the method obtains the path status bit from the root node to the leaf node on the access path corresponding to each cache line. For a specified cache line, the method calculates the PLRU path value corresponding to the specified cache line based on the path status bit from the root node to the leaf node on the access path corresponding to the specified cache line. The method obtains the importance level of the data already stored in the specified cache line. Based on the PLRU path value, the importance level of the data already stored in the specified cache line, and the specified PLRU path value weight and the specified importance level weight, the method calculates the replacement score of the specified cache line. Based on the replacement score of each cache line, the method determines replaceable cache lines in the data cache. The method stores the data to be cached and its corresponding importance level in the replaceable cache line to replace the already stored data and its importance level in the replaceable cache line. Therefore, by combining the PLRU path value and the importance level of the stored data to evaluate the cache line replacement priority, the cache hit rate is effectively improved and the system performance is optimized.

[0053] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0054] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0055] Figure 1 A flowchart illustrating a cache line replacement method according to an embodiment of the present disclosure is shown;

[0056] Figure 2 This diagram illustrates the correspondence between the binary tree structure containing N-1 nodes and the N cache lines in the PLRU algorithm.

[0057] Figure 3 This shows the determined path in the PLRU algorithm that can replace a cache line when the initial path status bit of all nodes is 0;

[0058] Figure 4 This shows the changes in the status bits of each node and the determination path of the next replaceable cache line in the PLRU algorithm after the first cache line is replaced.

[0059] Figure 5 The flowchart illustrates the method for determining replaceable cache lines based on the replacement score of each cache line.

[0060] Figure 6 This illustrates a determined path for selecting replaceable cache lines based on a replacement score, according to an embodiment of the present disclosure.

[0061] Figure 7 It shows that Figure 6 Taking the status bits of each node shown as an example, the changes in the status bits of each node after replacing the sixth cache line as a replaceable cache line are shown.

[0062] Figure 8 A flowchart illustrating another cache line replacement method according to an embodiment of this disclosure is shown;

[0063] Figure 9 A structural block diagram of a cache line replacement apparatus according to an embodiment of the present disclosure is shown;

[0064] Figure 10 A specific structural example of a cache line replacement apparatus according to an embodiment of the present disclosure is shown;

[0065] Figure 11 A structural block diagram of another cache line replacement apparatus according to an embodiment of the present disclosure is shown;

[0066] Figure 12A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0067] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of exemplary embodiments have been omitted from the drawings.

[0068] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0069] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0070] As mentioned above, traditional replacement algorithms (such as FIFO) cannot adapt to the complex data access patterns of NPUs, resulting in low cache hit rates. In addition, complex cache replacement algorithms such as LRU and LFU (Least Frequently Used) require large hardware resources, which are not suitable for lightweight NPUs. Furthermore, the cache in NPUs currently generally only supports one replacement algorithm, such as PLRU (Pseudo Least Recently Used). Although this algorithm has lower hardware overhead than LRU, it has poor dynamic adaptability and limited accuracy.

[0071] To address the aforementioned problems in the prior art, the inventors of this disclosure have conducted in-depth research on current cache replacement algorithms and innovatively proposed a high-performance cache line replacement method based on multi-factor evaluation. This method introduces a multi-factor dynamic evaluation mechanism on the basis of the traditional PLRU algorithm. By combining the PLRU path value corresponding to the cache line, the importance level of the data already stored in the cache line, and their respective weights, the replacement score of each cache line is calculated, and then the replaceable cache line is determined based on the replacement score of each cache line. The PLRU path value is a novel concept proposed in this disclosure, reflecting the access time locality of cache lines, namely: the access frequency of cache lines and the priority of access paths (the smaller the path value, the longer it has not been accessed and the easier it is to be replaced). The importance level is the business value of data blocks marked by software or hardware (such as weight data in AI models marked as key), which can reflect the importance of data. Existing PLRU algorithms usually only rely on binary tree path state bits to determine the cache lines to be replaced. However, the scheme in this disclosure introduces PLRU path value, data importance level and their respective dynamic weights, and then obtains a replacement score based on these parameters. Thus, the value of each cache line can be more comprehensively evaluated through quantitative scoring, thereby more accurately selecting truly "unimportant" and "infrequently used" cache lines for replacement. This avoids the unreasonable replacement caused by the qualitative judgment method of traditional PLRU that only relies on binary tree path state bits, reduces the situation of useful data being mistakenly replaced, and thus improves the cache hit rate. Moreover, while significantly improving the cache hit rate (compared to PLRU), the hardware overhead is also lower than that of the LRU algorithm.

[0072] Figure 1 A flowchart illustrating a cache line replacement method according to an embodiment of this disclosure is shown. The method is applied to a chip including a data cache comprising N cache lines containing stored data. The method manages the access path corresponding to each cache line based on a binary tree structure with N-1 nodes using a pseudo-Least Recently Used (PLRU) algorithm. Each node corresponds to a path status bit, which is used to dynamically record access direction preferences. .

[0073] In a specific example of this disclosure, the data cache includes a cache memory. The data mapping between the cache and main memory uses a set-associative mapping, that is, the cache is divided into multiple sets, each set contains N cache lines (i.e., cache lines). Data blocks in main memory are first mapped to which set by the set index, and then a fully associative method is used within that set, that is, the data block can be stored in any cache line within the set.

[0074] The cache line comprises a data area and a tag area. The data area stores the actual data read from main memory. The tag area stores metadata related to the cache line, including address information, validity bits, and dirty bits, used to manage and identify the cache line and ensure that the data in the cache line is correct and up-to-date. For N cache lines containing stored data, the stored data is located within the data area of ​​the cache line.

[0075] In the PLRU algorithm, a binary tree structure is used to efficiently manage the access status of cache lines. Its core function is to approximate LRU behavior through the bitmasks (status bits) of the nodes. For an N-way set-associative cache, N-1 binary status bits (nodes) are required, with each node corresponding to a hardware register or SRAM bit.

[0076] Figure 2 This diagram illustrates the correspondence between a binary tree structure with N-1 nodes and N cache lines in the PLRU algorithm. Taking a data cache containing 8 cache lines (the first to the eighth cache lines) as an example... Figure 2 As shown, the binary tree structure contains 7 nodes (Bit0~Bit6). The root node (Bit0) is located at the top level of the binary tree and determines the global replacement direction. The middle nodes (Bit1, Bit2) make branch decisions, pointing to subtrees. For example, Bit1 points to the left subtree (Bit1 branch), and Bit2 points to the right subtree (Bit2 branch). The leaf nodes are directly associated with specific cache lines. For example, Bit3 is associated with the first and second cache lines, Bit4 with the third and fourth cache lines, Bit5 with the fifth and sixth cache lines, and Bit6 with the seventh and eighth cache lines. The path from the root node to the leaf node in this binary tree maps to each cache line. For example, the access path for the first cache line is Bit0→Bit1→Bit3, and the access path for the sixth cache line is Bit0→Bit2→Bit5.

[0077] In this context, the path status bit (0 or 1) of each node indicates the priority replacement direction. In a specific example, 0 indicates priority replacement of the left subtree, and 1 indicates priority replacement of the right subtree. For example, if Bit0=0, the left subtree (Bit1 branch) is replaced first; if Bit0=1, the right subtree (Bit2 branch) is replaced first.

[0078] Figure 3 This illustrates the determined path in the PLRU algorithm where a cache line can be replaced when all nodes' initial path status bits are 0. For example... Figure 3As shown, in the initial state, when all nodes' initial path state bits are 0, according to the traditional PLRU algorithm rules, specifically as shown in the red path, the first cache line will eventually be replaced, for example, by storing data Data0.

[0079] Figure 4 This illustrates the changes in the status bits of each node and the determination path of the next replaceable cache line in the PLRU algorithm after the first cache line is replaced. Figure 4 As shown, when the first cache line is replaced, backtrack from the leaf node to the root node, flip all the state bits on the path, and set the state of the node on the access path to the non-current direction (i.e., the other side will be replaced first next time). That is, set Bit0, Bit1 and Bit3 to 1. In this way, according to the traditional PLRU algorithm rules, as shown in the red path, the next replaceable cache line is the fifth cache line.

[0080] Depend on Figure 3 and Figure 4 As shown, in the traditional PLRU algorithm, the direction preference (0 / 1) of the path status bit directly determines the replacement path, and the candidate line pointed to by the combinational logic of the path status bit is the replaceable cache line.

[0081] The method for determining replaceable cache lines in this disclosure is fundamentally different from the traditional PLRU algorithm described above. For example... Figure 1 As shown, the cache line replacement method disclosed herein includes the following steps S110~S140:

[0082] In step S110, in response to the received data replacement instruction, the path status bit from the root node to the leaf node on the access path corresponding to each cache line is obtained.

[0083] The data replacement instruction is typically triggered when a cache miss occurs (e.g., write miss or read miss) and there are no free cache lines (cache is full). When the processor needs to write new data but a write miss occurs, the data to be cached includes the data to be written. When the processor reads data but a read miss occurs, new data needs to be loaded from main memory, and the data to be cached includes the data to be loaded. After the data replacement instruction is triggered, the cache line replacement method disclosed herein is executed.

[0084] Unlike the traditional PLRU algorithm, which selects replaceable cache lines based on a combination of path status bits (0 / 1), the cache line replacement method in this disclosure first requires obtaining the path status bits from the root node to the leaf node on the access path corresponding to each cache line. For example: Figure 4Taking the status bits of each node as an example, the path status bits from the root node to the leaf node on the access paths corresponding to the first and second cache lines are all 1 (Bit0), 1 (Bit1), and 1 (Bit3). The path status bits from the root node to the leaf node on the access paths corresponding to the third and fourth cache lines are all 1 (Bit0), 1 (Bit1), and 0 (Bit4). The path status bits from the root node to the leaf node on the access paths corresponding to the fifth and sixth cache lines are all 1 (Bit0), 0 (Bit2), and 0 (Bit5). The path status bits from the root node to the leaf node on the access paths corresponding to the seventh and eighth cache lines are all 1 (Bit0), 0 (Bit2), and 0 (Bit6).

[0085] In step S120, for a specified cache line, the PLRU path value corresponding to the specified cache line is calculated based on the path status bits from the root node to the leaf node on the access path corresponding to the specified cache line; the importance level of the data already stored in the specified cache line is obtained; and the replacement score of the specified cache line is calculated based on the PLRU path value corresponding to the specified cache line, the importance level of the data already stored in the specified cache line, the specified PLRU path value weight, and the specified importance level weight.

[0086] According to embodiments of this disclosure, calculating the PLRU path value corresponding to the specified cache line based on the path status bits from the root node to the leaf node on the access path corresponding to the specified cache line includes:

[0087] The PLRU path value corresponding to the specified cache line is obtained by adding the path status bits from the root node to the leaf node on the access path corresponding to the specified cache line.

[0088] return Figure 4 Taking the status bit situation of each node as an example, for each cache line from the first cache line to the eighth cache line, after obtaining the path status bit from the root node to the leaf node on the corresponding access path, the PLRU path value corresponding to the first cache line and the second cache line is 3, the PLRU path value corresponding to the third cache line and the fourth cache line is 2, and the PLRU path value corresponding to the fifth cache line to the eighth cache line is 1.

[0089] In this disclosure, the fundamental principle behind introducing the PLRU path value concept is to transform the discrete logic of the PLRU binary tree state bits into a continuous priority score. By quantifying access history, "cold data" can be identified more accurately, thereby improving the hit rate. Specifically, the PLRU path value reflects the access frequency of cache lines and the priority of access paths, thus indirectly reflecting the locality of access to data. Using the PLRU path value as a factor in calculating the replacement score allows for the scientific use of the principle of locality of access to assess the importance of cache lines, aligning with the inherent laws of program operation. The smaller the PLRU path value, the lower the priority of the cache line in the access path, meaning it has not been accessed for a longer period; the larger the PLRU path value, the higher the priority of the cache line in the access path, meaning it is more likely to be accessed recently. This quantification method provides an objective and quantifiable indicator for replacement scoring, making replacement decisions more scientific and accurate.

[0090] In the traditional PLRU algorithm, a replacement path is directly selected based on the current value of the binary tree state bits to determine replaceable cache lines. This method only reflects the direction of the most recent access and may ignore historical access patterns. In this disclosure, the PLRU path value, obtained by adding the current values ​​of the binary tree state bits on the access path corresponding to each cache line, records the cache line's access history information, reflecting long-term access tendencies. It considers not only the most recent access but also previous access history, providing a more comprehensive reflection of cache line usage. This comprehensive consideration of access history allows the PLRU path value to better reflect the actual value of the data, providing a deep and broad evaluation basis for replacement scoring. Compared to algorithms that only consider the most recent access, evaluation based on the PLRU path value can more scientifically avoid misjudging the importance of cache lines due to accidental access. Furthermore, when frequently alternating accesses to certain cache lines, the state bits of the traditional PLRU algorithm will continuously flip, causing replacement targets to fluctuate. In this case, the advantage of adding path values ​​is that it smooths out short-term fluctuations by adding the values, prioritizing the replacement of cache lines that have not been given much attention for a long time (such as cache lines with consistently the lowest path values). Furthermore, the goal of cache replacement is to retain as much data as possible that may be accessed again in the future within a limited cache space, thereby improving cache hit rate. The PLRU path value, as an indicator reflecting the access priority of cache lines, is highly consistent with the cache replacement goal. By incorporating the PLRU path value into the replacement score calculation, replacement decisions can be made more aligned with the cache replacement objective: prioritizing the replacement of data with low access priority and that has not been accessed for a long time, thereby improving cache space utilization and cache hit rate.

[0091] While the PLRU path value is an important evaluation metric, this disclosure does not use it as the sole basis for decision-making. Instead, it combines it with other factors (such as the importance level of the data) for a comprehensive evaluation. This comprehensive evaluation method fully utilizes the advantages of various factors and avoids decision-making errors caused by a single factor. The PLRU path value plays a role in reflecting access history and access priority in the comprehensive evaluation, complementing other factors to make the replacement score more comprehensive and accurate, thereby improving the rationality of cache replacement decisions.

[0092] In this disclosure, importance levels can be represented using 2 bits or more. When represented using 2 bits or more (e.g., 3 bits), more granular importance levels are supported (e.g., weight differences between different layers in an AI model). The number of bits used can be selected according to the specific application scenario. Taking 2 bits as an example, 4 importance levels are supported, which can be set as follows: 00 represents low importance, 01 represents medium importance, 10 represents high importance, and 11 represents critical data blocks.

[0093] In addition, there are several ways to store the importance level of data already stored in the cache line. For example, a dedicated area for storing importance levels can be set up in the cache line structure (in parallel with the Tag area and Data area), or a separate mapping table can be maintained in the cache controller to query the corresponding importance level through the cache line index, or the importance level can be encoded into the header of the data block or ECC check bits (such as using the redundancy bits of ECC).

[0094] In a specific example, this disclosure stores the importance level of the data already stored in the specified cache line in the free or reserved space of the Tag area of ​​the specified cache line. In this way, when obtaining the importance level of the data already stored in the specified cache line, the importance level of the data already stored in the specified cache line can be obtained from the Tag area of ​​the specified cache line.

[0095] Storing importance levels in the Tag area of ​​cache lines has the following advantages compared to other storage methods mentioned above:

[0096] 1. Saves hardware costs as no additional storage units are required. Importance levels are stored in the Tag area, eliminating the need to allocate new storage space on the chip to store them separately. Utilizing available or reserved bits for importance level storage avoids additional storage unit costs.

[0097] 2. Reduced control circuit design. If a separate memory structure is used to store importance levels, additional control circuitry is required to manage read and write operations on that structure, including address decoding and data transfer. Storing importance levels in the Tag area allows for management using existing cache control circuitry, eliminating the need for complex additional control circuitry and further reducing hardware costs.

[0098] 3. Simplified program processing flow. When importance levels are stored in the Tag area, the program can directly access the corresponding importance level while accessing cached line data, without requiring additional access steps. For example, when reading cached line data, the program can retrieve the data and its corresponding importance level from the cached line at once, without first accessing the cached line to obtain the data and then accessing another independent storage structure to obtain the importance level, thus simplifying the data access process.

[0099] Compared to other methods, while storing data in conjunction with its importance level doesn't require additional storage, it increases the complexity of data processing. For example, when reading data, it's necessary to first parse the high or low bits to obtain the importance level before proceeding with normal data processing. This increases the number of program instructions and execution time. Furthermore, if the data format changes, the program may need to be modified to adapt to the new importance level storage method, increasing maintenance costs. Using a separate storage structure increases hardware costs and complicates the program's processing flow. The program requires additional instructions to access the independent storage structure and handles the mapping between data and importance levels, increasing complexity and the likelihood of errors.

[0100] After obtaining the PLRU path value corresponding to the specified cache line and the importance level of the data already stored in the specified cache line, the replacement score of the specified cache line is calculated according to the PLRU path value corresponding to the specified cache line, the importance level of the data already stored in the specified cache line, the specified PLRU path value weight, and the specified importance level weight, using the following formula. :

[0101] ;

[0102] in, This indicates the PLRU path value corresponding to the specified cache line. This indicates the importance level of the data already stored in the specified cache line. This indicates the weight of the specified PLRU path value. This indicates the weight of the specified importance level. and All are integers not less than 1.

[0103] Specifically, the PLRU path value weight and importance level weight can be specified by an external program, and for different levels of cache (such as L1 / L2 / L3 cache), their respective PLRU path value weight and importance level weight can be configured independently.

[0104] return Figure 4 Taking the status bits of each node shown as an example, let's assume... and If each is 1, then the PLRU path value corresponding to each cache line, as well as the importance level and replacement score RS of the stored data, are shown in Table 1 below.

[0105] Table 1. PLRU path values ​​for each cache line, along with the importance level and replacement score (RS) of the stored data, as a first reference table.

[0106]

[0107] According to an embodiment of this disclosure, when calculating the replacement score of the specified cache line using the formula, the multiplication operation in the formula is implemented by binary decomposition and weighted accumulation based on a shift-add circuit, and then the product results after the multiplication operation are added together by an adder to obtain the replacement score of the specified cache line.

[0108] Unlike setting the weights to decimals, in this disclosure, the weights are... and All values ​​are set to integers not less than 1, and weighted multiplication is implemented through shift and addition operations. The advantages of this approach are: firstly, it simplifies hardware implementation; for example, if... When the value is 0.3, it needs to be converted to a fixed-point number (such as Q1.15 format). Compared to... When the multiplier value is 3, the multiplier area needs to be increased by 3-5 times; on the other hand, it improves energy efficiency, respectively, for calculating 0. and For example, if the former hardware unit uses a 16-bit fixed-point multiplier, the number of gates is approximately 200, and the dynamic power consumption is approximately 2mW / MHz. In contrast, the latter hardware unit uses a shifter and adder, reducing the number of gates to approximately 30, with a dynamic power consumption of only approximately 0.5mW / MHz. Furthermore, integer multiplication and addition can be completed in a single cycle, while fractional multiplication requires multiple pipeline stages (e.g., a 3-cycle delay). Thus, under the same process technology, the integer scheme can increase the clock frequency by 10-15%, and integer operations can avoid floating-point truncation errors, ensuring lossless precision.

[0109] The shift-add circuit includes a shift register and an accumulator; the multiplication operation in the formula based on the shift-add circuit, through binary decomposition and weighted accumulation, includes:

[0110] Determine whether the importance level of the data already stored in the specified cache line is a preset low importance level. For example, when the importance level parameter is represented by 2 bits, the preset low importance level can be set to 00.

[0111] If not, then the full-precision calculation mode is activated, which includes: decomposing the multiplier into binary bits, checking each bit in order from right to left, if the i-th bit is 1, then shifting the multiplier left by i bits through the shift register to obtain the effective shift result of the i-th bit; if it is 0, then skipping, until all bits have been checked; and adding the effective shift results of all bits of the multiplier through the accumulator to obtain the final product result.

[0112] If so, the approximate calculation optimization mode is activated, including: decomposing the multiplier into binary bits, selecting the corresponding valid bits in the multiplier according to the preset truncation bit width configuration parameters, and checking the valid bits bit by bit from right to left. If the i-th bit is 1, the multiplier is shifted left by i bits through a shift register to obtain the valid shift result of the i-th bit; if it is 0, it is skipped until all bits in the valid bits have been checked; the valid shift results of all bits of the multiplier and the truncation error compensation value are added through an accumulator to obtain the final product result; wherein, the truncation error compensation value is calculated based on the offline statistical average truncation error compensation and dynamic weight. .

[0113] This disclosure provides two calculation modes for cache line replacement scoring: full-precision calculation mode for high-importance data, and full-precision calculation mode for low-importance data (such as...). (=00) Enables approximate computation, reducing power consumption by truncating low-order bits during accumulation, while maintaining system performance stability through error compensation. This solution significantly reduces power consumption while ensuring system performance through a collaborative design of hardware dynamic truncation and software error compensation, making it particularly suitable for importance-based cache management in edge AI devices.

[0114] In step S130, replaceable cache lines in the data cache are determined based on the replacement score of each cache line.

[0115] like Figure 5 As shown, determining the replaceable cache lines in the data cache based on the replacement score of each cache line includes the following steps S131~S134:

[0116] In step S131, the lowest replacement score in the replacement scores of each cache line is obtained.

[0117] In step S132, it is determined whether the number of the lowest replacement scores is one.

[0118] If the number of minimum replacement scores is one, then proceed to step S133; if the number of minimum replacement scores is multiple, then proceed to step S134.

[0119] In step S133, the cache line corresponding to the lowest replacement score is taken as a replaceable cache line.

[0120] In step S134, replaceable cache lines in the data cache are found based on the PLRU algorithm.

[0121] return Figure 4 Taking the status bits of each node as an example, Figure 6 This illustrates a determined path for selecting replaceable cache lines based on a replacement score, according to an embodiment of this disclosure. For example... Figure 6 As shown in the red path, for the results of each RS calculated in Table 1 above, if the number of lowest replacement scores is one, then the sixth cache line with the lowest replacement score will be selected as the replaceable cache line. Combined with... Figure 4 Based on the traditional PLRU algorithm, which considers the "fifth cache line" as a replaceable cache line, it's known that although the PLRU path values ​​corresponding to the fifth and sixth cache lines are the same (both are 1), the sixth cache line is prioritized for replacement because the data already stored in the fifth cache line is of higher importance than that in the sixth cache line. Thus, by determining replaceable cache lines according to the cache line replacement scheme of this disclosure, the possibility of important data being mistakenly replaced is reduced, thereby improving the cache hit rate.

[0122] This disclosure also considers the case where there are multiple minimum replacement scores. In this case, the processing method is to fall back to the traditional PLRU algorithm, that is, to find the replaceable cache line in the data cache based on the PLRU algorithm.

[0123] Figure 7 It shows that Figure 6 Taking the status bits of each node shown as an example, let's examine the changes in the status bits of each node after replacing the sixth cache line as a replaceable cache line. Assume the stored data... If the value is 0 or 1, then the PLRU path value corresponding to each cache line, as well as the importance level and replacement score (RS) of the stored data, are shown in Table 2. Based on the calculated RS results in Table 2, there are two cache lines with a minimum RS of 2, which meets the condition that there are multiple minimum replacement scores. In this case, the replaceable cache line in the data cache is found based on the PLRU algorithm. The next replaceable cache line is then the third cache line, as shown below. Figure 7 As shown by the red path in the middle.

[0124] Table 2. PLRU path values ​​for each cache line, along with the importance level and replacement score (RS) of the stored data, as a second reference table.

[0125]

[0126] According to embodiments of this disclosure, when obtaining the lowest replacement score among the replacement scores of each cache line, a multi-level comparison tree structure is used, wherein each level comparator in the multi-level comparison tree structure is a binary comparator, which is used to receive the replacement scores of any two cache lines as input and output the smaller value and its corresponding cache line index.

[0127] The implementation based on a multi-level comparison tree structure includes: the first-level comparator performs pairwise comparisons on all cache lines to obtain a first-level candidate minimum value set; each subsequent level comparator continues to group and compare the candidate minimum values ​​output by the previous level until the cache line index corresponding to the final lowest replacement score is output.

[0128] In step S140, the data to be cached and the importance level corresponding to the data to be cached are obtained, and the data to be cached and the importance level corresponding to the data to be cached are stored in the replaceable cache line to replace the stored data and the importance level of the stored data in the replaceable cache line.

[0129] The data to be cached can be obtained from the CPU or main memory, and the importance level corresponding to the data to be cached can be specified by software or hardware.

[0130] According to embodiments of this disclosure, storing the data to be cached and its corresponding importance level in the replaceable cache line includes:

[0131] The data to be cached is stored in the data area of ​​the replaceable cache line, and the importance level corresponding to the data to be cached is stored in the tag area of ​​the replaceable cache line. For example, it is stored in the free space or reserved space of the tag area of ​​the replaceable cache line.

[0132] In addition, after storing the data to be cached and the importance level corresponding to the data to be cached in the replaceable cache line, the path status bits of the nodes in the binary tree structure are updated based on the PLRU algorithm.

[0133] by Figure 6 Taking the status bits of each node as an example, after replacing the sixth cache line with the data to be cached and its corresponding importance level as a replaceable cache line, the status bits of the access path corresponding to the sixth cache line from the leaf node to the root node are flipped and set to the non-current direction, that is: Bit5 is set to 1, Bit2 is set to 1, and Bit0 is set to 0. When triggered again, steps S110~S140 are repeated to perform the next cache line replacement.

[0134] Figure 8 A flowchart illustrating another cache line replacement method according to an embodiment of this disclosure is shown. Figure 8 In addition to steps S110~S140 as described above, the method also includes the following step S150:

[0135] In step S150, at least one parameter among the historical hit rate, future predicted hit rate, and future predicted access pattern of the data cache is obtained; the weight of the specified PLRU path value and the weight of the specified importance level are dynamically adjusted according to at least one parameter among the historical hit rate, future predicted hit rate, and future predicted access pattern of the data cache.

[0136] The data cache uses two counters to record the number of cache hits and the total number of accesses, respectively. The ratio of the number of cache hits to the total number of accesses is the cache hit rate. The historical hit rate refers to the cache hit rate over a certain period of time or during certain accesses in the past, such as the recent 1K / 10K / 100K accesses.

[0137] Future hit rate prediction refers to predicting the cache hit rate within a future time period based on current access patterns and historical hit rate data. A hit rate prediction model can be built using machine learning algorithms (such as linear regression, time series analysis, etc.) or simple statistical methods. Historical hit rates and access patterns are used as input data to train the prediction model (such as a lightweight LSTM). Then, the trained model is used to predict the hit rate within a future time period based on current access patterns and historical hit rate data. The predicted hit rate is stored in a variable or data structure for later use.

[0138] Access patterns refer to the characteristic description of access behavior in a cache or storage system. They reflect the regularity and predictability of data access, including but not limited to access frequency (the number of times a certain data or address is accessed), access time interval (the time interval between two consecutive accesses), access type (the type of access, such as read or write), and access range (the access range can reflect the scope of data usage; a larger access range may indicate the importance and complexity of the data).

[0139] When dynamically adjusting the specified PLRU path value weight and the specified importance level weight based on at least one parameter among the historical hit rate, future predicted hit rate, and future predicted access pattern of the data cache, adjustment strategies and rules can be defined according to specific application scenarios and actual needs. For example: if the historical hit rate and / or future predicted hit rate are higher than a preset first reference threshold, it indicates that the current cache management strategy is relatively effective, and the current PLRU path value weight and importance level weight do not need to be adjusted; if the historical hit rate and / or future predicted hit rate are lower than a preset second reference threshold, it indicates that the current cache management strategy may need to be adjusted, and the PLRU path value weight can be appropriately increased while the importance level weight is decreased; if the future predicted access pattern shows that some data will be frequently accessed, these data are of high importance, so the importance level weight of these data can be appropriately increased to ensure that these data will not be replaced; if the future predicted access pattern shows that some data will be rarely accessed, these data are of low importance, so the importance level weight of these data can be appropriately decreased to avoid retaining unnecessary data and improve cache utilization.

[0140] This disclosure, by comprehensively considering historical hit rate, future predicted hit rate, and future predicted access patterns, enables the weight adjustment rules to dynamically adapt to different operating environments and data access patterns. This dynamic adjustment mechanism allows the cache management strategy to be optimized according to actual conditions, thereby improving cache hit rate and system performance. Furthermore, the PLRU path value primarily reflects the locality of access, while the importance level primarily reflects the importance of data. By dynamically adjusting the weights of these two factors, a balance can be achieved between locality and importance, making the cache management strategy more comprehensive and effective. Moreover, the introduction of future predicted hit rate and access patterns gives the cache management strategy a certain predictive capability, allowing it to adapt to future changes in advance, thereby improving cache adaptability and performance.

[0141] Figure 9 A structural block diagram of a cache line replacement apparatus according to an embodiment of the present disclosure is shown. The apparatus is applied to a chip including a data cache comprising N cache lines containing stored data. The cache line replacement apparatus includes a PLRU algorithm implementation module, a PLRU path value calculation module, a replacement score calculation module, a replaceable cache line determination module, and a replacement operation module. Figure 9 As shown, where:

[0142] The PLRU algorithm implementation module is configured to manage the access path corresponding to each cache line based on a binary tree structure containing N-1 nodes in the pseudo-Least Recently Used PLRU algorithm. Each node corresponds to a path status bit, which is used to dynamically record access direction preferences. The PLRU path value calculation module is configured to, in response to a received data replacement instruction, obtain the path status bit from the root node to the leaf node on the access path corresponding to each cache line; for a specified cache line, calculate the PLRU path value corresponding to the specified cache line based on the path status bit from the root node to the leaf node on the access path corresponding to the specified cache line; the replacement score calculation module is configured to: obtain the importance level of the data already stored in the specified cache line; calculate the replacement score of the specified cache line based on the PLRU path value corresponding to the specified cache line, the importance level of the data already stored in the specified cache line, and the specified PLRU path value weight and the specified importance level weight; the replaceable cache line determination module is configured to: determine the replaceable cache line in the data cache based on the replacement score of each cache line; the replacement operation module is configured to: obtain the data to be cached and the importance level corresponding to the data to be cached, store the data to be cached and the importance level corresponding to the data to be cached in the replaceable cache line, so as to replace the data already stored in the replaceable cache line and the importance level of the data already stored.

[0143] According to embodiments of this disclosure, the replacement score of the specified cache line is calculated using the following formula. :

[0144] ;

[0145] in, This indicates the PLRU path value corresponding to the specified cache line. This indicates the importance level of the data already stored in the specified cache line. This indicates the weight of the specified PLRU path value. This indicates the weight of the specified importance level. and All are integers not less than 1.

[0146] According to an embodiment of this disclosure, when calculating the replacement score of the specified cache line using the formula, the multiplication operation in the formula is implemented by binary decomposition and weighted accumulation based on a shift-add circuit, and then the product results after the multiplication operation are added together by an adder to obtain the replacement score of the specified cache line.

[0147] According to embodiments of this disclosure, the shift-add circuit includes a shift register and an accumulator; the multiplication operation in the formula based on the shift-add circuit, through binary decomposition and weighted accumulation, includes:

[0148] Determine whether the importance level of the data already stored in the specified cache line is a preset low importance level.

[0149] If not, then the full-precision calculation mode is activated, which includes: decomposing the multiplier into binary bits, checking each bit in order from right to left, if the i-th bit is 1, then shifting the multiplier left by i bits through the shift register to obtain the effective shift result of the i-th bit; if it is 0, then skipping, until all bits have been checked; and adding the effective shift results of all bits of the multiplier through the accumulator to obtain the final product result.

[0150] If so, the approximate calculation optimization mode is activated, including: decomposing the multiplier into binary bits, selecting the corresponding valid bits in the multiplier according to the preset truncation bit width configuration parameters, and checking the valid bits bit by bit from right to left. If the i-th bit is 1, the multiplier is shifted left by i bits through a shift register to obtain the valid shift result of the i-th bit; if it is 0, it is skipped until all bits in the valid bits have been checked; the valid shift results of all bits of the multiplier and the truncation error compensation value are added through an accumulator to obtain the final product result; wherein, the truncation error compensation value is calculated based on the offline statistical average truncation error compensation and dynamic weight. .

[0151] According to embodiments of this disclosure, determining replaceable cache lines in the data cache based on the replacement score of each cache line includes:

[0152] Obtain the lowest replacement score among the replacement scores of each cache line; if there is only one lowest replacement score, then the cache line corresponding to the lowest replacement score is taken as a replaceable cache line; if there are multiple lowest replacement scores, then the replaceable cache lines in the data cache are found based on the PLRU algorithm.

[0153] According to embodiments of this disclosure, when obtaining the lowest replacement score among the replacement scores of each cache line, a multi-level comparison tree structure is used. Each level comparator in the multi-level comparison tree structure is a binary comparator, which receives the replacement scores of any two cache lines as input and outputs the smaller value and its corresponding cache line index. The implementation based on the multi-level comparison tree structure includes: a first-level comparator performing pairwise comparisons on all cache lines to obtain a first-level candidate minimum value set; subsequent comparators continue to group and compare the candidate minimum values ​​output by the previous level until the cache line index corresponding to the final lowest replacement score is output, at which point the cache line index can be replaced; when the comparator identifies multiple lowest replacement scores, it outputs a PLRU_en signal to the PLRU algorithm implementation module, enabling the PLRU algorithm implementation module to select the final replaceable cache line based on the PLRU algorithm.

[0154] Figure 10This illustration shows a specific structural example of a cache line replacement device according to an embodiment of the present disclosure. Taking a data cache containing eight cache lines as an example, each module in the cache line replacement device is implemented using hardware circuitry. The PLRU algorithm implementation circuit corresponds to the PLRU algorithm implementation module, the PLRU path value calculation module is implemented using an adder, the replacement score calculation module includes a shift-add circuit and an adder, and the replaceable cache line determination module is implemented using a comparison tree structure circuit. For example... Figure 10 As shown, the PLRU algorithm implementation circuit outputs the path status bit (Path_status) from the root node to the leaf node on the access path corresponding to each cache line to the adder. The adder adds the path status bits of the same cache line to obtain the PLRU path value corresponding to each cache line (e.g., the PLRU path value corresponding to the first cache line is Line1_path_value). The adder then outputs the PLRU path value corresponding to each cache line to the shift-add circuit. The shift-add circuit calculates the importance level of the data already stored in each cache line based on the received PLRU path value, the importance level of the data already stored in each cache line (e.g., the importance level of the data already stored in the first cache line is Line1_importance), the specified PLRU path value weight w1, and the specified importance, etc. The replacement score for each cache line is calculated using a weight w2 (e.g., the replacement score for the first cache line is Line1_RS). The replacement score for each cache line is then output to the comparison tree structure circuit. The comparison tree structure circuit obtains the lowest replacement score among the replacement scores of each cache line. If there is only one lowest replacement score, the cache line corresponding to the lowest replacement score is selected as a replaceable cache line, and the cache line index corresponding to the lowest replacement score is output. If there are multiple lowest replacement scores, the PLRU_en signal is output to the PLRU algorithm implementation circuit, enabling the PLRU algorithm implementation circuit to select the final replaceable cache line based on the PLRU algorithm. Finally, the PLRU algorithm implementation circuit outputs the cache line index corresponding to the lowest replacement score.

[0155] Figure 11 A structural block diagram of another cache line replacement apparatus according to an embodiment of the present disclosure is shown. Figure 11 As shown, except Figure 10 In addition to the modules shown, according to embodiments of this disclosure, the cache line replacement device further includes: a weight dynamic adjustment module, configured to: obtain at least one parameter among the historical hit rate, future predicted hit rate, and future predicted access pattern of the data cache; and dynamically adjust the specified PLRU path value weight and the specified importance level weight based on at least one parameter among the historical hit rate, future predicted hit rate, and future predicted access pattern of the data cache.

[0156] This disclosure also provides a chip including the apparatus as described in any of the above apparatus embodiments; or, including a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method described in any of the above method embodiments.

[0157] Figure 12 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Figure 12 As shown, the electronic device includes a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method described in any of the above method embodiments.

[0158] According to the technical solution provided in the embodiments of this disclosure, a high-performance cache line replacement method based on multi-factor evaluation is proposed. This method is applied to a chip, which includes a data cache comprising N cache lines containing stored data. Based on the PLRU algorithm, a binary tree structure with N-1 nodes is used to manage the access path corresponding to each cache line. Each node corresponds to a path status bit, which dynamically records access direction preferences. In response to a received data replacement instruction, the method obtains the path status bit from the root node to the leaf node on the access path corresponding to each cache line. For a specified cache line, the method calculates the PLRU path value corresponding to the specified cache line based on the path status bit from the root node to the leaf node on the access path corresponding to the specified cache line. The method obtains the importance level of the data already stored in the specified cache line. Based on the PLRU path value, the importance level of the data already stored in the specified cache line, and the specified PLRU path value weight and the specified importance level weight, the method calculates the replacement score of the specified cache line. Based on the replacement score of each cache line, the method determines replaceable cache lines in the data cache. The method stores the data to be cached and its corresponding importance level in the replaceable cache line to replace the already stored data and its importance level in the replaceable cache line. Therefore, by combining the PLRU path value and the importance level of the stored data to evaluate the cache line replacement priority, the cache hit rate is effectively improved and the system performance is optimized.

[0159] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A cache line replacement method, characterized by, The method is applied to a chip, the chip comprising a data cache including N cache lines of stored data, and a binary tree structure including N-1 nodes in a pseudo least recently used (PLRU) algorithm is used to manage a corresponding access path of each cache line, wherein each node corresponds to a path state bit, and the path state bit is used to dynamically record an access direction preference, The method comprises: In response to the received data replacement instruction, path state bits from the root node to the leaf node on the access path corresponding to each cache line are obtained; For a specified cache line, a PLRU path value corresponding to the specified cache line is calculated according to the path state bits from the root node to the leaf node on the access path corresponding to the specified cache line; an importance level of the stored data in the specified cache line is obtained; and a replacement score of the specified cache line is calculated according to the PLRU path value corresponding to the specified cache line and the importance level of the stored data in the specified cache line, and a specified PLRU path value weight and a specified importance level weight. The replaceable cache line in the data cache is determined according to the replacement score of each cache line. The data to be cached and the importance level corresponding to the data to be cached are obtained, and the data to be cached and the importance level corresponding to the data to be cached are stored in the replaceable cache line to replace the stored data and the importance level of the stored data in the replaceable cache line. The PLRU path value corresponding to the specified cache line is calculated according to the path state bits from the root node to the leaf node on the access path corresponding to the specified cache line, including: The path state bits from the root node to the leaf node on the access path corresponding to the specified cache line are added to obtain the PLRU path value corresponding to the specified cache line.

2. The method of claim 1, wherein, The binary tree structure containing N-1 nodes in the pseudo least recently used PLRU algorithm is used to manage the access path corresponding to each cache line, including: After the data to be cached and the importance level corresponding to the data to be cached are stored in the replaceable cache line, the path state bits of the nodes in the binary tree structure are updated based on the PLRU algorithm.

3. The method of claim 1, wherein, The cache line includes a data area and a Tag area. The data to be cached and the importance level corresponding to the data to be cached are stored in the replaceable cache line, including: The data to be cached is stored in the data area of the replaceable cache line, and the importance level corresponding to the data to be cached is stored in the Tag area of the replaceable cache line. The importance level of the stored data in the specified cache line is obtained from the Tag area of the specified cache line.

4. The method of claim 1, wherein, The replaceable cache line in the data cache is determined according to the replacement score of each cache line, including: The lowest replacement score in the replacement score of each cache line is obtained; if the number of the lowest replacement score is one, the cache line corresponding to the lowest replacement score is taken as the replaceable cache line; if the number of the lowest replacement score is multiple, the replaceable cache line in the data cache is found based on the PLRU algorithm.

5. The method of claim 1, wherein, When the lowest replacement score in the replacement score of each cache line is obtained, a multi-level comparison tree structure is used, wherein each level of the multi-level comparison tree structure is a binary comparator, and the binary comparator is used to receive the replacement scores of any two cache lines as input and output the smaller value and the cache line index corresponding to the smaller value. The implementation based on the multi-level comparison tree structure comprises: a first-level comparator compares all cache lines in pairs to obtain a first-level candidate minimum value set; and each subsequent level comparator continues to compare the candidate minimum values output by the previous level in groups until the cache line index corresponding to the final lowest replacement score is output.

6. The method of claim 1, wherein, The method further comprises: obtaining at least one parameter in a history hit rate, a future prediction hit rate and a future prediction access pattern of the data cache; dynamically adjusting the specified PLRU path value weight and the specified importance level weight according to at least one parameter in a history hit rate, a future prediction hit rate and a future prediction access pattern of the data cache.

7. The method of claim 1, wherein, The replacement score for the specified cache line is calculated by the following equation : ； wherein, represents a PLRU path value corresponding to the specified cache line, represents a level of importance of data already stored in the specified cache line, represents a weight of the specified PLRU path value, represents a weight of the specified level of importance, and are integers not less than 1.

8. The method of claim 7, wherein, In the calculation of the replacement score of the specified cache line through the formula, a shift-add circuit is used to realize the multiplication operation in the formula through binary decomposition and weighted accumulation, and then the product results after the multiplication operation are added through an adder to obtain the replacement score of the specified cache line.

9. The method of claim 8, wherein, The shift-add circuit comprises a shift register and an accumulator, and the multiplication operation in the formula is realized through binary decomposition and weighted accumulation based on the shift-add circuit, which comprises: determining whether the importance level of the stored data in the specified cache line is a preset low importance level; if not, a full-precision calculation mode is started, which comprises: decomposing the multiplier into binary bits, detecting all bits in bit sequence from right to left, if the ith bit is 1, the multiplicand is left shifted by i bits through the shift register to obtain the effective shift result corresponding to the ith bit, if it is 0, it is skipped, until all bit sequences are detected; and the effective shift results of all bits of the multiplicand are added through the accumulator to obtain the final product result; if yes, an approximate calculation optimization mode is started, which comprises: decomposing the multiplier into binary bits, selecting the corresponding effective bits in the multiplier according to a preset truncation bit width configuration parameter, detecting the effective bits in bit sequence from right to left, if the ith bit is 1, the multiplicand is left shifted by i bits through the shift register to obtain the effective shift result corresponding to the ith bit, if it is 0, it is skipped, until all bit sequences in the effective bits are detected; and the effective shift results of all bits of the multiplicand and a truncation error compensation value are added through the accumulator to obtain the final product result; wherein the truncation error compensation value is obtained based on an offline statistical average truncation error compensation and a dynamic weight.

10. A cache line replacement apparatus, characterized by, The device is applied to a chip, the chip comprises a data cache including N cache lines of stored data, and the device comprises a PLRU algorithm implementation module, a PLRU path value calculation module, a replacement score calculation module, a replaceable cache line determination module and a replacement operation module, wherein: The PLRU algorithm implementation module is configured to manage the access path corresponding to each cache line based on a binary tree structure containing N-1 nodes in a pseudo least recently used (PLRU) algorithm, wherein each node corresponds to a path state bit, and the path state bit is used to dynamically record the access direction preference. ; The PLRU path value calculation module is configured to, in response to a received data replacement instruction, obtain path state bits from a root node to a leaf node on an access path corresponding to each cache line; and for a specified cache line, calculate a PLRU path value corresponding to the specified cache line according to the path state bits from the root node to the leaf node on an access path corresponding to the specified cache line. The replacement score calculation module is configured to: obtain an importance level of the stored data in the specified cache line; and calculate a replacement score of the specified cache line according to the PLRU path value corresponding to the specified cache line and the importance level of the stored data in the specified cache line, and a specified PLRU path value weight and a specified importance level weight. The replaceable cache line determination module is configured to: determine a replaceable cache line in the data cache according to the replacement scores of each cache line. The replacement operation module is configured to: obtain to-be-cached data and an importance level corresponding to the to-be-cached data; and store the to-be-cached data and the importance level corresponding to the to-be-cached data in the replaceable cache line to replace stored data in the replaceable cache line and an importance level of the stored data. The PLRU path value corresponding to the specified cache line is calculated according to path state bits from a root node to a leaf node on an access path corresponding to the specified cache line, and includes: The path state bits from the root node to the leaf node on the access path corresponding to the specified cache line are added to obtain the PLRU path value corresponding to the specified cache line.

11. The apparatus of claim 10, wherein, The replaceable cache line in the data cache is determined according to the replacement scores of each cache line, and includes: The lowest replacement score in the replacement scores of each cache line is obtained; if the number of the lowest replacement scores is one, the cache line corresponding to the lowest replacement score is taken as the replaceable cache line; if the number of the lowest replacement scores is multiple, the replaceable cache line in the data cache is found based on a PLRU algorithm.

12. The apparatus of claim 11, wherein, When the lowest replacement score in the replacement scores of each cache line is obtained, a multi-stage comparison tree structure is implemented, wherein each stage comparator in the multi-stage comparison tree structure is a binary comparator, and the binary comparator is configured to receive the replacement scores of any two cache lines as input and output a smaller value and a cache line index corresponding to the smaller value. The implementation based on the multi-stage comparison tree structure includes: a first-stage comparator compares all cache lines in pairs to obtain a first-stage candidate minimum value set; and each subsequent stage comparator continues to compare the candidate minimum values output by the previous stage in groups until the cache line index corresponding to the final lowest replacement score is output.

13. The apparatus of claim 10, wherein, The apparatus further includes: The weight dynamic adjustment module is configured to: obtain at least one parameter in a history hit rate, a future predicted hit rate, and a future predicted access pattern of the data cache; and dynamically adjust the specified PLRU path value weight and the specified importance level weight according to the at least one parameter in the history hit rate, the future predicted hit rate, and the future predicted access pattern of the data cache.

14. A chip, characterized by The apparatus includes any one of the apparatuses in claims 10-13; or includes a memory and a processor; wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the method in any one of claims 1-9.

15. An electronic device, comprising: A computer program product comprising a storage medium to store one or more computer instructions, wherein the one or more computer instructions are executable by a processor to implement the method of any one of claims 1-9. A computer program product comprising a storage medium to store one or more computer instructions, wherein the one or more computer instructions are executable by a processor to implement the method of any one of claims 1-9. A computer program product comprising a storage medium to store one or more computer instructions, wherein the one or more computer instructions are executable by a processor to implement the method of any one of claims 1-9. A computer

Images