A hierarchical cooperative cache replacement method, device and chip

Abstract

This invention discloses a hierarchical collaborative cache replacement method, apparatus, and chip, comprising: responding to an eviction event of a data block by the secondary cache within the local node, obtaining context information of the data block; inputting the context information into a filtering module within the local node, filtering the data block according to current configuration parameters; if the filtering result indicates that the data block is potentially important data, querying and updating the entry corresponding to the data block in the prediction module within the local node based on the context information to obtain a reuse probability value of the data block; dynamically adjusting the configuration parameters and / or the reuse probability threshold in the prediction module based on the performance monitoring data of the local node; and assigning a replacement strategy for the data block to the last-level cache of the local node by combining the reuse probability value, the reuse probability threshold, and the source access node identifier. This invention can effectively solve the problems of cross-level information fragmentation, insufficient load adaptability, and lack of NUMA topology awareness.

Description

Technical Field

[0001] This invention relates to the field of computer storage technology, and in particular to a hierarchical collaborative cache replacement method, apparatus, and chip. Background Technology

[0002] In a multi-core processor non-uniform memory access (NUMA) architecture, the rationality of the cache replacement strategy directly affects data access latency, cache resource utilization, and overall system performance.

[0003] Traditional cache replacement strategies have the following problems: First, information is fragmented across cache levels. When the second-level cache performs a data block eviction operation, it cannot pass core information such as the physical address of the data block to the last-level cache. This results in the last-level cache lacking a basis for accurately identifying potentially important data, which in turn leads to the eviction of highly reused data or causes low-value data to remain in the last-level cache for a long time, occupying a large amount of cache space. Second, there is insufficient load adaptability. The threshold cannot be dynamically adjusted, so it cannot adapt to different dynamic changes in load. Third, there is a lack of NUMA topology awareness, which makes it impossible to allocate targeted replacement strategies for data blocks, resulting in high latency for cross-node access.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This specification provides a hierarchical collaborative cache replacement method, apparatus, and chip to address the problems of cross-level information fragmentation, insufficient load adaptability, and lack of NUMA topology awareness in existing technologies.

[0006] Firstly, embodiments of this specification provide a hierarchical collaborative cache replacement method, applied to access nodes, including: In response to an eviction event of a data block by the secondary cache within this node, the context information of the data block is obtained, and the context information includes at least the physical address and the source access node identifier; Input the physical address and source access node identifier into the filtering module within this node, and filter the data blocks according to the current configuration parameters; If the filtering result indicates that the data block is potentially important data, then the table entry corresponding to the data block in the prediction module within this node is queried and updated according to the context information to obtain the reuse probability value of the data block; Based on the performance monitoring data of this node, dynamically adjust the configuration parameters and / or the reuse probability threshold in the prediction module; By combining the reuse probability value, the reuse probability threshold, and the source access node identifier, a replacement strategy for the last-level cache of this node is assigned to the data block.

[0007] In some embodiments, the context information further includes access history, which includes at least the access frequency of the data block and the most recent access timestamp, to assist in updating the table entry corresponding to the data block in the prediction module within the local node.

[0008] In some embodiments, the filtering module includes a Bloom filter, and the configuration parameters include the bit array size and the number of hash functions; correspondingly, filtering the data block according to the current configuration parameters includes: Multiple sets of hash results are obtained by jointly calculating the physical address and the source access node identifier using multiple sets of parallel hash functions. If all bit array positions corresponding to all hash results are 1, then the data block is determined to be potentially important data with a high probability. If any bit array position corresponding to any hash result is 0, the data block is determined to be short-term invalid data and is discarded directly.

[0009] In some embodiments, the prediction module includes a cross-layer metadata table, the table entries of which include at least: The physical address hash value is used to match and index the table entries corresponding to the data block; A saturation counter, the value of which is used to quantify the reuse probability of a data block, and the value of which is dynamically updated based on the access history of the data block; Source access node identifier, used to identify the initial home node of a data block and to assist in matching table entries; The status flag is used to identify the current status of the data block.

[0010] In some embodiments, querying and updating the table entry corresponding to the data block in the prediction module within the current node based on the context information includes: Perform a hash calculation on the physical address to obtain the physical address hash value; By combining the physical address hash value with the source access node identifier, the corresponding entry is queried in the cross-layer metadata table; If a corresponding table entry is found, the saturation counter is adjusted according to the access frequency. If no corresponding table entry is found, a new table entry is created, and the physical address hash value, source access node identifier, and status flag are written into the table entry, and the value of the saturation counter is initialized.

[0011] In some embodiments, the performance monitoring data includes the number of misses per thousand instructions, the ratio of read to write requests, and access locality characteristics; correspondingly, the configuration parameters are dynamically adjusted based on the performance monitoring data of this node, including: If the number of misses per thousand instructions is higher than the first miss threshold and the write request ratio is lower than the write ratio threshold, it is determined to be a high throughput load, and the bit array size and the number of hash functions are reduced. If the write request ratio is detected to be higher than the write ratio threshold, it is determined to be a write-intensive load, and the bit array size is increased and the number of hash functions is increased. If the access locality characteristic is detected as random access ratio exceeding the random access threshold, it is determined to be random access load, and the number of hash functions is increased and the bit array size is increased.

[0012] In some embodiments, dynamically adjusting the reuse probability threshold in the prediction module based on the performance monitoring data of this node includes: If the number of misses per thousand instructions is higher than the second miss threshold, it is determined to be a high miss scenario, and the reuse probability threshold is lowered. If the number of misses per thousand instructions is lower than the third miss threshold, it is determined to be a low miss scenario, and the reuse probability threshold is increased. If the fluctuation range of the read / write request ratio is detected to be higher than the fluctuation range threshold, it is determined to be an unstable load scenario, the reuse probability threshold is kept unchanged and the adjustment interval is extended; Wherein, the second miss threshold is greater than or equal to the first miss threshold and greater than or equal to the third miss threshold.

[0013] In some embodiments, the step of assigning a replacement strategy for the last-level cache of the local node to the data block by combining the reuse probability value, the reuse probability threshold, and the source access node identifier includes: If the reuse probability value is higher than the reuse probability threshold and the source access node identifier identifies the local node, then a first type of management strategy for long-term retention is assigned. If the reuse probability value is lower than the reuse probability threshold or the source access node identifier identifies a remote node, then a second type of management strategy for rapid eviction is assigned.

[0014] In some embodiments, the first type of management strategy is a least recently used strategy, which is used to insert the data block into the least recently used end of the linked list maintained by the strategy; the second type of management strategy is a static rereference interval prediction strategy, which is used to associate a rereference prediction value with the data block and initialize the rereference prediction value to a preset maximum value to indicate that the data block has the lowest recent reuse priority.

[0015] In some embodiments, the method further includes: When implementing the replacement policy for the last-level cache, priority is given to evicting data blocks managed by the second type of management policy whose re-reference prediction value has reached its maximum value; If eviction is still necessary, then eviction will be carried out on data blocks that are at the cold end of the least recently used list and are managed by the first type of management strategy.

[0016] In some embodiments, the method further includes: If, within a preset statistical period, the frequency of access to remote access node data blocks by this node exceeds a preset access threshold, a data migration request will be triggered. The node routing module migrates the data block from the last-level cache of the source access node to the last-level cache of this node, and synchronously updates the table entries of the prediction module of this node.

[0017] Secondly, embodiments of this specification also provide a hierarchical collaborative cache replacement apparatus, comprising: The first acquisition module is used to acquire the context information of the data block in response to the eviction event of the secondary cache within the local node. The context information includes at least the physical address and the source access node identifier. The filtering module is used to input the physical address and source access node identifier into the filtering module within this node, and filter the data blocks according to the current configuration parameters; The second acquisition module is used to query and update the table entry corresponding to the data block in the prediction module within this node according to the context information if the filtering result indicates that the data block is potentially important data, so as to obtain the reuse probability value of the data block. The adjustment module is used to dynamically adjust the configuration parameters and / or the reuse probability threshold in the prediction module based on the performance monitoring data of this node. The replacement module is used to combine the reuse probability value, the reuse probability threshold, and the source access node identifier to assign a replacement strategy for the last-level cache of the current node to the data block.

[0018] Thirdly, embodiments of this specification also provide a chip, including a processing unit and a storage unit for storing instructions executable by the processing unit, wherein the processing unit executes the instructions to implement the steps of the method described in the first aspect above.

[0019] In some embodiments, the processing unit includes at least a filtering module, a prediction module, a load adaptive threshold engine, and a policy replacement and execution module. The filtering module is used to filter data blocks according to the current configuration parameters. The prediction module is used to output the reuse probability value of the data block. The load adaptive threshold engine is used to dynamically adjust the configuration parameters and / or the reuse probability threshold in the prediction module. The policy replacement and execution module is used to assign a replacement policy for the last-level cache of the current node to the data block.

[0020] Fourthly, embodiments of this specification also provide a computer storage medium storing computer program instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0021] Fifthly, embodiments of this specification also provide a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0022] This specification provides a hierarchical collaborative cache replacement method, apparatus, and chip. First, in response to an eviction event of a data block by the secondary cache within the local node, the context information of the data block is obtained. This context information includes at least the physical address and the source access node identifier. Then, the physical address and the source access node identifier are input to a filtering module within the local node to filter the data block according to current configuration parameters. If the filtering result indicates that the data block is potentially important data, the entry corresponding to the data block in the prediction module within the local node is queried and updated based on the context information to obtain the reuse probability value of the data block. Then, based on the performance monitoring data of the local node, the configuration parameters and / or the reuse probability threshold in the prediction module are dynamically adjusted. Finally, combining the reuse probability value, the reuse probability threshold, and the source access node identifier, a replacement strategy for the last-level cache of the local node is assigned to the data block. In this specification embodiment, by actively obtaining the context information of the data block (which may include at least the physical address and the source access node identifier) ​​in response to an eviction event of a data block by the local node's secondary cache, the fragmented state of the secondary cache "only evictions data, without transmitting core information" can be broken. By inputting the physical address and source access node identifier into the filtering module of this node, data blocks are filtered according to the current configuration parameters, effectively preventing low-value data from entering the last-level cache and causing space occupation. For blocks identified as potentially important data, the prediction module entries are queried and updated based on context information to obtain quantified reuse probability values. This transforms the core information passed by the second-level cache into a "data reuse value indicator" directly identifiable by the last-level cache, fundamentally preventing the erroneous removal of highly reused data and the long-term retention of low-value data. By dynamically adjusting the configuration parameters and / or the reuse probability threshold in the prediction module based on the performance monitoring data of this node, the changes in parameters / thresholds can be tailored to the actual operating characteristics of the current load, rather than remaining fixed. This allows the cache replacement strategy to adapt to the dynamic changes of different loads in real time, significantly reducing cache miss rate fluctuations under different load scenarios and significantly improving system performance stability. By combining reuse probability value, reuse probability threshold and source access node identifier, targeted last-level cache replacement strategies can be assigned to data blocks with different topology affiliations and values. This solves the problem of high cross-node access latency caused by the lack of NUMA topology awareness when all evicted data blocks are replaced with an indiscriminate cache replacement strategy. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of this specification and form part of it, do not constitute a limitation thereof. In the drawings: Figure 1 This is an architecture diagram of the cache replacement system provided in the embodiments of this specification; Figure 2This is a flowchart illustrating a hierarchical collaborative cache replacement method provided in the embodiments of this specification; Figure 3 This is a schematic diagram illustrating the working principle of the Bloom filter provided in the embodiments of this specification; Figure 4 This is a structural diagram of the cross-layer metadata table provided in the embodiments of this specification; Figure 5 This is an adaptive mapping table between caching strategies and load characteristics provided in the embodiments of this specification; Figure 6 This is a schematic diagram of the high-reusability block LRU replacement strategy provided in the embodiments of this specification; Figure 7 This is a schematic diagram of the low-reuse block SRRIP replacement strategy provided in the embodiments of this specification; Figure 8 This is a schematic diagram illustrating the structural composition of a hierarchical collaborative cache replacement device provided in the embodiments of this specification; Figure 9 This is a schematic diagram of the structure of a chip provided in an embodiment of this specification. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0026] This specification provides the operational steps of the methods described in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or device products, the methods shown in the embodiments or drawings can be executed sequentially or in parallel.

[0027] It should be noted that in the embodiments of this specification, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of the present invention. However, they do not mean that the applicant has used or necessarily used such solutions.

[0028] The following is a description of the key terms used in this application: 1. NUMA (Non-Uniform Memory Access): A non-uniform memory access architecture where multiple processor nodes accessing local memory have low latency, while cross-node access has high latency.

[0029] 2. LLC / L3 (Last-Level Cache): Last-level cache, usually referring to L3 cache, is used for shared data storage.

[0030] 3. Bloom filter: A space-efficient probabilistic data structure used to quickly determine whether an element is in a set without missing any values.

[0031] 4. CMT (Cross-layer Metadata Table): A cross-layer metadata table that records information about cache blocks or data blocks evicted from L2 (second-level cache) and predicts their reuse probability.

[0032] 5. SRRIP (Static Re-Reference Interval Prediction): A cache replacement strategy that predicts evicted data based on the static re-reference interval.

[0033] 6. LRU (Least Recently Used): The least recently used cache replacement strategy.

[0034] 7. MPKI (Misses Per Kilo Instructions): The number of misses per kilo instruction, used to quantify cache performance.

[0035] 8. CHAR (Cache Hierarchy-Aware Replacement): Hierarchy-aware cache replacement strategy.

[0036] 9. HAWACR (Hierarchy-Aware & Workload-Adaptive Cache Replacement): The cache replacement method proposed in this application emphasizes hierarchy awareness and workload adaptation.

[0037] As mentioned earlier, in multi-core processor non-uniform memory access (NUMA) architectures, cache hierarchy cooperation efficiency is a key factor affecting system performance. Traditional cache replacement strategies (such as LRU, LRU-like, and SRRIP) rely solely on single-level cache access history (such as L3 local access records), failing to utilize eviction information at the L1 / L2 levels, leading to two major problems: Cold data retention: Low-reusability data occupies LLC space for a long time, crowding out high-value data resources; Hybrid load sensitivity: When AI training (high spatial locality) and database (random access) coexist, the last-level cache miss rate fluctuates significantly (e.g., 20%). The limitation of this strategy stems from the fragmentation of information across levels, making it impossible to construct a global view of the data lifecycle.

[0038] To address the information fragmentation problem, existing technologies have proposed hierarchical-aware cache replacement algorithms. For example, the CHAR algorithm analyzes the eviction flow of the L2 cache to provide decision-making references for the L3 cache. This scheme introduces a hierarchical coordination mechanism for the first time, using a static classifier to divide L2 evicted blocks into three categories: single access, low-frequency reuse, and high-frequency reuse, and assigns a fixed reuse probability to each category based on historical hit rate to guide LLC replacement. However, the CHAR scheme has significant limitations: first, its classification threshold is fixed, making it unable to respond to dynamic load migration, and performance fluctuations remain high (e.g., 15%) under mixed loads; second, it does not consider NUMA node distance, lacks topology awareness, and cannot optimize cross-node access; third, the hardware overhead is not optimal, requiring an 8-bit counter to record access history, and its metadata storage accounts for approximately 7% of the LLC capacity.

[0039] On the other hand, dynamic cache replacement strategies based on machine learning have emerged in recent years. These schemes utilize neural network predictors (such as lightweight LSTM models) to analyze access sequences to predict the probability of data block reuse and dynamically update model parameters through online training mechanisms. While theoretically offering better adaptability, they suffer from a fatal flaw: excessive hardware overhead (existing dynamic and machine learning schemes introduce significant area and capacity overhead, making them unsuitable for commercial server chips): LSTM units require a large number of logic gates, exceeding commercially viable area overhead (e.g., 10%) under advanced processes; feature storage occupies a considerable proportion of cache capacity (e.g., 8%-12%), crowding out effective data space. Furthermore, their real-time performance is insufficient; prediction and update latency cannot meet the demands of high-throughput scenarios, and there is a risk of training convergence failure. The model may become inaccurate during sudden load changes, requiring millions of cycles for reconvergence, during which performance degrades significantly.

[0040] In summary, existing technical solutions all have significant shortcomings: static solutions (such as CHAR) have poor adaptability and are topology-aware; dynamic solutions (such as ML-based) have huge overhead and high latency. Therefore, there is an urgent need in this field for a new cache replacement method that can effectively break down information barriers between cache levels under strictly controllable hardware overhead, achieve rapid adaptation to dynamic loads, and deeply integrate NUMA topology-aware optimization, thereby achieving high-performance and highly stable system operation in complex mixed load environments.

[0041] To address the aforementioned issues, embodiments of this specification provide a hierarchical collaborative cache replacement method, apparatus, and chip. This provides a hierarchical collaborative, load-adaptive, and topology-aware cache replacement scheme that can achieve a precise balance between cache utilization, data access latency, and system performance stability.

[0042] Specifically, to address the issue of fragmented hierarchical caching information (e.g., the inability of L1 / L2 eviction information to be passed to L3, leading to highly reusable data (e.g., hot database indexes) being evicted by L2 and then misjudged as cold data by L3, or low-value data (e.g., temporary variables) remaining in L3 for extended periods, consuming significant cache space), the complete context information of L2-evicted data blocks is captured, achieving continuous data lifecycle transmission from L2 to L3. To address the issue of insufficient load adaptability (e.g., the inability to respond in real-time to load changes (e.g., switching from AI training to database queries), causing performance fluctuations. For AI training loads, which exhibit high spatial locality, static strategies can lead to excessive retention of intermediate data. For real-time database loads, characterized by random access and high-frequency updates, static strategies can result in the mistaken eviction of hot data), the configuration parameters and / or the reuse probability threshold in the prediction module can be dynamically adjusted based on the node's performance monitoring data (i.e., the process of generating load fingerprints in real-time through performance counters). Simultaneously, adaptive adjustment of thresholds or configuration parameters can reduce hardware overhead. To address the issue of high cross-node access latency caused by the lack of topology differentiation in the replacement strategy, a replacement strategy can be assigned to the data block by combining the reuse probability value, the reuse probability threshold, and the source access node identifier.

[0043] The hierarchical collaborative cache replacement method in the embodiments of this specification will be described in detail below with reference to the accompanying drawings.

[0044] The hierarchical collaborative cache replacement method described in the embodiments of this specification can be applied to a hierarchical collaborative cache replacement system (HAWACR system (Hierarchy-Aware & Workload-Adaptive Cache Replacement)). The HAWACR system adopts a distributed deployment, and its core architecture can be as follows: Figure 1 As shown, it may include the following components: 1. Multi-level caching components: This includes Level 1 Cache (L1), Level 2 Cache (L2), and Last Level Cache (LLC Cache, L3). The Level 1 cache is configured specifically for each core (processor core), the Level 2 cache is deployed independently per core / slice, and the Last Level Cache is a shared cache within the node, used to handle access requests that have missed in other cache levels and store frequently reused data.

[0045] 2. L2 Evict Stream Processing Module: When the L2 cache capacity is saturated and an eviction operation is triggered, this module extracts the core information of the evicted data block (which may include physical address, access information, etc.) to form the L2 Evict Stream, which can provide data input for subsequent processing.

[0046] 3. Filtering module: A Bloom filter can be used as the first-level data filter to filter cache blocks or data blocks evicted by L2, quickly intercept short-term invalid data, and forward only data that may be reused (potentially reused data) to subsequent modules, reducing invalid computation overhead.

[0047] 4. Prediction Module: The core is the Cross-Layer MetaData Table (CMT), which can adopt a 4-way set associative structure to record information of blocks evicted from L2, update entries (address, probability counter, etc.), and predict the reuse probability of data blocks to achieve the second level of filtering.

[0048] 5. Load Adaptive Threshold Engine: Real-time monitoring of hardware performance counters, generation of load feature fingerprints, dynamic adjustment of Bloom filter configuration parameters (bit array size m, number of hash functions K) and CMT reuse probability threshold to adapt to different load scenarios.

[0049] 6. Policy Replacement and Execution Module: Based on the reuse probability output by CMT and combined with NUMA node topology information, the LRU policy is used to retain highly reused data, and the SRRIP policy is used to quickly evict low-reuse data, thereby realizing differentiated cache management.

[0050] 7. Node ID Routing Module: Responsible for data interaction between this node and other NUMA nodes, supporting cross-node data migration and access, and working together to achieve topology-aware optimization.

[0051] 8. Memory Controller: Handles access requests that are not found in the LLC Cache, enabling data read and write interactions between the cache and memory.

[0052] The HAWACR system and the access nodes in the NUMA architecture have a one-to-one deployment relationship: each access node independently deploys a complete HAWACR system, which can include all the core components mentioned above. This deployment mode enables each access node to autonomously complete the entire process of "data eviction-filtering-prediction-parameter adjustment-policy allocation" without relying on cross-node collaboration, achieving efficient localized processing. Simultaneously, through the node routing module's linkage with the HAWACR systems of other access nodes, cross-node data migration and access can be completed, balancing efficient localized processing with global collaborative optimization.

[0053] The above components can be tightly coupled through a dedicated hardware pipeline, achieving high-accuracy prediction while maintaining low latency. The Bloom filter can employ a multi-bank parallel design, with each bank corresponding to a different set of hash functions, supporting 3-5 hash calculations per cycle. The CMT can use a 4-way associative structure, effectively reducing the hash collision rate through dual indexing of physical address hashing and NUMA node IDs. The hardware overhead of the entire system has been carefully optimized, resulting in significantly lower hardware costs compared to existing solutions based on machine learning or complex counters.

[0054] The system described above can deeply integrate NUMA topology information to achieve cross-node optimization. The Node ID field of each CMT entry is linked to the routing table. Local node data automatically receives a 1.2x retention weight, while remote node data is processed according to topological distance: one hop (same socket) reduces weight by 10%, two hops (cross socket) by 20%, and three hops (cross motherboard) by 30%. During data migration, the system prioritizes target nodes with closer topological distances to reduce cross-node access latency. For example, if local data block X (reuse counter 10) of node A and remote data block Y (reuse counter 10) of node B are both in node A's LLC, when the LLC is full, the probability of data block Y being evicted is 1.2 times that of data block X, ensuring that high-value local data is retained first. If node A's LLC contains a one-hop remote data block M (node ​​C, same socket) and a three-hop remote data block N (node ​​D, cross motherboard), both with the same reuse counter, when the LLC is full, data block N has a higher probability of being evicted (due to a larger reduction in weight). When a remote node frequently accesses a certain data block, the system triggers data migration, prioritizing the target node with the closest topological distance to reduce cross-node latency for subsequent accesses.

[0055] Figure 2This is a flowchart illustrating a hierarchical collaborative cache replacement method provided in the embodiments of this specification. Although this specification provides method operation steps or apparatus structures as shown in the following embodiments or figures, more or fewer operation steps or module units may be included in the method or apparatus based on conventional or non-inventive effort. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the apparatus is not limited to the execution order or module structure described in the embodiments and figures of this specification. When the method or module structure is applied in actual devices or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or figures (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed processing environment). Figure 2 As shown, this method can be applied to access nodes (or processor nodes in a NUMA architecture), and the method may include: S201: In response to an eviction event of a data block by the secondary cache within this node, obtain the context information of the data block, wherein the context information includes at least the physical address and the source access node identifier; S202: Input the physical address and source access node identifier into the filtering module within this node, and filter the data blocks according to the current configuration parameters; S203: If the filtering result indicates that the data block is potentially important data, then query and update the table entry corresponding to the data block in the prediction module within this node according to the context information to obtain the reuse probability value of the data block; S204: Based on the performance monitoring data of this node, dynamically adjust the configuration parameters and / or the reuse probability threshold in the prediction module; S205: Combining the reuse probability value, the reuse probability threshold, and the source access node identifier, assign a replacement strategy for the last-level cache of this node to the data block.

[0056] Specifically, when a data block eviction event is triggered due to capacity saturation in the local L2 cache, the context information of the evicted data block is directly extracted from the L2 cache eviction stream. This information can at least include the physical address (uniquely identifying the data block's location in memory) and the source access node identifier (identifying the NUMA node to which the data block initially belonged; for example, node ID=0 represents the local node, and ID≥1 represents the remote node). Then, the physical address and source access node identifier are used as joint input to the local node's filtering module (e.g., a Bloom filter). The filtering operation is performed according to the current configuration parameters (e.g., bit array size, number of hash functions) to quickly distinguish between short-term invalid data (i.e., low-value data) and potentially important data (i.e., highly reusable data). If the filtering result indicates potentially important data, the corresponding entry is queried in the prediction module (e.g., CMT) based on the context information (using a joint index of the physical address hash value and the source access node identifier). The entry is updated according to the data block's access characteristics (e.g., adjusting the saturation counter value in the entry), and the saturation counter value in the entry is used as the data block's reuse probability value. Subsequently, real-time performance monitoring data of this node (such as the number of misses per thousand instructions, the ratio of read to write requests, access locality characteristics, etc.) can be collected. Based on this data, the load adaptive threshold engine can dynamically adjust the configuration parameters of the Bloom filter and / or the reuse probability threshold of the CMT to adapt the strategy to the current load state. Finally, by combining the reuse probability value of the data block, the current reuse probability threshold, and the source access node identifier, a replacement strategy for the last-level cache (LLC Cache / L3) can be assigned to the data block, such as prioritizing long-term retention for locally high-reuse data and assigning fast eviction strategies to remote or low-reuse data.

[0057] The filtering results indicate that the data block is potentially important data, meaning that the filtering results indicate that the data block may have reuse value (which can be determined by a Bloom filter).

[0058] The above approach constructs a core framework of "hierarchical collaboration + load adaptation + topology awareness," which overcomes the shortcomings of traditional cache replacement strategies, such as cross-level information fragmentation, insufficient load adaptation, and lack of topology differentiation. This makes the resource allocation of the last-level cache (LLC Cache / L3) more accurate, reduces data access latency, and improves cache utilization.

[0059] In some embodiments, the process before S201 may further include: the local node core initiating a data access request, prioritizing a query of the Level 1 Cache (L1 Cache), which is a core-specific cache used to store frequently accessed data blocks with minimal access latency. If the L1 Cache is hit, the data is directly returned to the core, and the access status of the data block is updated. If the L1 Cache is not hit, the access request is forwarded to the local node's Level 2 Cache. The Level 2 Cache receives the access request; if the target data block is present in the cache, it returns the data to the core and synchronously updates the access status and context information of the data block. If the Level 2 Cache is not hit, the target data block is loaded from the local node's last-level cache or system memory into the Level 2 Cache, and the physical address, source access node identifier, and initial access history of the data block are recorded to form context information. As data blocks are continuously written to the Level 2 Cache, when the used capacity of the Level 2 Cache reaches a preset saturation condition, a replacement decision mechanism for the Level 2 Cache is triggered, filtering data blocks to be evicted and performing eviction operations.

[0060] In some embodiments, the context information in S201 above may also include access history, which may at least include the access frequency of the data block and the most recent access timestamp, and may be used to assist in updating the table entry corresponding to the data block in the prediction module within the local node.

[0061] Specifically, context information can include access history, such as the access frequency of a data block (the number of times it is accessed by the core within a unit cycle, such as 5 times within 100 clock cycles) and the most recent access timestamp (recording the clock cycle in which the data block was last accessed). This information is collected in real time by the L2 cache while the data block is cached, and is output along with the physical address and source access node identifier during eviction. It can be used to assist in updating CMT entries (such as adjusting the saturation counter based on access frequency, and determining the reuse timeliness of the data block based on the most recent access timestamp).

[0062] By acquiring the access history of data blocks, we can provide richer decision-making basis for the prediction module (CMT), make the quantification of data block reuse probability more accurate, reduce misjudgments caused by insufficient information, and improve the accuracy of high reuse data identification.

[0063] In some embodiments, the filtering module in S202 above may include a Bloom filter, and the configuration parameters may include the bit array size and the number of hash functions; correspondingly, filtering data blocks according to the current configuration parameters in S102 above may, in specific implementation, include: Multiple sets of hash results are obtained by jointly calculating the physical address and the source access node identifier using multiple sets of parallel hash functions. If all bit array positions corresponding to all hash results are 1, then the data block is determined to be potentially important data with a high probability. If any bit array position corresponding to any hash result is 0, the data block is determined to be short-term invalid data and is discarded directly.

[0064] Specifically, when the aforementioned filtering module is a Bloom filter, the configuration parameters can include the size of the bit array (which can affect filtering accuracy; the larger the size, the lower the false positive rate) and the number of hash functions (which can affect computational efficiency; more hash functions result in slightly higher computational latency but improved accuracy). The filtering logic can adopt a "joint hashing + bit array matching" mechanism: multiple sets of parallel hash functions (such as 3-5 sets of CRC32 hash functions) are used to jointly calculate the concatenation result of the physical address and the source access node identifier to obtain multiple sets of hash results; the bit array of the Bloom filter is traversed, and if all the bit array positions corresponding to the hash results are 1, it indicates that the data block is likely to be potentially important data (i.e., the data block is judged to be potentially important data with a high probability), and it is allowed to pass to the prediction module; if any position is 0, it is judged to be short-term invalid data and is directly discarded without further processing.

[0065] Specifically, the concatenation result of the physical address and the source access node identifier (Node ID) is jointly calculated to obtain the hash key: Hash Key = Physical Address ^ Node ID. By incorporating the NUMA node ID into the hash key calculation, data from the same node can be more evenly distributed across the bit array, effectively reducing the cross-node collision rate. The filter state is globally reset every cycle to prevent the accumulation of outdated mappings from increasing the false positive rate.

[0066] The Bloom filter serves as the first line of defense in the system, located at the entry point of each NUMA node. It employs a multi-layered optimization design to ensure efficient interception, filtering out short-lived invalid cache blocks or data blocks evicted by L2 cache to reduce CMT update pressure. Its core is a dynamically adjustable bit array structure, whose size can elastically vary between 8n and 13n (where n is the number of prediction elements) based on load characteristics. The system has three built-in selectable hash function sets (k=3 / 5 / 8), selected in real-time by the threshold engine based on PMC metrics: for high-throughput scenarios (such as AI training), a lightweight configuration of k=3 is selected; for low-latency scenarios (such as database queries), a high-precision mode of k=8 is enabled.

[0067] See Figure 3As shown, a physical address can be simultaneously fed into multiple parallel hash functions (such as hash function H1, hash function H2, hash function H3, hash function H4, and hash function Hn). Each hash function calculates independently, producing a hash value that points to a specific position (i.e., bit array index) in a large bit array. The result calculated by each hash function corresponds to a bit array position in the bit array. The system queries the values ​​of the binary bits at these positions in parallel (whether they are "0" or "1"). This bit array is the core storage structure of the filter, and all its bits are initialized to 0. When a data block is determined to be "potentially important," the bits corresponding to all its hash positions are set to 1. If all bits are 1, this result indicates or suggests that the data block may be reused, and it will be transferred to CMT. If any bit is 0, this result indicates that the data block is short-term invalid data (such as a one-time accessed temporary variable or streaming data), and it will be discarded immediately.

[0068] In some embodiments, the Node ID field can be combined with QoS policies to assign higher retention weights to high-priority applications, an alternative that can be applied to cloud-native environments.

[0069] In some embodiments, a Bloom filter can act as a short-term data interception layer. Through pre-filtering, it can intercept short-term data (such as temporary variables and streaming data) with ≤1 access frequency, preventing it from entering the high-overhead CMT system. By dynamically adjusting parameters, the false positive rate can be suppressed to <a first preset false positive rate threshold (e.g., 0.1%, compared to >1% in traditional solutions). The Bloom filter can use a hardware-friendly hash function (e.g., CRC32c) to optimize single-cycle computation latency, ensuring ≤2 cycles. The hash function k is dynamically adjustable (k=3~8). In high-throughput scenarios (AI training), k=3, sacrificing accuracy for speed; in low-latency scenarios (databases), k=8, suppressing the false positive rate to a second preset false positive rate threshold (e.g., 0.01%). In the design of the bit array in the Bloom filter, the size is determined as m=n×c (n=number of elements, c=compression factor). Under dynamic load, c=8~13. For write-intensive loads: c=8 → effective area reduction; for read-intensive loads: c=13 → effective decrease in false positive rate. Meanwhile, the Bloom filter supports NUMA partition optimization, with each NUMA node having an independent bit array and the high bits of the physical address being merged with the Node ID, reducing the cross-node collision rate.

[0070] In some embodiments, a Cuckoo filter can be used instead of a Bloom filter to reduce the false positive rate (the difference is that the Cuckoo filter supports deletion operations, but the area is slightly larger).

[0071] In some embodiments, the prediction module in S203 above may include a cross-layer metadata table, the table entries of which may include at least: The physical address hash value can be used to match and index the table entries corresponding to the data block; A saturation counter, the value of which can be used to quantify the reuse probability of a data block, and the value of which is dynamically updated based on the access history of the data block; The source access node identifier can be used to identify the initial home node of a data block and assist in matching table entries; The status flag can be used to identify the current status of the data block.

[0072] Specifically, the aforementioned prediction module can be a cross-layer metadata table (CMT), which serves as the system's intelligent prediction center, recording information about blocks evicted from L2 and predicting their reuse probability. The CMT structure is a group-associative table, storing metadata by NUMA node partitions to reduce hash collisions.

[0073] See Figure 4 As shown, its entries can adopt a fixed 32-bit structure, and the entries can specifically include: A 20-bit physical address hash (Address Tag) can cover more than 90% of the 64-bit address space; a 2-bit reuse counter can quantify the probability of block reuse, 00 = cold data → 11 = hot data; a 4-bit source access node identifier (NUMA Node ID) can support 16-node systems, identifying the source access node of the block for topology-aware optimization; and 6-bit state bits can include Clean / Dirty flags (clean flags indicate that the data in the cache line is completely consistent with the corresponding data in main memory, and dirty flags indicate that the data in the cache line is inconsistent with the corresponding data in main memory), lock flags, etc.

[0074] The table update adopts a write-first strategy. Newly arrived L2 eviction blocks trigger parallel queries and updates in the CMT: first, the target group is located via hashing, then the tag value is compared; if a match is found, the counter is updated; otherwise, the old entry with the highest RRPV is replaced. To reduce power consumption, the CMT can employ bank hibernation technology, with idle banks automatically entering a low-power mode. Real-world testing shows that this design maintains high prediction accuracy while reducing static power consumption.

[0075] It should be noted that the size of CMT is configurable, and a typical implementation occupies less than 3% of the L3 cache area.

[0076] Specifically, the physical address hash value (Address Tag) is obtained by performing a lightweight hash calculation on the physical address of the data block. It can be used to quickly match and index target entries in CMT, reducing storage overhead. The saturation counter (ReuseCounter) has a value range of 00 (lowest reuse probability) to 11 (highest reuse probability). It is dynamically updated based on the access history of the data block (i.e., the saturation counter is dynamically adjusted based on the access frequency: the counter increments for every N accesses; if it has not been accessed for a long time, the counter decrements or remains unchanged), directly quantifying the reuse probability value of the data block. The source access node identifier (NUMA Nodel ID) identifies the initial home node of the data block. It can work with the physical address hash value to achieve accurate matching of entries, avoiding hash collisions between data blocks with the same name on different nodes, and providing a basis for topology-aware strategies. The status flag indicates the current status of the data block, assisting in cache replacement decisions (e.g., prioritizing Clean data blocks during eviction).

[0077] The aforementioned CMT table structure is compact, with low storage overhead (accounting for less than 5% of the LLC capacity). The 4-way set-associative design results in a low hash collision rate and low latency for table lookup and update, providing efficient and accurate hardware support for reuse probability quantification.

[0078] In some embodiments, the 2-bit saturation counter of CMT can be replaced with a 3-bit saturation counter or a probabilistic counter chain (such as the Markov model) to accommodate more complex loads, but this requires a trade-off in hardware overhead.

[0079] In some embodiments, the step of S203 above, which involves querying and updating the table entry corresponding to the data block in the prediction module within the current node based on the context information, may specifically include: Perform a hash calculation on the physical address to obtain the physical address hash value; By combining the physical address hash value with the source access node identifier, the corresponding entry is queried in the cross-layer metadata table; If a corresponding table entry is found, the saturation counter is adjusted according to the access frequency. If no corresponding table entry is found, a new table entry is created, and the physical address hash value, source access node identifier, and status flag are written into the table entry, and the value of the saturation counter is initialized.

[0080] Specifically, the physical address of the data block is first hashed to obtain the physical address hash value. Then, combined with the source access node identifier, a matching entry is searched in the corresponding group of CMT (the physical address hash value and the source access node identifier must match). If a matching entry is found, the saturation counter is adjusted according to the access frequency in the access history (e.g., incrementing by 2 levels for access frequency ≥ 5 times, incrementing by 1 level for access frequency 3 ≤ access frequency < 5 times, and not incrementing for access frequency < 3 times), and the status flag is refreshed synchronously. If no matching entry is found, a new entry is created, the physical address hash value, the source access node identifier, and the initial status flag (e.g., valid bit = 1, Clean / Dirty = Clean) are written in, and the saturation counter value is initialized to 01 (weak reuse tendency).

[0081] It should be noted that if a hash collision occurs or the table is full, it will be handled according to a predefined replacement strategy (such as replacing the old table entry with the largest RRPV).

[0082] By dynamically updating table entries, it can be ensured that the reuse probability value is aligned with the access characteristics of the data block in real time, avoiding decision bias caused by outdated information and improving the continuous identification accuracy of highly reused data.

[0083] In some embodiments, the CMT described above can be equivalent to a long-term reuse probability quantization layer. It can use a 2-bit saturated counter chain to implement 4-level reuse probability quantization with minimal hardware, integrate cross-layer metadata integration, and record the complete context (address, node, state) of L2 eviction blocks.

[0084] When a data block is evicted from the L2 cache, a Bloom filter is used for pre-screening to intercept short-term invalid data (such as temporary variables accessed ≤1 times). Potentially reusable blocks that pass the screening are then entered into the Cross-Layer Metadata Table (CMT) system. During initialization, a 2-bit reuse counter is set to "01" (indicating weak reuse tendency), and the NUMA node ID is recorded to identify the data source. If the data block is accessed again later, its reuse level is dynamically increased (01→10→11), and local node data is given an additional +1 weight to accelerate its upgrade to a high reuse state (reflecting NUMA topology optimization). During the eviction decision phase, the system performs differentiated processing based on the real-time status recorded in the CMT: data blocks whose reuse counter drops to "00" are marked as low-value data and directly sent to the highest priority position in the SRRIP eviction queue for rapid release. High-reuse blocks whose counter reaches "11" are locked in the head area of ​​the LRU retention queue for long-term residence, and high-reuse data on local nodes also receive additional retention weights.

[0085] In some embodiments, the performance monitoring data in S204 above may include the number of misses per thousand instructions, the ratio of read to write requests, and access locality characteristics; correspondingly, the performance monitoring data based on the local node in S204 above, dynamically adjusting the configuration parameters, may include, in specific implementations: If the number of misses per thousand instructions is higher than the first miss threshold and the write request ratio is lower than the write ratio threshold, it is determined to be a high throughput load, and the bit array size and the number of hash functions are reduced. If the write request ratio is detected to be higher than the write ratio threshold, it is determined to be a write-intensive load, and the bit array size is increased and the number of hash functions is increased. If the access locality characteristic is detected as random access ratio exceeding the random access threshold, it is determined to be random access load, and the number of hash functions is increased and the bit array size is increased.

[0086] Specifically, the performance monitoring data mentioned above may include: misses per thousand instructions (MPKI, Quantized Cache Hit Rate), read / write request ratio (distinguishing between read / write intensive loads), and access locality characteristics (distinguishing between sequential / random access). The dynamic adjustment logic for the above configuration parameters can be as follows: High throughput load determination: If the number of misses per thousand instructions (MPKI) is higher than the first miss threshold (e.g., 10) and the write request ratio is lower than the write ratio threshold (e.g., 60%), then prioritize processing speed, reduce the bit array size of the Bloom filter (e.g., from 10n to 8n) and reduce the number of hash functions (e.g., from 5 groups to 3 groups).

[0087] Write-intensive load assessment: When the proportion of write requests exceeds the write proportion threshold, prioritize reducing the false positive rate of filtering, increasing the size of the bit array (e.g., from 10n to 13n), and increasing the number of hash functions (e.g., from 5 groups to 8 groups).

[0088] Random access load determination: When the proportion of random access in the access locality feature is higher than the random access threshold (e.g., 70%), balance accuracy and efficiency by increasing the number of hash functions (e.g., from 5 groups to 6 groups) and appropriately increasing the size of the bit array (e.g., from 10n to 11n).

[0089] By dynamically adapting configuration parameters to load, the false positive rate of the filtering module can be effectively reduced under different load scenarios, the processing speed in high-throughput scenarios can be improved, the false positive rate in write-intensive scenarios can be reduced, and the load adaptability of the caching system can be significantly enhanced.

[0090] In some embodiments, the dynamic adjustment of the reuse probability threshold in the prediction module based on the performance monitoring data of the local node in S204 above may, in specific implementation, include: If the number of misses per thousand instructions is higher than the second miss threshold, it is determined to be a high miss scenario, and the reuse probability threshold is lowered. If the number of misses per thousand instructions is lower than the third miss threshold, it is determined to be a low miss scenario, and the reuse probability threshold is increased. If the fluctuation range of the read / write request ratio is detected to be higher than the fluctuation range threshold, it is determined to be an unstable load scenario, the reuse probability threshold is kept unchanged and the adjustment interval is extended; Wherein, the second miss threshold is greater than or equal to the first miss threshold and greater than or equal to the third miss threshold.

[0091] Specifically, the second miss threshold (e.g., 15), the first miss threshold (e.g., 10), and the third miss threshold (e.g., 5) cover the entire MPKI range. The specific adjustment logic for the above reuse probability thresholds can be as follows: High miss scenario: MPKI is higher than the second miss threshold, the cache hit rate is extremely low, the reuse probability threshold is lowered (e.g., from 10 to 01), the judgment criteria for high reuse data are relaxed, and the false elimination of high-value data is reduced.

[0092] Low miss scenario: MPKI is below the third miss threshold, the cache hit rate is high, the reuse probability threshold is increased (e.g., from 01 to 10), the judgment criteria are tightened, low-value data is eliminated more quickly, and cache space is freed up.

[0093] In scenarios with unstable load: If the fluctuation range of the read / write request ratio exceeds the fluctuation range threshold (e.g., 20% / 100 cycles), keep the reuse probability threshold unchanged and extend the adjustment interval from 100 cycles to 200 cycles to prevent frequent parameter fluctuations from causing strategy oscillation.

[0094] By precisely matching the reuse probability threshold with the load status, the fluctuation range of cache miss rate under mixed load can be effectively controlled, thereby improving system performance stability.

[0095] In some embodiments, configuration parameters and / or reuse probability thresholds in the prediction module can be dynamically adjusted based on the load adaptive threshold engine. The load adaptive threshold engine is the intelligent hub of the system, achieving precise adjustment through a three-level control loop. The first-level sensing loop continuously monitors PMC (Performance Monitoring Counter) metrics (sampled every 100 cycles), which can include multi-dimensional features such as MPKI, read / write ratio, and access locality. The second-level analysis loop uses a hardware finite state machine to calculate the load fingerprint in real time, dynamically adjusting Bloom filter parameters (bit array size m, number of hash functions k) and CMT reuse probability thresholds to balance false positive rate and throughput. For example, if MPKI > 10 and write ratio < 30%, it is determined to be an AI load, automatically relaxing the Bloom filter conditions and lowering the CMT retention threshold. The third-level execution loop distributes the adjustment parameters to each module via a dedicated bus, with a latency of < 15 cycles. For sudden load changes, the engine also has a fast response channel, capable of hot-switching key parameters within 5 cycles.

[0096] In some embodiments, dynamic threshold judgment can also introduce lightweight AI models (such as decision trees) to replace direct monitoring by PMC, which can be applied to edge computing scenarios (the difference is that it is trained offline and has no online updates).

[0097] In some embodiments, the load-adaptive threshold engine can achieve precise control through a multi-dimensional collaborative mechanism: First, a load type adaptive matrix is ​​constructed, and differentiated strategies are preset for typical scenarios such as AI training (high spatial locality) and database OLTP (random access). For example, when a database load characteristic (MPKI=15) is detected, the reuse threshold is automatically reduced to 01 (i.e., if the counter ≥ 01, it is retained) to prevent cold indexes from being prematurely eliminated. Under scientific computing loads (MPKI=5), a strict threshold of 10 is maintained to ensure that only highly reused data resides. For write-intensive loads, when the proportion of dirty blocks exceeds 30%, an asynchronous batch write-back mechanism is immediately initiated to reduce bus contention by merging write requests, while prioritizing the release of clean blocks (zero write-back overhead) to quickly reclaim resources, and a delayed write-back strategy is adopted for dirty blocks. The entire adjustment process employs a sophisticated closed-loop control mechanism. The system samples PMC data in real time at 1000 clock cycles and calculates two key load characteristic indicators: Read Intensity Ratio (RIR = number of read requests / total number of requests) and Working Set Change Rate based on MPKI change rate (characterized by the relative change rate of MPKI ΔMPKI). The core of the control engine is a threshold-based state machine, whose decision logic is as follows: Figure 5 As shown: The engine compares the calculated RIR and ΔMPKI with preset thresholds to determine the current load scenario state of the system. Based on the mapping table, the engine generates explicit parameter adjustment instructions. For example, when RIR ≥ 0.7 and -5% ≤ ΔMPKI ≤ +5%, the system is classified as "stable read-intensive," meaning the working set is stable and primarily uses sequential / continuous reads. Typical scenarios include AI inference and streaming media. In this case, the control instructions are: increase the number of hash functions k in the Bloom filter by 1 (approaching k=7), increase its bit array size m by 2n (approaching m=12n), and simultaneously increase the CMT reuse threshold by 1 unit. The goal is to minimize false positives by increasing k and m, protecting more data in the cache, and increasing the CMT threshold to prevent high-value reads from being evicted. To prevent frequent parameter jitter, all adjustments are made in a "stepped" manner; for example, each parameter adjustment is limited in magnitude and has an adjustment interval to prevent frequent parameter jitter and system performance oscillations caused by slight load fluctuations. After each adjustment, the system will ensure that each parameter is controlled within its physically permissible upper and lower limits (e.g., k is always between 3 and 8).

[0098] When ΔMPKI > +20% or ΔMPKI < -20%, the system is classified as "drastic change type," indicating a drastic switch or replacement of the working set. Typical scenarios include starting a new application or performing a database index scan. In this case, the control instructions are: reduce the number of hash functions k in the Bloom filter by 1 (approaching k=3), reduce its bit array size m by 1n (approaching m=9n), and simultaneously reduce the CMT reuse threshold by 2 units. The goal is to achieve rapid adaptation, prevent contamination, reduce k to decrease hash computation latency, ensure fast response, and significantly reduce the CMT threshold to accelerate the eviction of old data, making room for the new working set.

[0099] When RIR ≤ 0.3, the system is classified as "write-intensive," meaning the load is dominated by a large number of write operations. Typical scenarios include database transaction commits and large-scale data writes. In this case, the control instructions are: keep the number of hash functions k in the Bloom filter unchanged (i.e., 0) (maintaining k=5), reduce its bit array size m by 1n (approaching m=9n), and simultaneously reduce the CMT reuse threshold by 3 units. The goal is to optimize write-back, maintain throughput, keep the Bloom parameters neutral, and, most importantly, significantly reduce the CMT threshold, coordinating with the asynchronous write-back mechanism to prioritize evicting Clean blocks and quickly return to cached resources.

[0100] When any of the above conditions are not met (typically 0.3 < RIR < 0.7 and -20% < ΔMPKI < +20%), the system is classified as "hybrid balanced," meaning it involves mixed read and write operations with gradual changes in the working set. Typical scenarios include general-purpose computing and web services. In this case, the control instructions are: keep the number of hash functions k in the Bloom filter unchanged (i.e., 0) (maintaining k=5), keep its bit array size m unchanged (approaching m=10n), and keep the CMT reuse threshold unchanged. The goal is to achieve balanced stability by using globally optimized default parameters to achieve the best balance between hit rate, latency, and overhead.

[0101] Through this explicit strategy of quantifying load characteristics, looking up tables, and making step-by-step adjustments, the system can transform the fuzzy "intelligent adaptation" into a deterministic hardware control flow. This ensures high throughput while precisely suppressing the false positive rate of the Bloom filter below the first preset false positive rate threshold, ultimately achieving stable and high-performance performance with minimal fluctuations in cache hit rate under diverse load scenarios.

[0102] In some embodiments, the process in S205 above, which combines the reuse probability value, the reuse probability threshold, and the source access node identifier to assign a replacement strategy for the last-level cache of the local node to the data block, may include, in specific implementations: If the reuse probability value is higher than the reuse probability threshold and the source access node identifier identifies the local node, then a first type of management strategy for long-term retention is assigned. If the reuse probability value is lower than the reuse probability threshold or the source access node identifier identifies a remote node, then a second type of management strategy for rapid eviction is assigned.

[0103] Specifically, if the source access node identifier matches the ID of the current access node (e.g., current node ID = 0, source identifier = 0), it is a local node; otherwise, it is a remote node (which can include nodes in the same slot, across slots, or across motherboards). The allocation logic for the above replacement strategy can be as follows: Local high-reusability data: If the reuse probability value is higher than the reuse probability threshold and the source access node identifier identifies the local node, the first type of management policy (long-term retention) is assigned, and LLC Cache resources are used preferentially.

[0104] Remote or low-reuse data: If the reuse probability value is lower than the reuse probability threshold, or if the source access node identifier identifies the remote node, a second type of management policy (fast eviction) is assigned to reduce LLC Cache resource consumption.

[0105] By combining the reuse probability value, the reuse probability threshold, and the source access node identifier, a replacement strategy for the last-level cache of the local node can be assigned to the data block. This enables topology-aware differentiated strategy allocation, improves the retention rate of highly reused local data, solves the problem of low-value data from the remote end crowding out local cache resources, and reduces the frequency of cross-node access.

[0106] In some embodiments, the first type of management strategy is a least recently used strategy, which can be used to insert the data block into the least recently used end of the linked list maintained by the strategy; the second type of management strategy is a static rereference interval prediction strategy, which can be used to associate a rereference prediction value with the data block and initialize the rereference prediction value to a preset maximum value to indicate that the data block has the lowest recent reuse priority.

[0107] Specifically, the first management strategy (LRU strategy) involves maintaining an LRU linked list in the LLC cache. Data blocks are inserted at the most recently used end of the list (hot end, the first 20%). Each time a data block is accessed, it is automatically moved to the head of the list. When the list is full, the least recently accessed data block is evicted from the tail (cold end), ensuring that highly reused data is retained for a long time.

[0108] The second management strategy (SRRIP strategy) associates a 3-bit Recall Predicted Value (RRPV) with a data block, initialized to the preset maximum value of 3, indicating that the data block has the lowest recent reuse priority. When the data block is not accessed, the RRPV remains or increases; when it is accessed, the RRPV is reset to 0, increasing its priority. Data blocks with RRPV=3 are preferentially selected for eviction.

[0109] The two strategies mentioned above complement each other. The LRU strategy ensures the long-term residence of highly reused local data, while the SRRIP strategy accelerates the eviction of remote / low-reuse data, thereby improving the resource utilization of the LLC Cache and further reducing data access latency.

[0110] In some embodiments, the policy replacement and execution module can assign a replacement policy for the last-level cache of the local node to the data block. This policy combines the advantages of LRU and SRRIP algorithms, and guides L3 replacement decisions based on CMT output. For blocks marked as high reuse (counter ≥ 10) by CMT, the system allocates LRU management area resources to retain them in the cache as much as possible to reduce future access misses.

[0111] See Figure 6As shown, the LRU strategy effectively preserves recently accessed blocks by maintaining a time order (most recently used at the head, least recently used at the tail). Specifically, when a new data block is determined to be "highly reusable data," i.e., a newly inserted high-reusability block, the LRU strategy initialization operation is executed—the data block is inserted at the head of the LRU queue (the most recently used end), marking it as the highest priority cache block and the starting point of the LRU strategy. When the data block is accessed again by the kernel (hit), the LRU queue state update operation is triggered—the data block is moved from its current position to the head of the queue, continuously maintaining its high priority and ensuring that frequently accessed data is always kept in the cache, avoiding accidental eviction. When the last-level cache capacity reaches saturation, the LRU queue eviction operation is triggered—a data block is selected from the tail of the queue (the least recently used end) for eviction; this area stores low-priority data that has not been accessed for a long time.

[0112] Low-reuse blocks (counter ≤ 01) are included in the SRRIP management area. See also... Figure 7 As shown, when a data block is determined to be "low-reuse data," i.e., a newly inserted low-reuse block, the SRRIP policy initialization operation is executed—the Reference Predicted Value (RRPV) of the data block is marked to the preset maximum value "3," indicating that it has the lowest recent reuse priority, which is the starting point for the SRRIP policy. If the data block is not accessed (missed): the RRPV state machine triggers the "RRPV++" operation, incrementing the RRPV value (e.g., from 2 to 3), continuously reducing its reuse priority; if the data block is accessed (missed): the RRPV state machine triggers the "RRPV=0" operation, resetting the RRPV value to the minimum value "0," temporarily increasing its reuse priority to prevent high-reuse data from being mistakenly evicted. If the RRPV value increases to the maximum value "3," the data block is determined to be "not likely to be reused in the near future," and enters the "immediate eviction" process. If the RRPV value is reset to "0," the data block is determined to be "recently accessed," retained in the last-level cache, and its priority is subsequently updated through the RRPV state machine. The SRRIP strategy achieves rapid eviction by setting a high Reference Interval Prediction Value (RRPV), causing these blocks to be evicted after only a few misses, thus freeing up space for potentially more important blocks.

[0113] By dynamically dividing the management area using CMT's 2-bit reuse counter, and combining the advantages of two classic algorithms—LRU for efficient retention of hot data and SRRIP for rapid eviction of cold data—the overall caching efficiency is improved.

[0114] In some embodiments, the SRRIP eviction module may integrate an ARC (Adaptive ReplacementCache) policy to replace LRU to enhance adaptability (the difference is that ARC requires additional metadata).

[0115] In some embodiments, after S205 above, the following may also be included: When implementing the replacement policy for the last-level cache, priority is given to evicting data blocks managed by the second type of management policy whose re-reference prediction value has reached its maximum value; If eviction is still necessary, then eviction will be carried out on data blocks that are at the cold end of the least recently used list and are managed by the first type of management strategy.

[0116] Specifically, when performing eviction operations, the LLC Cache follows a "tiered eviction" rule: First, it traverses all data blocks managed by the SRRIP policy, prioritizing the eviction of data blocks with RRPV=3 (the maximum value). If sufficient cache space is still not released after eviction, the LRU list is traversed, and the least accessed data block in the cold end (last 20%) is evicted. During the eviction process, if the status flag of the data block to be evicted is "Dirty" (modified), the data is first written back to memory before the eviction operation is performed to avoid data loss.

[0117] By precisely targeting low-value data with the eviction logic, the probability of highly reused data being mistakenly evicted is reduced, cache space release efficiency is improved, and LLC Cache always reserves sufficient resources for high-value data.

[0118] In some embodiments, after S205 above, the following may also be included: If, within a preset statistical period, the frequency of access to remote access node data blocks by this node exceeds a preset access threshold, a data migration request will be triggered. The node routing module migrates the data block from the last-level cache of the source access node to the last-level cache of this node, and synchronously updates the table entries of the prediction module of this node.

[0119] Specifically, if the frequency of access to a data block from a remote access node by this node exceeds a preset access threshold (e.g., 4 times / 100 cycles) within a preset statistical period, it indicates that the data block is frequently accessed locally, directly triggering a data migration request. The migration request is sent to the source access node via the node routing module. Upon receiving the request, the source access node reads the target data block from its LLC Cache and transmits it to this node. This node writes the received data block into its own LLC Cache and simultaneously creates or updates the corresponding entry in the CMT (updating the source access node identifier to the local node ID and initializing or adjusting the saturation counter). The source access node then sets the valid bit of the CMT entry for that data block to 0, indicating that the entry is invalid.

[0120] After migrating frequently accessed remote data to the local machine, the access latency is reduced from cross-node latency to local latency. The reduction in cross-node access latency significantly improves the overall data access efficiency of the system.

[0121] In some embodiments, the cache block metadata extension design specifically supports the efficient collaborative operation of hybrid replacement strategies in the cache block metadata storage structure. Each cache block adds three key metadata fields: First, a 2-bit RRPV (Re-Reference Prediction Value) field, used by the SRRIP strategy to predict the time interval for the next access to the data block; a larger value indicates a further time before the next access. Second, an LRU linked list pointer field, using a doubly linked list pointer, used to accurately locate the predecessor and successor positions of the block in the LRU-maintained list. Finally, a 1-2 bit policy flag, used to explicitly identify whether the current block is managed by the LRU or SRRIP strategy. This metadata extension adds only about 5% storage overhead in hardware implementation, yet supports complex policy collaboration mechanisms.

[0122] In some embodiments, the intelligent policy allocation when a new block is inserted can be as follows: When a new data block is inserted into the cache, the system queries the reuse counter value of the block recorded in the CMT (Cross-Layer Metadata Table) and makes an intelligent policy allocation decision accordingly. Specifically: if the CMT shows that the reuse counter value of the block is 10 or 11 (indicating a high reuse probability), the block is marked as managed under the LRU policy and immediately inserted into the hottest end (MRU position) of the LRU list to ensure its long-term residence. Conversely, if the reuse counter is 00 or 01 (indicating a low reuse probability), it is marked as managed under the SRRIP policy, and its RRPV is initialized to the maximum value of 3, indicating that the block is predicted not to be accessed again in the near future. This prediction-based initial allocation mechanism effectively achieves rapid separation of hot and cold data.

[0123] In some embodiments, the dynamic handling mechanism for cache hits can be as follows: When a cache hit occurs, the system adopts differentiated processing methods based on the current management policy of the hit block. For LRU-managed blocks, they are immediately moved to the hottest end of the LRU list, and the last access timestamp is updated (implemented by a hardware cycle counter), while maintaining their RRPV value. For SRRIP-managed blocks, the RRPV is reset to 0, indicating that the block has just been accessed. The system also intelligently monitors the hit pattern of SRRIP blocks. When the number of consecutive hits exceeds a preset threshold (e.g., 2 times), a policy switch is triggered, converting the block to LRU management and inserting it at the head of the LRU list. This dynamic upgrade mechanism ensures that potentially hot data can be identified in a timely manner and its management level upgraded.

[0124] In some embodiments, the priority of cache eviction processing can be as follows: When data blocks need to be evicted to free up space, the system adopts a tiered eviction strategy to maximize cache utility. First, it prioritizes searching for the block with the highest RRPV value (i.e., the block predicted to be least likely to be accessed again) among the SRRIP-managed blocks. If multiple candidates exist, a FIFO or random selection mechanism is used. When SRRIP blocks are insufficient, the system checks the RRPV values ​​of all SRRIP blocks. If none reach the maximum value, they are all aged out (RRPV value incremented by 1). Finally, when it is indeed necessary to evict an LRU-managed block, the system selects the least accessed block at the tail of the LRU list and initiates a write-back process for the dirty block. This tiered eviction mechanism ensures both the rapid release of cold data and avoids the erroneous eviction of hot data.

[0125] In some embodiments, dynamic policy adjustment and optimization can specifically involve the system continuously optimizing through periodic policy evaluation. All SRRIP-managed blocks undergo periodic RRPV aging (the value is incremented by 1 but does not exceed the maximum value) to simulate the impact of time on data popularity. Simultaneously, the system monitors the usage of the tail blocks of the LRU list, downgrading "pseudo-hot" data that has not been accessed for a long time to SRRIP management (RRPV set to the maximum value), freeing up valuable cache space occupied by these blocks for truly high-value data. This bidirectional policy switching mechanism, combined with fine-grained RRPV adjustment, enables cache management to dynamically adapt to various complex workload characteristics.

[0126] This sophisticated cache management solution achieves optimal utilization of cache resources through the synergy of five key aspects: metadata expansion, intelligent initial allocation, differentiated hit processing, tiered eviction mechanism, and dynamic policy adjustment. It ensures high performance while keeping hardware overhead within a reasonable range.

[0127] In some embodiments of a NUMA architecture, the latency for different nodes to access local and remote memory differs significantly (50-100ns for local nodes versus 150-300ns for remote nodes). Therefore, when replacing cached data, it is necessary to consider the NUMA node where the data resides (recorded via the CMT Node ID field), giving local node data a higher retention priority, while accelerating the eviction of remote node data that may result in high-latency access. This achieves fine-grained topology-aware management, and combined with differentiated retention strategies, significantly reduces the frequency of cross-node access.

[0128] A NUMA-based topology-aware differentiated cache management strategy achieves high-performance cache optimization through a three-level mechanism: For highly reusable blocks on local nodes (CMT reuse counter 10 or 11), an enhanced LRU management strategy is adopted. Through hardware-level operations, these blocks are inserted into the first 20% of the LRU list (calculation formula: insertion position = total list length × 0.2). Each time a hit occurs, the block moves up a certain percentage (e.g., 24%) in the list to accelerate its migration to the hot end (MRU end). In actual testing, this strategy significantly extends the average lifetime of local hot data (an improvement of approximately 35% was observed under specific loads) while also helping to reduce the frequency of cross-node access. For low-reusable blocks on remote nodes (reuse counter 00 or 01), an aggressive eviction strategy is adopted. These blocks are placed in the SRRIP queue and given the highest eviction priority (RRPV initial value = 3). Simultaneously, a secondary sorting based on topological distance ensures that remote cold data is released first. Furthermore, this solution reserves space for technological evolution, including an adaptive mechanism (5%-30% fluctuation range) that can dynamically adjust local weights based on bus congestion, and a dynamic hot-cold data migration strategy that migrates frequently accessed remote blocks to local nodes. Through refined weight allocation and a topology-aware eviction mechanism, cache utilization efficiency under mixed loads is significantly improved while ensuring low hardware overhead.

[0129] The above solution can achieve the following technical effects: 1. Performance Improvement: Under mixed loads (such as AI + database), this solution can significantly improve the L3 cache hit rate. Compared with existing hierarchical awareness solutions, the performance fluctuation range is greatly reduced, and the system operation stability is substantially improved (comparable to the CHAR algorithm).

[0130] 2. Hardware overhead optimization: The design adopts a combination of lightweight probability filtering and simplified counter (Bloom + 2-bit counter), which results in a hardware area increase that is much lower than existing solutions based on machine learning or complex statistical models, and has excellent compatibility with commercial server chips.

[0131] 3. Reduced latency: Through a deeply integrated NUMA topology-aware mechanism, the frequency of cross-node access is effectively reduced, the average memory access latency is significantly improved, and the level of data locality is comprehensively enhanced.

[0132] 4. Energy efficiency improvement: By prioritizing the elimination of clean data blocks and optimizing the write-back strategy, the amount of memory bus transactions is effectively reduced, and the overall power consumption of the system is significantly optimized.

[0133] The technical effectiveness has been verified through simulation, supporting high-concurrency scenarios such as cloud computing and AI inference. Through a load adaptive optimization mechanism, it effectively solves the problems of cross-level information fragmentation, insufficient load adaptability, and excessive hardware overhead while controlling hardware costs, significantly improving system performance and stability under mixed loads.

[0134] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. For details, please refer to the foregoing descriptions of the relevant processing embodiments; they will not be repeated here.

[0135] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0136] In a specific implementation scenario, the overall process of cache replacement can be as follows: Phase 1: L2 Destroyer Pre-processing Phase Before performing an L2 cache line eviction operation, the system first collects complete context information of the evicted data block (including physical address, access history, and necessary metadata such as NUMA node identifier) ​​to provide a basis for subsequent intelligent cache replacement strategies. This information is sent to a Bloom filter deployed locally on the current NUMA node for preprocessing. The Bloom filter performs a fast lookup and comparison in the bit array using multiple hash functions in parallel computation (typically 3-5 hardware-optimized CRC32 hash functions): if all hash positions are set to 1, the data block is determined to have potential reuse value and is forwarded to the CMT (Cross-Layer Metadata Table) for the next stage of in-depth tracing and analysis; conversely, if any hash bit is 0, it is directly determined to be short-term invalid data (such as temporary variables or streaming data) and discarded immediately. Through this probabilistic pre-screening mechanism based on the Bloom filter, the system can intercept and discard most data blocks with extremely low future access probability at an early stage with very low hardware cost, thereby significantly reducing the processing pressure on downstream complex tracing logic such as CMT. This mechanism achieves high-throughput filtering while ensuring an extremely low false positive rate.

[0137] Phase Two: L3 Intelligent Replacement Decision-Making Phase

[0138] Data blocks entering the CMT processing flow trigger dynamic updates to metadata: First, the specific CMT entry is located based on the physical address hash value, and the 20-bit simplified address tag is updated; then, the 2-bit reuse counter is adjusted based on the access pattern (incrementing on hits and decrementing on misses), and topology information such as the NUMA node ID is recorded. Simultaneously, the load monitoring module collects key metrics from the PMC (Performance Monitoring Counter) in real time (such as MPKI, write ratio, access randomness, etc.), and the dynamic threshold engine performs load characteristic analysis: when a significant increase in MPKI is detected (e.g., >15), the reuse threshold is automatically lowered (e.g., from 10 to 01) to handle random access load; when the write ratio exceeds 60%, the Bloom filter parameters are optimized (increasing the number of hash functions k from 3 to 5) to improve filtering accuracy. These dynamic parameter adjustments can be completed within 10 clock cycles, ensuring the system responds quickly to load changes.

[0139] CMT generates a joint decision vector based on data reuse intensity, NUMA node location, and data state (e.g., Clean / Dirty) by combining reuse counters and NUMA node information: For data blocks with high reuse (counter ≥ 10) and local nodes, they are marked as LRU retention candidates, inserted at the head of the LRU list, and given an additional 20% retention weight; for data blocks with low reuse (counter ≤ 01) and remote nodes, they are assigned the highest eviction priority (RRPV=3) and added to the SRRIP queue. During eviction, the system prioritizes Clean state data blocks to eliminate write-back overhead. For Dirty blocks that must be written back, the system uses routing protocols that are aware of the NUMA topology and prioritize low hop count or low congestion paths to complete the cross-node write-back. Through this mechanism, in actual testing, compared to the traditional indiscriminate replacement strategy, this scheme can significantly reduce the cross-node Dirty block write-back latency. Cross-stage collaborative optimization enables the system to achieve significant reductions in cross-NUMA node cache access latency under typical mixed loads, while keeping hardware resource overhead at a low level and maintaining performance fluctuations within a narrow range.

[0140] Cross-stage collaborative optimization

[0141] The two processing stages are tightly coupled and seamlessly integrated through a hardware pipeline: after an L2 eviction event is triggered, the filtering module completes pre-filtering in a very short time, while the prediction module employs a deeply optimized pipeline design, coupled with a real-time feedback adjustment mechanism from a dynamic threshold engine. This allows the system to significantly reduce cross-node cache access latency while maintaining extremely low hardware overhead, and to keep performance fluctuations under mixed loads within a very small range. This refined hierarchical processing mechanism effectively balances the trade-off between processing speed and decision accuracy, providing a highly efficient cache management solution for multi-core processors.

[0142] Based on the same inventive concept, this specification also provides a hierarchical collaborative cache replacement device in its embodiments, as described in the following examples. Since the principle of the hierarchical collaborative cache replacement device in solving the problem is similar to that of the hierarchical collaborative cache replacement method, the implementation of the hierarchical collaborative cache replacement device can refer to the embodiments of the hierarchical collaborative cache replacement method; repeated details will not be elaborated further. Reference Figure 8 As shown, this device can be applied to an access node, and the device may include: The first acquisition module 801 can be used to acquire the context information of the data block in response to the eviction event of the secondary cache in this node. The context information includes at least the physical address and the source access node identifier. The filtering module 802 can be used to input the physical address and source access node identifier into the filtering module within this node, and filter the data block according to the current configuration parameters; The second acquisition module 803 can be used to query and update the table entry corresponding to the data block in the prediction module within this node according to the context information if the filtering result indicates that the data block is potentially important data, so as to obtain the reuse probability value of the data block. The adjustment module 804 can be used to dynamically adjust the configuration parameters and / or the reuse probability threshold in the prediction module based on the performance monitoring data of this node. The replacement module 805 can be used to combine the reuse probability value, the reuse probability threshold and the source access node identifier to allocate a replacement strategy for the last-level cache of the local node to the data block.

[0143] In some embodiments, the context information in the first acquisition module 801 may further include access history, which may include at least the access frequency and the most recent access timestamp of the data block, for assisting in updating the table entry corresponding to the data block in the prediction module within the local node.

[0144] In some embodiments, the filtering module in the filtering processing module 802 described above may include a Bloom filter, and the configuration parameters may include the size of the bit array and the number of hash functions. Accordingly, the filtering processing module 802 may be used to jointly calculate the physical address and the source access node identifier using multiple sets of parallel hash functions to obtain multiple sets of hash results. If the bit array position corresponding to all hash results is 1, the data block is determined to be potentially important data with a high probability. If the bit array position corresponding to any hash result is 0, the data block is determined to be short-term invalid data and is directly discarded.

[0145] In some embodiments, the prediction module in the second acquisition module 803 described above may include a cross-layer metadata table, the table entries of which may include at least: The physical address hash value is used to match and index the table entries corresponding to the data block; A saturation counter, the value of which is used to quantify the reuse probability of a data block, and the value of which is dynamically updated based on the access history of the data block; Source access node identifier, used to identify the initial home node of a data block and to assist in matching table entries; The status flag is used to identify the current status of the data block.

[0146] In some embodiments, the second acquisition module 803 described above can be specifically used to perform hash calculation on the physical address to obtain a physical address hash value; combine the physical address hash value with the source access node identifier to query the corresponding table entry in the cross-layer metadata table; if a corresponding table entry is found, the saturation counter is adjusted according to the access frequency; if no corresponding table entry is found, a new table entry is created, the physical address hash value, the source access node identifier and the status flag are written into the table entry, and the value of the saturation counter is initialized.

[0147] In some embodiments, the performance monitoring data in the adjustment module 804 may include the number of misses per thousand instructions, the ratio of read to write requests, and access locality characteristics. Accordingly, the adjustment module 804 may be specifically used to determine a high throughput load if the number of misses per thousand instructions is higher than a first miss threshold and the write request ratio is lower than a write ratio threshold, thereby reducing the bit array size and the number of hash functions; if the write request ratio is higher than the write ratio threshold, thereby determining a write-intensive load, thereby increasing the bit array size and the number of hash functions; and if the access locality characteristic is that the proportion of random access is higher than the random access threshold, thereby determining a random access load, thereby increasing the number of hash functions and the bit array size.

[0148] In some embodiments, the adjustment module 804 may further be used to determine a high miss scenario and lower the reuse probability threshold if the number of misses per thousand instructions is higher than the second miss threshold; determine a low miss scenario and raise the reuse probability threshold if the number of misses per thousand instructions is lower than the third miss threshold; and determine an unstable load scenario if the fluctuation range of the read / write request ratio is higher than the fluctuation range threshold, maintain the reuse probability threshold unchanged and extend the adjustment interval if the load is unstable. The second miss threshold is greater than or equal to the first miss threshold and greater than or equal to the third miss threshold.

[0149] In some embodiments, the replacement module 805 can be specifically used to allocate a first type of management strategy for long-term retention if the reuse probability value is higher than the reuse probability threshold and the source access node identifier identifies a local node; and to allocate a second type of management strategy for rapid elimination if the reuse probability value is lower than the reuse probability threshold or the source access node identifier identifies a remote node.

[0150] In some embodiments, the first type of management strategy in the replacement module 805 is a least recently used strategy, which is used to insert the data block into the least recently used end of the linked list maintained by the strategy; the second type of management strategy is a static rereference interval prediction strategy, which is used to associate a rereference prediction value with the data block and initialize the rereference prediction value to a preset maximum value to indicate that the data block has the lowest recent reuse priority.

[0151] In some embodiments, after the replacement module 805, it can also be used to prioritize the elimination of data blocks managed by the second type of management strategy and whose re-reference prediction value has reached the maximum value when executing the replacement strategy of the last-level cache; if elimination is still necessary, then eliminate data blocks managed by the first type of management strategy that are at the cold end of the least recently used list.

[0152] In some embodiments, after the replacement module 805, it can also be used to trigger a data migration request if the access frequency of the local node core to the remote access node data block exceeds a preset access frequency threshold within a preset statistical period; and to migrate the data block from the last-level cache of the source access node to the last-level cache of the local node through the node routing module, and synchronously update the table entries of the local node prediction module.

[0153] As can be seen from the above, the hierarchical collaborative cache replacement device provided in the embodiments of this specification generates load feature fingerprints in real time based on hardware performance counters (PMC) to achieve threshold adaptation. Deeply integrating the source access node identifier (NUMANode ID) into the prediction module's CMT entry enables globally topology-aware L3 retention decisions. The simplified combination of the filtering module (Bloom filter) and a 2-bit saturation counter optimizes the false positive rate and throughput. Real-time updating of the CMT using L2 eviction flows overcomes the limitations of static CHAR classification, and combining it with group-associative CMT improves prediction accuracy.

[0154] This specification also provides an embodiment of a chip, such as... Figure 9 As shown, the chip may include a processing unit 901 and a storage unit 902, wherein the processing unit 901 and the storage unit 902 can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0155] The processing unit 901 can be a central processing unit (CPU). The processing unit 901 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0156] Storage unit 902, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the data prefetching method in the embodiments of the present invention. Processing unit 901 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in storage unit 902, thereby implementing the data prefetching method in the above method embodiments.

[0157] Storage unit 902 may include a stored program area and a stored data area. The stored program area may store the operating system and applications required for at least one function; the stored data area may store data created by processor 901, etc. Furthermore, storage unit 902 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, storage unit 902 may optionally include memory remotely located relative to processing unit 901, and this remote memory may be connected to processing unit 901 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0158] The one or more modules are stored in the storage unit 902, and when executed by the processing unit 901, they perform the following: Figure 2 The illustrated hierarchical collaborative cache replacement method.

[0159] For details regarding the chip, please refer to the relevant descriptions and effects in the above method embodiments; they will not be repeated here.

[0160] In some embodiments, the processing unit may include at least a filtering module, a prediction module, a load adaptive threshold engine, and a policy replacement and execution module. The filtering module is used to filter data blocks according to the current configuration parameters. The prediction module is used to output the reuse probability value of the data block. The load adaptive threshold engine is used to dynamically adjust the configuration parameters and / or the reuse probability threshold in the prediction module. The policy replacement and execution module is used to assign a replacement policy for the last-level cache of the local node to the data block.

[0161] This specification also provides a computer storage medium storing computer program instructions that, when executed, implement the steps of the above-described hierarchical collaborative cache replacement method.

[0162] This specification also provides a computer program product comprising a computer program that, when executed, implements the steps of the above-described hierarchical collaborative cache replacement method.

[0163] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0164] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. The focus of each embodiment is to describe the differences from other embodiments.

[0165] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

[0166] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0167] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute certain parts of the methods of various embodiments of this application.

[0168] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer chips, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0169] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0170] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to the embodiments described herein by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A hierarchical collaborative cache replacement method, characterized in that, Applied to access nodes, including: In response to an eviction event of a data block by the secondary cache within this node, the context information of the data block is obtained, and the context information includes at least the physical address and the source access node identifier; Input the physical address and source access node identifier into the filtering module within this node, and filter the data blocks according to the current configuration parameters; If the filtering result indicates that the data block is potentially important data, then the table entry corresponding to the data block in the prediction module within this node is queried and updated according to the context information to obtain the reuse probability value of the data block; Based on the performance monitoring data of this node, dynamically adjust the configuration parameters and / or the reuse probability threshold in the prediction module; By combining the reuse probability value, the reuse probability threshold, and the source access node identifier, a replacement strategy for the last-level cache of this node is assigned to the data block.

2. The method according to claim 1, characterized in that, The context information also includes access history, which includes at least the access frequency of the data block and the most recent access timestamp, and is used to assist in updating the table entries corresponding to the data block in the prediction module within the current node.

3. The method according to claim 1, characterized in that, The filtering module includes a Bloom filter, and the configuration parameters include the bit array size and the number of hash functions; correspondingly, filtering the data block according to the current configuration parameters includes: Multiple sets of hash results are obtained by jointly calculating the physical address and the source access node identifier using multiple sets of parallel hash functions. If all bit array positions corresponding to all hash results are 1, then the data block is determined to be potentially important data with a high probability. If any bit array position corresponding to any hash result is 0, the data block is determined to be short-term invalid data and is discarded directly.

4. The method according to claim 1, characterized in that, The prediction module includes a cross-layer metadata table, and the table entries include at least: The physical address hash value is used to match and index the table entries corresponding to the data block; A saturation counter, the value of which is used to quantify the reuse probability of a data block, and the value of which is dynamically updated based on the access history of the data block; Source access node identifier, used to identify the initial home node of a data block and to assist in matching table entries; The status flag is used to identify the current status of the data block.

5. The method according to claim 4, characterized in that, The step of querying and updating the table entry corresponding to the data block in the prediction module within this node based on the context information includes: Perform a hash calculation on the physical address to obtain the physical address hash value; By combining the physical address hash value with the source access node identifier, the corresponding entry is queried in the cross-layer metadata table; If a corresponding table entry is found, the saturation counter is adjusted according to the access frequency. If no corresponding table entry is found, a new table entry is created, and the physical address hash value, source access node identifier, and status flag are written into the table entry, and the value of the saturation counter is initialized.

6. The method according to claim 1, characterized in that, The performance monitoring data includes the number of misses per thousand instructions, the ratio of read to write requests, and access locality characteristics; correspondingly, based on the performance monitoring data of this node, the configuration parameters are dynamically adjusted, including: If the number of misses per thousand instructions is higher than the first miss threshold and the write request ratio is lower than the write ratio threshold, it is determined to be a high throughput load, and the bit array size and the number of hash functions are reduced. If the write request ratio is detected to be higher than the write ratio threshold, it is determined to be a write-intensive load, and the bit array size is increased and the number of hash functions is increased. If the access locality characteristic is detected as random access ratio exceeding the random access threshold, it is determined to be random access load, and the number of hash functions is increased and the bit array size is increased.

7. The method according to claim 6, characterized in that, The dynamic adjustment of the reuse probability threshold in the prediction module based on the performance monitoring data of this node includes: If the number of misses per thousand instructions is higher than the second miss threshold, it is determined to be a high miss scenario, and the reuse probability threshold is lowered. If the number of misses per thousand instructions is lower than the third miss threshold, it is determined to be a low miss scenario, and the reuse probability threshold is increased. If the fluctuation range of the read / write request ratio is detected to be higher than the fluctuation range threshold, it is determined to be an unstable load scenario, the reuse probability threshold is kept unchanged and the adjustment interval is extended; Wherein, the second miss threshold is greater than or equal to the first miss threshold and greater than or equal to the third miss threshold.

8. The method according to claim 1, characterized in that, The step of combining the reuse probability value, the reuse probability threshold, and the source access node identifier to assign a replacement strategy for the last-level cache of the local node to the data block includes: If the reuse probability value is higher than the reuse probability threshold and the source access node identifier identifies the local node, then a first type of management strategy for long-term retention is assigned. If the reuse probability value is lower than the reuse probability threshold or the source access node identifier identifies a remote node, then a second type of management strategy for rapid eviction is assigned.

9. The method according to claim 8, characterized in that, The first type of management strategy is the least recently used strategy, which is used to insert the data block into the least recently used end of the linked list maintained by the strategy; the second type of management strategy is the static rereference interval prediction strategy, which is used to associate a rereference prediction value with the data block and initialize the rereference prediction value to a preset maximum value to indicate that the data block has the lowest recent reuse priority.

10. The method according to claim 1, characterized in that, The method further includes: When implementing the replacement policy for the last-level cache, priority is given to evicting data blocks managed by the second type of management policy whose re-reference prediction value has reached its maximum value; If eviction is still necessary, then eviction will be carried out on data blocks that are at the cold end of the least recently used list and are managed by the first type of management strategy.

11. The method according to claim 1, characterized in that, The method further includes: If, within a preset statistical period, the frequency of access to remote access node data blocks by this node exceeds a preset access threshold, a data migration request will be triggered. The node routing module migrates the data block from the last-level cache of the source access node to the last-level cache of this node, and synchronously updates the table entries of the prediction module of this node.

12. A hierarchical collaborative cache replacement device, characterized in that, Applied to access nodes, including: The first acquisition module is used to acquire the context information of the data block in response to the eviction event of the secondary cache within the local node. The context information includes at least the physical address and the source access node identifier. The filtering module is used to input the physical address and source access node identifier into the filtering module within this node, and filter the data blocks according to the current configuration parameters; The second acquisition module is used to query and update the table entry corresponding to the data block in the prediction module within this node according to the context information if the filtering result indicates that the data block is potentially important data, so as to obtain the reuse probability value of the data block. The adjustment module is used to dynamically adjust the configuration parameters and / or the reuse probability threshold in the prediction module based on the performance monitoring data of this node. The replacement module is used to combine the reuse probability value, the reuse probability threshold, and the source access node identifier to assign a replacement strategy for the last-level cache of the current node to the data block.

13. A chip, characterized in that, It includes a processing unit and a storage unit for storing instructions executable by the processing unit, wherein the processing unit, when executing the instructions, implements the steps of the method according to any one of claims 1 to 11.

14. The chip according to claim 13, characterized in that, The processing unit includes at least a filtering module, a prediction module, a load adaptive threshold engine, and a policy replacement and execution module. The filtering module is used to filter data blocks according to the current configuration parameters. The prediction module is used to output the reuse probability value of the data block. The load adaptive threshold engine is used to dynamically adjust the configuration parameters and / or the reuse probability threshold in the prediction module. The policy replacement and execution module is used to assign a replacement policy for the last-level cache of the current node to the data block.

15. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the method according to claims 1 to 11.

16. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the method described in claims 1 to 11.

Images