Memory access optimization method based on intelligent cache management

Abstract

The invention discloses a memory access optimization method based on intelligent cache management, and relates to the technical field of computer storage. According to the method, spatial-temporal characteristics and semantic association data of memory access requests are collected in real time, a dynamic heat matrix is constructed, and a multi-dimensional access rule is fused to improve modeling precision. And inputting the dynamic popularity matrix into a hybrid prediction model, predicting a future access probability by using a time convolutional network, analyzing a competition relationship between data blocks through a graph attention network, generating a conflict pre-judgment weight and a corrected popularity ranking, and effectively reducing the cache jitter risk. On the basis of popularity ranking and conflict weight, a fragmented reinforcement learning algorithm is adopted to divide logic sub-regions, a differential reward function is designed to dynamically decide cache operation, and performance and energy efficiency requirements are balanced; and finally, through an online learning mechanism, combining real-time feedback to dynamically adjust a prediction model weight and strategy parameters, forming a closed-loop optimization link, and realizing adaptive stability under long-term load fluctuation.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, and in particular to a memory access optimization method based on intelligent cache management. Background Art

[0002] In today's technological wave of rapid development of big data, artificial intelligence and cloud computing, the demand for computing efficiency in data-intensive applications (such as real-time recommendation systems, autonomous driving, high-concurrency financial transactions, etc.) is growing exponentially. Memory access speed has become a key bottleneck restricting computing performance. Especially in the context of the gradual failure of Moore's Law and the limited improvement of hardware performance, optimizing memory access efficiency through the software layer has become a common focus of academia and industry. At the same time, the popularity of edge computing and IoT devices has further amplified the demand for low latency and high energy efficiency. Cache management is the core hub connecting computing units and storage systems, and its optimization effect directly affects system throughput, response latency and energy consumption. Traditional cache strategies are gradually exposing their limitations in the face of dynamic, heterogeneous and unpredictable modern workloads, and it is urgent to introduce intelligent methods to achieve adaptive optimization.

[0003] The current mainstream cache management technologies are mostly based on static rules or locality assumptions, which are difficult to cope with dynamic load characteristics and complex data access patterns. For example, traditional replacement strategies are prone to cause cache jitter under bursty workloads or long-tail access distributions, resulting in a sharp drop in hit rate; while prefetching technology relies on fixed pattern predictions, has poor adaptability to sparse or non-continuous access sequences, and is prone to bandwidth waste. In addition, although cache optimization methods based on machine learning have potential, they generally have problems such as high computational overhead, large online learning delays, and insufficient model generalization capabilities. In heterogeneous computing architectures, existing methods lack the ability to collaboratively manage cache resources across hardware levels, making it difficult to balance data locality and consistency requirements between different computing units. Therefore, there is an urgent need for a lightweight, adaptive, and cross-platform scalable intelligent cache management method to break through the bottleneck of existing technologies. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a memory access optimization method based on intelligent cache management, which solves the problems of the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: A memory access optimization method based on intelligent cache management, comprising the following steps: S1. By collecting the spatio-temporal characteristics and semantic association data of memory access requests in real time, a dynamic heat matrix is constructed, where the spatio-temporal characteristics include the spatial position offset and time access interval distribution of data blocks, and the semantic association data extracts the context dependence relationship through a lightweight pre-trained model, and generates short-term and long-term access trend scores based on sliding window statistics; S2. The dynamic heat matrix is input into a hybrid prediction model, the time convolutional network is used to predict the access probability within the future time window, and the competition relationship between data blocks is analyzed based on the graph attention network to generate a conflict prediction weight and a corrected heat ranking; S3. According to the heat ranking and the conflict prediction weight, the cache is divided into logical sub-regions through a sharded reinforcement learning algorithm, and different reward functions are designed for different sub-regions to dynamically decide cache replacement or prefetch operations; S4. According to the hit rate and conflict times real-time feedback from the cache operation results, the weight allocation of the hybrid prediction model is dynamically adjusted through an online learning mechanism, and the sharded reinforcement learning strategy parameters are updated to generate a closed-loop optimization link.

[0006] Further, the specific process of constructing a dynamic heat matrix by collecting the spatio-temporal characteristics and semantic association data of memory access requests in real time is as follows: The memory access requests are clustered and analyzed according to the spatial position offset of data blocks, and the access density of adjacent regions is marked; The short-term burst access intensity and long-term periodic access probability are calculated based on the time access interval distribution; The context dependence relationship of data blocks is extracted through a lightweight pre-trained model to generate a semantic association vector, which is concatenated with the spatio-temporal characteristics to form a dynamic heat matrix.

[0007] Further, the specific process of extracting the context dependence relationship through a lightweight pre-trained model and generating short-term and long-term access trend scores based on sliding window statistics is as follows: The scale of the pre-trained model is compressed through knowledge distillation, the key semantic association layer is retained, and the context dependence intensity of data blocks is output; The continuous access times and interval variance of data blocks are statistically calculated within the sliding window, and used as short-term heat and long-term stability scores respectively.

[0008] Further, the specific process of inputting the dynamic heat matrix into a hybrid prediction model and predicting the access probability within the future time window through a time convolutional network is as follows: A causal time convolutional network is constructed, and the multi-scale temporal dependence relationship is captured through the dilated convolutional layer; The spatio-temporal characteristics and semantic association vectors in the dynamic heat matrix are concatenated into an input tensor, and the access probability distribution of each data block within the future window is output; Potential hot data is screened based on the access probability distribution to generate an initial heat ranking.

[0009] Further, the specific process of generating the conflict prediction weight and the corrected popularity ranking based on the graph attention network to analyze the competition relationship between data blocks is as follows: Construct a competition relationship graph with data blocks as nodes and semantic association strength as edge weights; Calculate the conflict influence coefficient between nodes through the graph attention network to generate the conflict prediction weight; Weightedly fuse the conflict prediction weight and the access probability output by the temporal convolutional network to correct the initial popularity ranking and generate the corrected popularity ranking.

[0010] Further, the specific process of dividing the cache into logical sub-regions according to the popularity ranking and the conflict prediction weight through the slice-based reinforcement learning algorithm is as follows: Divide the cache into high, medium, and low popularity logical sub-regions according to the corrected popularity ranking, and assign an independent DQN agent to each sub-region; Dynamically adjust the sub-region boundary according to the conflict prediction weight, and preferentially isolate high-conflict data blocks; Reduce the computational redundancy of each agent by sharing the underlying feature encoding network to generate a slice-based decision-making unit.

[0011] Further, design a differentiated reward function for different sub-regions, and the dynamic decision-making of cache replacement or prefetch operations includes: High-popularity sub-region: The reward function is dominated by hit rate and latency optimization, and punishes the origin access caused by cache eviction; Medium and low-popularity sub-regions: The reward function is dominated by conflict avoidance and energy consumption savings, and rewards low-power cache operations; Output the replacement or prefetch actions of each sub-region.

[0012] Further, the specific process of dynamically adjusting the weight allocation of the hybrid prediction model through the online learning mechanism according to the hit rate and conflict times feedback in real time of the cache operation results is as follows: Calculate the deviation value between the predicted access probability and the actual hit data, and update the convolutional kernel weight of the temporal convolutional network by backpropagation; Dynamically adjust the attention aggregation coefficient of the graph attention network according to the correlation between the conflict times and the prediction weight; Generate a dynamic weight allocation table of the hybrid prediction model to optimize the prediction accuracy within the future window.

[0013] Further, the specific process of updating the slice-based reinforcement learning policy parameters to generate a closed-loop optimization link is as follows: Construct an independent experience replay pool for each slice DQN agent, and preferentially sample the state-action pair data in high-conflict scenarios; Gradually synchronize the main network and target network parameters through the soft update mechanism to maintain training stability; Link the model weights updated by the online learning mechanism with the reinforcement learning policy parameters to generate an adaptive closed-loop optimization link.

[0014] The present invention has the following beneficial effects:

[0015] (1) A memory access optimization method based on intelligent cache management. By collecting the spatio-temporal characteristics and semantic association data of memory access requests in real time, constructing a dynamic heat matrix, and combining a hybrid prediction model for access probability prediction and competition relationship analysis, it can accurately identify hot data and improve the accuracy of access prediction. The statistical analysis of spatio-temporal characteristics combined with semantic association information helps to capture the short-term and long-term trends of data access, making cache management more intelligent. The introduction of a temporal convolutional network can effectively learn time series characteristics, and a graph attention network can model the competition relationship between data blocks, thereby optimizing cache resource allocation, reducing memory access conflicts, and improving the overall data access efficiency.

[0016] (2) A memory access optimization method based on intelligent cache management. Dynamically divide the cache area through a sharded reinforcement learning algorithm, and combine an online learning mechanism to optimize the cache strategy to achieve more adaptable cache management. Design differentiated reward functions for different sub-regions to make cache replacement and prefetch decisions more targeted, thereby optimizing data storage and scheduling. The real-time feedback mechanism can dynamically adjust the model weights according to the cache hit rate and the number of conflicts, enabling the system to have better generalization ability under different workloads, improving cache utilization, and reducing access latency.

[0017] Of course, it is not necessary for any product implementing the present invention to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of a memory access optimization method based on intelligent cache management according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The embodiments of the present application solve the problems of low cache hit rate, serious data access conflicts, insufficient prediction accuracy, and difficulty in adapting cache strategies to dynamic workloads in existing methods through a memory access optimization method based on intelligent cache management.

[0020] The general idea of the solution in the embodiments of the present application is as follows:

[0021] By collecting the spatio-temporal characteristics and semantic association data of memory access requests in real time, constructing a dynamic heat matrix, where the spatio-temporal characteristics include the spatial position offset and time access interval distribution of data blocks, and the semantic association data extracts context dependencies through a lightweight pre-trained model and generates short-term and long-term access trend scores based on sliding window statistics.

[0022] Input the dynamic heat matrix into a hybrid prediction model, predict the access probability within a future time window through a temporal convolutional network, and analyze the competition relationship between data blocks based on a graph attention network to generate conflict pre-judgment weights and corrected heat rankings.

[0023] According to the popularity ranking and conflict prediction weight, the cache is divided into logical sub-regions by a sharded reinforcement learning algorithm. Different differential reward functions are designed for different sub-regions to dynamically decide cache replacement or prefetch operations.

[0024] Based on the hit rate and conflict count obtained from the real-time feedback of cache operation results, the weight assignment of the hybrid prediction model is dynamically adjusted through an online learning mechanism, and the parameters of the sharded reinforcement learning strategy are updated to generate a closed-loop optimization link.

[0025] Please refer to Figure 1 , an embodiment of the present invention provides a technical solution: a memory access optimization method based on intelligent cache management, including the following steps: S1. By collecting the spatio-temporal features and semantic association data of memory access requests in real time, a dynamic popularity matrix is constructed, where the spatio-temporal features include the spatial position offset and time access interval distribution of data blocks, and the semantic association data extracts context dependencies through a lightweight pre-trained model and generates short-term and long-term access trend scores based on sliding window statistics; S2. The dynamic popularity matrix is input into a hybrid prediction model, and the access probability within a future time window is predicted through a temporal convolutional network, and the competition relationship between data blocks is analyzed based on a graph attention network to generate conflict prediction weights and a corrected popularity ranking; S3. According to the popularity ranking and conflict prediction weight, the cache is divided into logical sub-regions by a sharded reinforcement learning algorithm. Different differential reward functions are designed for different sub-regions to dynamically decide cache replacement or prefetch operations; S4. Based on the hit rate and conflict count obtained from the real-time feedback of cache operation results, the weight assignment of the hybrid prediction model is dynamically adjusted through an online learning mechanism, and the parameters of the sharded reinforcement learning strategy are updated to generate a closed-loop optimization link.

[0026] In this implementation scheme: S1. In this step, by collecting the spatio-temporal characteristics and semantic association data of memory access requests in real time, a dynamically updated heat matrix is established to characterize the access patterns of data blocks. Spatio-temporal characteristics: - Spatial position offset: That is, the relative position change of the data block in memory, which is used to analyze the locality characteristics of data access. - Time access interval distribution: That is, the interval at which the data block is accessed at different time points, which is used to judge the frequency of access. Semantic association data: Extract the context dependence relationship between data blocks through a lightweight pre-trained model to mine potential access rules. Short-term and long-term access trend scores: Based on the sliding window statistics, calculate the short-term (such as the last N accesses) and long-term (such as within a past period of time) access trends of data blocks to form a more accurate heat evaluation. S2. In this step, a hybrid prediction model is used to combine time series analysis and data competition relationship modeling to improve the accuracy of access pattern prediction. Temporal Convolutional Network (TCN): Used to process time series data and predict the access probability of data blocks within a future time window. Compared with the RNN structure, TCN has parallel computing capabilities and is suitable for high-throughput access prediction tasks. Graph Attention Network (GAT): Based on the access association relationship between data blocks, analyze the competition relationship of data blocks and calculate the conflict prediction weight, that is, which data blocks may cause replacement conflicts in the cache. Revise the heat ranking: Combine the predicted access probability with the conflict prediction weight to optimize the heat ranking of data blocks, ensure that high-heat but low-conflict data is preferentially retained in the cache, and reduce memory access conflicts. S3. Based on the prediction results, in this step, a sharded reinforcement learning algorithm is used to dynamically adjust the cache partitioning method and optimize the cache replacement and prefetch strategies. Cache partitioning: Divide the cache into multiple logical sub-regions through reinforcement learning, and dynamically allocate cache space according to different data access patterns. Differentiated reward function: The caches in different sub-regions have different strategies. For example: - High-heat region: Preferentially retain frequently accessed data to avoid frequent eviction. - Low-heat region: Adopt a more flexible replacement strategy to improve cache utilization. Cache replacement and prefetch: Reinforcement learning dynamically decides whether to perform data replacement (evict low-value data) or prefetch (pre-load data that may be accessed) based on historical access data and reward feedback to optimize the cache hit rate. S4. In this step, an online learning mechanism is constructed to continuously optimize the cache management strategy according to the actual operation of the cache. Feedback signal: Real-time monitor the cache hit rate and the number of conflicts as the training signal for reinforcement learning. Dynamically adjust the prediction model: If it is found that the prediction of some data blocks is inaccurate, adjust the weights of the hybrid prediction model and optimize the parameters of the temporal convolutional network and the graph attention network. Update the reinforcement learning strategy: The cache management strategy will be continuously adjusted to adapt to the dynamically changing access patterns and achieve the adaptive optimization of cache management.

[0027] Specifically, the specific process of constructing a dynamic heat matrix by collecting the spatio-temporal characteristics and semantic association data of memory access requests in real time is as follows: By clustering and analyzing the memory access requests according to the spatial position offset of data blocks, the access density of adjacent regions is marked; Based on the time access interval distribution, the short-term burst access intensity and long-term periodic access probability are calculated; The context dependence of data blocks is extracted through a lightweight pre-trained model to generate semantic association vectors, which are concatenated with spatio-temporal characteristics to form a dynamic heat matrix.

[0028] In this implementation scheme, spatio-temporal characteristics are collected and analyzed. The spatio-temporal characteristics of memory access requests consist of two parts: spatial position offset and time access interval distribution. For the clustering analysis of spatial position offset, let the data block number be D i , and other address offsets be X i . Then, the hierarchical clustering method is used to group the data blocks so that adjacent data blocks have higher clustering tightness. Calculate the access density of adjacent data blocks: Among them, N(D i ) represents the neighborhood of data block D i , and α is the attenuation coefficient, which controls the influence degree of the offset on clustering. For the calculation of time access interval distribution, let data block D i be accessed at time t k . Then, calculate its access interval distribution: By counting the access intervals within multiple time windows, calculate the short-term burst access intensity F short,i and the long-term periodic access probability P long,i . For the extraction of semantic association relationships, this scheme uses a lightweight prediction training model to extract the context dependence of data blocks in order to optimize memory access prediction: Knowledge distillation to compress the model scale. Let the representation of the original pre-trained model be M full . Then, through the knowledge distillation method, extract the key semantic word association layer to form a lightweight model M light , where: M light = f(W distill , M full ); Among them, W distill is the distillation weight matrix, and f(·) is the feature extraction function. For the generation of semantic association vectors, let the semantic association vector of data block D i be represented as V i . Then, it is calculated by the lightweight model: V i = M light (D i ), and this vector is used to measure the logical association degree between data blocks. For the generation of the dynamic heat matrix, finally, the spatial position clustering information, time access interval characteristics, semantic association vectors, and short-term / long-term trend scores are concatenated to form a dynamic heat matrix: M heat = [C(D i0,F short,i ,P long,i ,V i ,H short,i ,S long,i ; This matrix is used for subsequent content access prediction and cache management optimization.

[0029] Specifically, the process of extracting context dependencies through a lightweight pre-trained model and generating short-term and long-term access trend scores based on sliding window statistics is as follows: Compress the scale of the pre-trained model through knowledge distillation, retain the key semantic association layer, and output the context dependency strength of the data block; count the continuous access times and interval variances of the data block within the sliding window, and use them as short-term popularity and long-term stability scores respectively.

[0030] In this implementation, for the short-term and long-term access trend scores, the access pattern of the data block is statistically analyzed within the sliding window W: The short-term popularity score calculates the continuous access times of the data block within the window: H short,i = ∑ t∈W II(A t = D i ); where A t is the data block accessed at time t, II() is the indicator function, which takes the value of 1 when accessing the data block D i , and 0 otherwise. The long-term stability score calculates the time interval variance to measure the stability of the data block access pattern: where T gap,i is the mean of the access intervals of the data block D i , and N is the number of statistical times.

[0031] Specifically, the process of inputting the dynamic popularity matrix into the hybrid prediction model and predicting the access probability within the future time window through the temporal convolutional network is as follows: Construct a causal temporal convolutional network, and capture multi-scale temporal dependencies through the dilated convolutional layer; concatenate the spatio-temporal features and semantic association vectors in the dynamic popularity matrix into an input tensor, and output the access probability distribution of each data block within the future window; screen potential hot data based on the access probability distribution to generate an initial popularity ranking.

[0032] In this implementation, a causal temporal convolutional network (CTCN) is constructed. The causal temporal convolutional network is used to capture multi-scale dependencies of temporal data. Its main feature is to expand the receptive field through dilated convolution, so as to be able to capture dependencies in a long time range while maintaining causality, that is, to ensure that the prediction result only depends on past information. Dilated convolutional layer: Dilated convolution adjusts the interval between adjacent convolutional kernels in the convolutional layer, enabling the convolution to cover a longer time series and capture multi-scale temporal features. The convolution formula of the dilated convolutional layer is as follows: where: X t+1 is the output at time step t + 1; Wa is the weight of the convolutional kernel; X t-dk is the input feature at the current time t and historical times; d is the dilation rate, used to increase the receptive field of the convolution; A is the length of the convolutional kernel. The dilated convolutional layer effectively captures long-term and short-term dependencies through this mechanism. Concatenate the input tensors, and concatenate the spatio-temporal features and semantic association vectors of the dynamic heat matrix into an input tensor, which serves as the input to the causal temporal convolutional network. The dynamic heat matrix includes: spatio-temporal features: such as the spatial position offset of data blocks, the time access interval distribution, etc.; semantic association vectors: the context dependencies extracted by the scene-level pre-trained model. Through concatenation, these features are combined into a high-dimensional vector X input , which serves as the input to the temporal convolutional network. The formula is as follows: X input =concat(M heat , V); where: M heat is the spatio-temporal feature in the dynamic heat matrix; V is the semantic association vector. Pass the concatenated input tensor through the causal temporal convolutional network, and the output of the network is the access probability distribution of each data block within the future time window. The output can be normalized through the Softmax function to obtain the access probability of each data block within the future time window. The formula is as follows: where: P i (D i ) is the access probability of the i-th data block predicted at time step t; f(X input , θ) is the non-linear mapping function of the network for the input quantity X input , and θ is the learnable parameter of the network; the Softmax function is used to normalize the predicted values output by the network to obtain the access probability distribution of each data block. Screen potential hot data blocks to generate an initial heat ranking. According to the predicted access probability distribution, screen out potential hot data blocks with higher access probabilities, and generate a preliminary heat ranking based on their access probabilities. The initial heat ranking Rank(D i ) can be obtained through sorting: Rank(D i )=argsort(P t (D i )); where: argsort(P t (D i )) represents sorting the data blocks in descending order according to the access probability P t (D i ) to obtain the ranking of the data blocks. The heat rankings of these potential hot data blocks will provide a reference basis for subsequent cache management decisions.

[0033] Specifically, the specific process of generating the conflict prediction weight and the corrected popularity ranking based on the graph attention network to analyze the competition relationship between data blocks is as follows: Construct a competition relationship graph with data blocks as nodes and semantic association strength as edge weights; Calculate the conflict influence coefficient between nodes through the graph attention network to generate the conflict prediction weight; Weightedly fuse the conflict prediction weight and the access probability output by the temporal convolutional network to correct the initial popularity ranking and generate the corrected popularity ranking.

[0034] In this implementation scheme, a competition relationship graph is constructed, where: Nodes: Each node in the graph represents a data block. Edges: The edges between data blocks represent their semantic association strength, that is, the similarity or dependency relationship between data blocks. The above-mentioned dependency relationship extracted by the lightweight pre-trained model can be used as the weight of the edge. Edge weight E df represents the competition relationship strength between data blocks D a and D f . The formula is as follows: E df = f(V d , V f ); where: E df is the edge weight between data blocks D a and D f ; f(V d , V f ) is the similarity calculation function between semantic association vectors V d and V f . Calculate the conflict influence coefficient. Through the graph attention network, calculate the conflict influence coefficient between each node (data block) in the graph. The graph attention network combines the features of the nodes (access features of data blocks) and the weights of the edges (semantic association strength) through the self-attention mechanism to dynamically learn the influence of each node. In the graph attention network, the output feature H d of the node can be calculated through the following steps: For each node Dd, aggregate the information of its adjacent nodes, and the weight is determined by the attention mechanism. Based on the edge weight E df in the graph, calculate the conflict influence coefficient α df between each pair of nodes. The conflict influence coefficient formula is as follows: where: α df is the conflict influence coefficient between nodes D d and D f ; W is the learning weight matrix of the network; || represents the feature concatenation operation; N(d) is the set of adjacent nodes of node Dd. By calculating the conflict influence coefficient α df , the competition relationship between each data block and other data blocks can be evaluated. Generate the conflict prediction weight. The conflict prediction weight C d is used to represent the possible cache competition degree of each data block. This weight is related to the conflict influence coefficient α of the nodedf is associated with and calculated by combining all the conflict information related to the data block. The calculation formula for the conflict prediction weight is as follows: C d = ∑ f∈N ( d )α df ·P f ; where: C d is the conflict prediction weight of data block D d ; α df is the conflict influence coefficient between nodes D d and D f ; P f is the access probability of data block D f . The initial heat ranking is corrected by weighted fusion of the conflict prediction weight and the access probability output by the temporal convolutional network to correct the initial heat ranking. The corrected heat ranking R' d is the result of combining the access probability of the data block with the conflict prediction weight. The calculation formula for the corrected heat ranking is as follows: R' d = P d ·(1 - λ·C d ); where: R' d is the corrected heat ranking; P d is the access probability of data block D d ; C d is the conflict prediction weight of data block D d ; λ is a weight factor that controls the influence degree of the conflict prediction weight on the heat ranking. Through this correction, the competition relationship between data blocks can be taken into account, providing a more accurate basis for the cache management strategy.

[0035] Specifically, according to the heat ranking and the conflict prediction weight, the specific process of dividing the cache into logical sub-regions by the sharded reinforcement learning algorithm is as follows: The cache is divided into high, medium, and low heat logical sub-regions according to the corrected heat ranking, and an independent DQN agent is assigned to each sub-region; The sub-region boundaries are dynamically adjusted according to the conflict prediction weight, and high-conflict data blocks are preferentially isolated; The calculation redundancy of each agent is reduced by sharing the underlying feature encoding network to generate a sharded decision unit.

[0036] In this implementation, the cache is divided according to the popularity ranking: Based on the revised popularity ranking, the entire cache is divided into different logical sub-regions. According to the popularity ranking of each data block, it is assigned to sub-regions in the following three categories: High-popularity sub-region: Stores data blocks with a higher access frequency and that need to be accessed frequently. Medium-popularity sub-region: Stores data blocks with a moderate access frequency and may need to be accessed regularly. Low-popularity sub-region: Stores data blocks with a lower access frequency and a smaller access probability. An independent DQN agent is assigned to each sub-region: For each logical sub-region, an independent Deep Q-Network (DQN) agent is assigned. Each agent is responsible for managing the cache in its sub-region and executing cache replacement, prefetching, and other policies. These DQN agents continuously interact with the environment to learn how to make decisions based on the cache access patterns, thereby optimizing the overall cache management effect. Dynamically adjust the sub-region boundaries: According to the conflict prediction weight, the sub-region boundaries of the cache are dynamically adjusted. The conflict prediction weight reflects the competition relationship between data blocks and possible cache conflicts, and the agent adjusts the size and boundaries of the sub-region based on this information. High-conflict data blocks are preferentially isolated into different sub-regions to avoid cache conflicts and improve the cache hit rate. To reduce computational redundancy and improve computational efficiency, all DQN agents share a bottom-layer feature encoding network. This network extracts common bottom-layer features from the global cache data and provides these features to the agents in each sub-region. By sharing the bottom-layer feature encoding network, the repetitive calculation of each agent extracting features separately is avoided, and the computational efficiency of the entire system is improved. Generate a sliced decision unit. Through the design of the sliced reinforcement learning algorithm, multiple independent decision units are formed, and each decision unit corresponds to a DQN agent in a sub-region. Each agent makes corresponding cache management decisions according to the specific requirements (popularity ranking and conflict prediction weight) of its sub-region. Through this sliced structure, more fine-grained cache management can be achieved, and the overall performance of the system can be improved.

[0037] Specifically, different reward functions are designed for different sub-regions. The dynamic decision-making of cache replacement or prefetching operations includes: High-popularity sub-region: The reward function is dominated by the hit rate and latency optimization, and punishes the origin access caused by cache eviction; Medium- and low-popularity sub-regions: The reward function is dominated by conflict avoidance and energy consumption savings, and rewards low-power cache operations; Output the replacement or prefetch actions of each sub-region.

[0038] In this implementation plan, for the design of the reward function in the high-heat sub-region, the goals are as follows: For the high-heat sub-region, the main focuses are on the hit rate of data blocks and latency optimization to ensure that frequently accessed data blocks can be obtained quickly and the latency is reduced. Reward function: Hit rate optimization: When a data block is correctly hit in the cache, the system gives a reward. This helps to improve the cache hit rate and ensures that frequently accessed data blocks can be preferentially retained in the cache. Latency optimization: When a cache hit occurs, not only should we focus on whether there is a hit, but also optimize the hit latency. If a data block can respond to a request in a shorter time, the system gives a higher reward according to the latency optimization. Penalize backhaul access: If a frequently accessed data block is replaced in the cache, resulting in subsequent accesses needing to be reloaded from external storage, this will cause a longer access latency. Therefore, the system penalizes this "backhaul access" behavior, reduces the frequency of cache eviction operations, and encourages the system to retain high-heat data blocks as much as possible. For the design of the reward function in the medium-low heat sub-region, the goals are as follows: For the medium-low heat sub-region, the key focuses are on conflict avoidance and energy consumption savings. The data blocks in these sub-regions are accessed less frequently. Therefore, more energy-efficient cache operation strategies can be adopted while avoiding cache conflicts caused by data competition. Reward function: Conflict avoidance: When conflicts occur between two or more data blocks, it may lead to frequent cache replacements and performance degradation. To avoid this, the system rewards cache operations that effectively avoid conflicts. For example, data blocks with competitive relationships are allocated to different cache sub-regions to reduce the replacement conflicts that may be caused by competition. Energy consumption savings: In the medium-low heat sub-region, data blocks with lower access frequencies may not need to be frequently retained in the cache. The system rewards low-power cache operations, such as storing these data blocks in regions with lower energy consumption or implementing a more efficient cache replacement strategy to achieve the effect of energy savings. Output replacement or prefetch actions. In the high-heat sub-region and the medium-low heat sub-region, the system makes dynamic adjustments through differentiated reward functions and decides on cache replacement or prefetch operations according to the access requirements of the current sub-region. Replacement operation: Based on the heat of data blocks and conflict prediction, the system decides whether to replace the data blocks in the cache. For the high-heat sub-region, the system preferentially retains frequently accessed data blocks, while for the medium-low heat sub-region, it may choose to evict infrequently accessed data blocks from the cache. Prefetch operation: The system preloads data blocks that may be accessed in advance according to historical access patterns and predicted future access trends. For the high-heat sub-region, the prefetch operation can effectively reduce access latency; for the medium-low heat sub-region, the prefetch operation mainly focuses on energy consumption savings and avoiding latency during subsequent accesses.

[0039] Specifically, the specific process of dynamically adjusting the weight allocation of the hybrid prediction model according to the hit rate and conflict times fed back in real time by the cache operation results is as follows: By calculating the deviation value between the predicted access probability and the actual hit data, the convolutional kernel weights of the temporal convolutional network are updated through backpropagation; According to the correlation between the conflict times and the predicted weights, the attention aggregation coefficient of the graph attention network is dynamically adjusted; Generate a dynamic weight allocation table for the hybrid prediction model to optimize the prediction accuracy within the future window.

[0040] In this implementation, calculate the deviation between the predicted access probability and the actual hit data: By comparing the prediction result of the model with the actual access result, the parameters of the prediction model are adjusted, thereby improving the accuracy of future access prediction. Process: First, the model generates the access probability distribution of the future time window according to the input dynamic popularity matrix. Then, compare the actual access result with the predicted value of the model to calculate the deviation between the predicted access probability and the actual hit data. This deviation value reflects the prediction error of the current model. Update the convolutional kernel weights of the temporal convolutional network through backpropagation: Update the convolutional kernel of the temporal convolutional network (TCN) according to the prediction error, so as to optimize the prediction accuracy within the future time window. Process: Through the backpropagation algorithm, use the above deviation value (prediction error) to adjust the convolutional kernel weights in the temporal convolutional network. The update direction of the convolutional kernel is to minimize the prediction error, so that the model is more accurate in predicting the future access probability. The update process performs backpropagation through the gradient rule, propagates the error to each layer in the network, and gradually adjusts the parameters of the convolutional kernel. Formula representation: Where: W new represents the updated convolutional kernel weight; W old represents the current convolutional kernel weight; η is the learning rate; represents the gradient calculated according to the prediction error. Dynamically adjust the attention aggregation coefficient of the graph attention network: According to the correlation between the conflict times between data blocks and the predicted weights, optimize the attention coefficient in the graph attention network, so as to improve the calculation accuracy of the conflict impact between nodes. Process: In the graph attention network, the attention aggregation coefficient between nodes determines how the relationship between nodes affects the final prediction. In order to reduce the interference of cache conflicts on model prediction, based on the correlation between the actual conflict times and the predicted weights, the attention aggregation coefficient is dynamically adjusted. If some data blocks have more conflicts in practice, then reduce their influence in future predictions by adjusting the attention aggregation coefficient, thereby reducing the influence of these conflicts on cache operation decisions. Formula representation: Formula representation: Where: α k,l represents the attention coefficient between node p and node l; v is the learning parameter of the graph attention network; h p and h lThey are the feature vectors of nodes k and l respectively, and ∥ represents the feature vector concatenation operation. Generate the dynamic weight allocation table of the hybrid prediction model: Considering the optimization results of the temporal convolutional network and the graph attention network comprehensively, generate the final dynamic weight allocation table to optimize the prediction accuracy within the future window. Process: After updating the convolutional kernels of the temporal convolutional network and the attention coefficients of the graph attention network, the system generates a dynamic weight allocation table based on these adjustment results. This table records the weight allocation of each model component (such as the temporal convolutional network and the graph attention network) in the entire prediction task. The weight allocation table is comprehensively adjusted according to multiple factors such as the actual deviation, conflict prediction weight, access probability, etc., to ensure more accurate and reliable future access predictions.

[0041] Specifically, the process of updating the sharded reinforcement learning policy parameters and generating the closed-loop optimization link is as follows: Construct an independent experience replay pool for each sharded DQN agent, and preferentially sample the state-action pair data in high-conflict scenarios; Gradually synchronize the parameters of the main network and the target network through the soft update mechanism to maintain training stability; Link the model weights updated by the online learning mechanism with the reinforcement learning policy parameters to generate an adaptive closed-loop optimization link.

[0042] In this implementation, an independent experience replay pool is constructed for each sharded DQN agent: An independent experience replay pool is provided for each sharded agent, enabling it to separately store and replay the state-action pairs encountered in cache management, thereby improving the efficiency of policy learning. Process: Each agent executes actions in its independent cache sub-region and generates a series of state-action pairs. These data are stored in their respective experience replay pools. State-action pairs in high-conflict scenarios are preferentially selected for sampling to ensure that the agent can better learn when dealing with situations with more conflicts. Preferential sampling helps accelerate the convergence of the policy in high-conflict situations. The parameters of the main network and the target network are gradually synchronized through a soft update mechanism: To maintain the stability of the training process, the parameters of the main network and the target network are gradually synchronized through a soft update mechanism to prevent unstable behaviors during the training process. Process: In the Deep Q-Network (DQN), the main network is responsible for generating the current Q-value estimate, while the target network is used to stabilize the training process. After each training, instead of directly replacing the parameters of the target network, a soft update method is used to slowly update the parameters of the main network into the target network at a small update ratio, avoiding interference from sudden parameter changes to the training process. The model weights updated by the online learning mechanism are linked with the reinforcement learning policy parameters: The weights of the reinforcement learning model are updated through the online learning mechanism, making the agent's policy match the actual requirements of cache management and generating an adaptive policy. Process: The online learning mechanism dynamically adjusts the model weights based on real-time feedback information such as the hit rate and the number of conflicts. The update of the reinforcement learning policy parameters depends on these dynamic feedbacks, enabling the policy to gradually adapt to different cache operation scenarios. The online learning continuously updates the policy based on the reward signal during the training process. Combined with the update of the model weights, the reinforcement learning policy can adaptively adjust the cache replacement or prefetch policy to cope with the changing environment. An adaptive closed-loop optimization link is generated: An adaptive closed-loop optimization link is formed through the above process, enabling the cache management policy to automatically adjust according to the real-time environment and feedback information, thereby achieving efficient cache optimization. Process: Each sharded DQN agent dynamically optimizes its policy through an independent experience replay pool, a soft update mechanism, and an online learning mechanism. The policy updates of each agent interact with each other in the cache management of different sub-regions, thereby continuously optimizing the cache access pattern, conflict prevention, energy consumption control, etc. The adaptive closed-loop optimization link can provide real-time feedback on the learning results and adjust the decision-making process, thus realizing the continuous optimization of the cache management system.

[0043] In summary, this application has at least the following effects:

[0044] A memory access optimization method based on intelligent cache management. By combining heat ranking, conflict prediction weights, and multiple prediction models, the cache replacement and prefetching strategies are optimized, thereby improving the cache hit rate and reducing the backhaul access, and enhancing the overall system performance. Utilizing online learning and reinforcement learning mechanisms, the system can dynamically adjust the strategies according to real-time feedback data (such as hit rate, conflict times, etc.), thus adaptively coping with different cache operation scenarios and enhancing the intelligence level of the system. By sharing the underlying feature encoding network, each shard agent can reduce computational redundancy and lower the computational overhead while ensuring the cache management efficiency. Combining multi-dimensional data such as short-term and long-term access trends, spatio-temporal features, and semantic association vectors, more accurate prediction and decision-making support are provided to optimize cache allocation, replacement, and prefetching operations. According to different sub-regions, differentiated reward functions are designed to optimize the hit rate and latency of high-heat sub-regions, and the cache operations of medium- and low-heat sub-regions are optimized through conflict avoidance and energy-saving reward functions to enhance the overall energy efficiency. Through independent experience replay pools and soft update mechanisms, it is ensured that each shard agent can be stably trained to generate an adaptive closed-loop optimization link, enabling the entire cache management system to continuously improve and achieve the optimal effect.

[0045] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0046] The present invention is described with reference to the flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the flows or multiple flows and / or blocks Figure 1 one or more of the blocks or multiple blocks.

[0047] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions in the flowFigure 1 one process or multiple processes and / or boxes Figure 1 the functions specified in one box or multiple boxes

[0048] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or boxes Figure 1 one box or multiple boxes

[0049] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0050] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A memory access optimization method based on intelligent cache management, characterized in that It includes the following steps: S1. By collecting the spatio-temporal features and semantic association data of memory access requests in real time, a dynamic heat matrix is constructed, where the spatio-temporal features include the spatial position offset and time access interval distribution of data blocks. The semantic association data extracts context dependencies through a lightweight pre-trained model and generates short-term and long-term access trend scores based on sliding window statistics; S2. The dynamic heat matrix is input into a hybrid prediction model. The access probability within the future time window is predicted through a temporal convolutional network, and the competition relationship between data blocks is analyzed based on a graph attention network to generate conflict prediction weights and a corrected heat ranking; S3. According to the heat ranking and conflict prediction weights, the cache is divided into logical sub-regions through a sharded reinforcement learning algorithm. Differentiated reward functions are designed for different sub-regions to dynamically decide cache replacement or prefetch operations; S4. According to the hit rate and conflict times feedback in real time by the cache operation results, the weight allocation of the hybrid prediction model is dynamically adjusted through an online learning mechanism, and the parameters of the sharded reinforcement learning strategy are updated to generate a closed-loop optimization link.

2. The memory access optimization method based on intelligent cache management according to claim 1, wherein: The specific process of constructing a dynamic heat matrix by collecting the spatio-temporal features and semantic association data of memory access requests in real time is as follows: The memory access requests are clustered and analyzed according to the spatial position offset of data blocks, and the access density of adjacent regions is marked; The short-term burst access intensity and long-term periodic access probability are calculated based on the time access interval distribution; The context dependencies of data blocks are extracted through a lightweight pre-trained model to generate semantic association vectors, which are concatenated with the spatio-temporal features to form a dynamic heat matrix.

3. The memory access optimization method based on intelligent cache management according to claim 2, characterized in that: The specific process of extracting context dependencies through a lightweight pre-trained model and generating short-term and long-term access trend scores based on sliding window statistics is as follows: The scale of the pre-trained model is compressed through knowledge distillation, the key semantic association layer is retained, and the context dependency intensity of data blocks is output; The continuous access times and interval variance of data blocks are statistically calculated within the sliding window as the short-term heat and long-term stability scores respectively.

4. An optimized method for memory access based on intelligent cache management according to claim 2, characterized in that: The specific process of inputting the dynamic heat matrix into a hybrid prediction model and predicting the access probability within the future time window through a temporal convolutional network is as follows: A causal temporal convolutional network is constructed, and multi-scale temporal dependencies are captured through dilated convolutional layers; The spatio-temporal features and semantic association vectors in the dynamic heat matrix are concatenated into an input tensor, and the access probability distribution of each data block within the future window is output; Potential hot data is screened based on the access probability distribution to generate an initial heat ranking.

5. The memory access optimization method based on intelligent cache management according to claim 4, wherein: The specific process of analyzing the competition relationship between data blocks based on a graph attention network to generate conflict prediction weights and a corrected heat ranking is as follows: A competition relationship graph is constructed with data blocks as nodes and semantic association intensity as edge weights; The conflict influence coefficient between nodes is calculated through a graph attention network to generate conflict prediction weights; The conflict prediction weights are weighted and fused with the access probability output by the temporal convolutional network to correct the initial heat ranking and generate a corrected heat ranking.

6. The memory access optimization method based on intelligent cache management according to claim 5, wherein: The specific process of dividing the cache into logical sub-regions through a sharded reinforcement learning algorithm according to the heat ranking and conflict prediction weights is as follows: The cache is divided into high, medium, and low heat logical sub-regions according to the corrected heat ranking, and an independent DQN agent is assigned to each sub-region; Dynamically adjust the sub-region boundary according to the conflict prediction weight, and give priority to isolating high-conflict data blocks; Reduce the computational redundancy of each agent by sharing the underlying feature encoding network to generate a sharded decision-making unit.

7. A memory access optimization method based on intelligent cache management according to claim 6, characterized in that: Design a differentiated reward function for different sub-regions. The dynamic decision-making cache replacement or prefetch operation includes: High-heat sub-region: The reward function is dominated by hit rate and latency optimization, and punishes the origin access caused by cache eviction; Medium and low heat sub-regions: The reward function is dominated by conflict avoidance and energy consumption savings, and rewards low-power cache operations; Output the replacement or prefetch actions of each sub-region.

8. An optimization method for memory access based on intelligent cache management according to claim 7, characterized in that: The specific process of dynamically adjusting the weight allocation of the hybrid prediction model according to the hit rate and conflict times feedback in real time by the cache operation result through the online learning mechanism is as follows: Calculate the deviation value between the predicted access probability and the actual hit data, and update the convolution kernel weight of the temporal convolutional network by backpropagation; Dynamically adjust the attention aggregation coefficient of the graph attention network according to the correlation between the conflict times and the prediction weight; Generate a dynamic weight allocation table for the hybrid prediction model to optimize the prediction accuracy within the future window.

9. The memory access optimization method based on intelligent cache management according to claim 8, wherein: The specific process of updating the sharded reinforcement learning policy parameters to generate a closed-loop optimization link is as follows: Construct an independent experience replay pool for each sharded DQN agent, and preferentially sample the state-action pair data in high-conflict scenarios; Gradually synchronize the main network and target network parameters through the soft update mechanism to maintain training stability; Link the model weights updated by the online learning mechanism with the reinforcement learning policy parameters to generate an adaptive closed-loop optimization link.

Images