A cache data prefetching processing method based on neural network

Abstract

This invention belongs to the field of data prefetching technology, and particularly relates to a cache data prefetching processing method based on neural networks. This invention designs a perceptron and an evaluation mechanism. The perceptron uses a lightweight multilayer perceptron to learn from historical access records and quickly infers access patterns when access addresses are updated, thereby dynamically selecting a suitable prefetcher. The evaluation mechanism uses the average access state within a short window to predict the accuracy of the address and assigns a score, updating the prefetcher in a timely manner based on the score. This invention constructs a lightweight integrated prefetcher, unifying and dynamically scheduling multiple different prefetchers online, and quickly selecting the optimal prefetcher based on the current memory access behavior, thereby improving prefetch accuracy, cache hit rate, and processor execution efficiency, while reducing cache pollution caused by erroneous prefetching.

Description

Technical Field

[0001] This invention belongs to the field of data prefetching technology, and in particular relates to a cached data prefetching processing method based on neural networks. Background Technology

[0002] With the rapid development of high-performance processors, cloud computing, and large-scale data center technologies, processor computing power continues to improve, while main memory access latency increases relatively slowly, leading to the increasingly prominent "memory wall" problem. During program execution, a significant number of cycles are consumed waiting for data to return from main memory, severely limiting overall system performance. Therefore, how to prefetch potentially accessed data and reduce access latency caused by cache misses has become an important research direction in the field of computer architecture.

[0003] To alleviate the main memory access latency problem, researchers have proposed a data prefetching technique. This technique analyzes a program's historical memory access behavior and loads potentially accessed data into the cache level before the processor actually initiates an access request, thereby reducing cache miss rates and improving processor execution efficiency.

[0004] However, with the increasing complexity of modern applications, program access behavior exhibits more dynamic and diverse characteristics. The same program may alternate between sequential access, cross-step access, linked pointer access, and random access at different execution stages, making it difficult for a single type of prefetcher to continuously adapt to all access scenarios. Existing patents, such as a data processing method and related apparatus (patent application number: 2025101485120), dynamic memory fusion and prefetching technology (patent application number: 2021111716918), and a memory access structure based on multi-bank cache prefetching technology (patent application number: 2022105119929), mainly focus on increasing the storage table and hierarchical prefetching. Although these inventions improve performance under specific program patterns, they still cannot improve prefetch accuracy or reduce cache pollution after access patterns change. In summary, existing cache prefetching technologies struggle to effectively handle dynamic access patterns and cannot update the prefetching model promptly after access occurs, thus limiting their adaptability and prefetching performance in complex program scenarios. Therefore, prefetching of cached data is both urgent and necessary.

[0005] Currently, cache data prefetching methods are categorized into three types: time-based prefetching, spatial prefetching, and spatiotemporal prefetching. These methods typically rely on fixed models or long-term historical statistical information for prefetching. When program access patterns change rapidly, the prefetcher struggles to adjust its prefetching strategy in a timely manner, leading to significant adaptation delays between different stages and impacting overall prefetching performance. Therefore, a method is needed to improve cache data prediction capabilities and reduce cache data pollution. Summary of the Invention

[0006] The purpose of this invention is to provide a neural network-based cache data prefetching method to address the problems of insufficient adaptability, degraded prefetching performance, and cache pollution in existing data prefetching methods under complex program access patterns. This invention designs a perceptron and an evaluation mechanism. The perceptron uses a lightweight multilayer perceptron to learn from historical access records and quickly infers access patterns when access addresses are updated, thereby dynamically selecting a suitable prefetcher. The evaluation mechanism uses the average access state within a short window to predict the accuracy of the address and assigns a score, updating the prefetcher based on the score. This invention constructs a lightweight integrated prefetcher, unifying and dynamically scheduling multiple different prefetchers online. It quickly selects the optimal prefetcher based on the current memory access behavior, thereby improving prefetch accuracy, cache hit rate, and processor execution efficiency, while reducing cache pollution caused by erroneous prefetching.

[0007] The technical solution of the present invention is as follows: A method for prefetching cached data based on neural networks includes the following steps: Step 1: When a last-level cache LLC miss occurs in the processor, multiple different prefetchers are invoked in parallel to generate a set of candidate predicted addresses.

[0008] The prefetcher is at least two of the following: a time prefetcher, a spatial prefetcher, and a spatiotemporal prefetcher. Preferred prefetchers are BO, SPP, VLDP, and ISB. Each prefetcher outputs a corresponding candidate prediction address according to its own prefetching mechanism.

[0009] Step 2: Obtain the current memory access context characteristics and perform data preprocessing operations.

[0010] The memory access context features include the program counter (PC), access address, page number, page offset, and window statistics. The memory access context features are then normalized.

[0011] For discrete features such as the program counter (PC) and access address, a bit-fold hashing method is used for compression mapping to obtain normalized feature values: (1) in, Represents the normalized features. Indicates input features, This represents the result of the bit-folded hash function. Indicates the bit width of the hash code.

[0012] For numerical features such as page number, offset, and window statistics, maximum value normalization is used: (2) in, This represents the theoretical maximum value or preset upper bound of the corresponding feature.

[0013] Step 3: Input the normalized memory access context features into the perceptron and calculate the probability of all prefetchers in the current state.

[0014] The normalized features are input into the perceptron. The perceptron employs a lightweight multi-layer MLP structure, consisting of an input layer, hidden layers, and an output layer. Subsequently, the input features undergo forward propagation within the network, and the output of layer l is... The calculation is as follows: (3) Where, when l=0, It is the input layer, i.e., the normalized feature values. , and These represent the weights and biases of the l-th layer, respectively. The activation function is used. The perceptron output layer uses the Softmax function to calculate the adaptation probability for each prefetcher: (4) in, This indicates that in the current memory access state, the first... i The probability that a prefetcher can produce the optimal prefetching effect.

[0015] Step 4: Select a prefetcher based on the perceptron output and generate the final prefetch request.

[0016] The prefetcher with the highest fit probability is selected as the current control strategy, and the candidate prediction addresses of this strategy are sent to the cache level as future prefetch data.

[0017] Step 5: Build an instant evaluation mechanism to estimate the revenue of prefetch requests in real time.

[0018] Before the actual cache hit feedback is returned, calculate the estimated benefit score of the prefetch candidate prediction address based on the current context information: (5) in, and These represent the spatial information score and the pattern consistency score, respectively. This indicates the estimation of the accuracy score of candidate predicted addresses within the t-window.

[0019] Step 6: Calculate the spatial information score.

[0020] If the candidate predicted address is located on the current page or in a nearby area, the spatial information score is calculated using the following formula: (6) in, Indicates the page number of the predicted address. This indicates the page with the most recent historical address. Indicates page size. Indicates the predicted address. This indicates the most recent historical address.

[0021] Step 7: Calculate the pattern consistency score.

[0022] By comparing the consistency between historical address increment patterns and predicted address increment patterns, prefetch requests are evaluated using pattern matching. The pattern consistency score is calculated as follows: (7) in, This represents the increment between the predicted address and the nearest address. This represents the increment of the most recent historical address. Indicates the direction of incremental change.

[0023] Step 8: Use the real-time evaluation results to train and update the perceptron online.

[0024] The prefetching results are scored in real time using an instant evaluation mechanism, and the corresponding prefetching score is calculated according to formula (5). This score is then used as a pseudo-label of prefetching accuracy to guide the online updating of the perceptron. In this way, the present invention can determine the effectiveness of prefetching in advance and adjust the perceptron model parameters in a timely manner without waiting for the actual access results to return, thereby improving the model's online adaptability to dynamic access patterns.

[0025] Step 9: Construct a delay correction mechanism to provide real feedback and correction to the immediate evaluation results.

[0026] Record the prefetch requests that have been sent. When a subsequent processor accesses the corresponding predicted address, the prefetch request is marked as 1, indicating a valid prefetch, and the perceptron is retrained. If a subsequent processor accesses the corresponding predicted address but does not hit it, the prefetch request is marked as 0, indicating an invalid prefetch, and the perceptron is retrained.

[0027] The beneficial effects of this invention are: (1) This invention integrates multiple prefetchers with different characteristics to achieve dynamic switching between different access modes, thereby improving the data prefetching accuracy under complex workloads.

[0028] (2) The present invention uses a lightweight multilayer perceptron as an integrated decision model, which has lower hardware implementation complexity and inference latency compared to complex recurrent neural network structures.

[0029] (3) The present invention uses an instant evaluation mechanism to estimate the pre-collected benefits in advance before the actual feedback is returned, thereby improving the speed of online decision updates and reducing the performance loss caused by feedback lag. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the workflow of a neural network-based cached data prefetching method according to the present invention.

[0031] Figure 2 This is a flowchart of the pre-fetching process in this invention. Detailed Implementation

[0032] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0033] This embodiment uses the deployment of an integrated prefetch framework on the ChampSim-ML simulation platform as an example to illustrate the method of the present invention, but the present invention is not limited to the following embodiment.

[0034] In this embodiment, the method of the present invention is deployed on a Linux server, and the server configuration is shown in Table 1. The present invention uses the ChampSim-ML simulation platform to run the SPEC CPU 2006 program assembly and conducts experiments on cached data prefetching technology.

[0035] Table 1 Linux Server Configuration Information

[0036] In this embodiment, the system installs the ChampSim-ML platform and deploys this invention on top of it. This invention, acting as the upper-level controller of the last-level cache LLC layer, runs in parallel with multiple prefetchers to dynamically select the current optimal prefetch result. Figure 1 As shown, this invention records access data and invokes a prefetcher to generate candidate predicted addresses. Then, this invention normalizes the input features and calculates the probability of the prefetcher based on the normalized features, dynamically selecting the prefetcher with the highest probability. Finally, this invention utilizes contextual information to quickly estimate, score, and provide feedback correction for the prefetching results.

[0037] The specific steps in this embodiment are as follows: Step 1: Build a cache simulation environment, which means downloading the ChampSim-ML platform and deploying the method of this invention.

[0038] Specifically, a single-core processor model was configured in the ChampSim-ML platform, with a processor frequency of 4 GHz and a last-level cache (LLC) capacity of 8 MB. Prefetchers included BO, SPP, VLDP, and ISB, all deployed at the LLC level. All prefetchers used publicly available implementation parameters to ensure experimental fairness. In this embodiment, each workload underwent a fixed-length warm-up process before the formal statistical phase to eliminate the impact of cache cold starts on the experimental results.

[0039] Step 2: Load the benchmark assembly and perform memory access behavior simulation.

[0040] Specifically, this embodiment selects the SPEC CPU 2006 assembly to cover different types of memory access modes. During program execution, the ChampSim-ML platform records the processor's memory access behavior and current memory access status information cycle by cycle.

[0041] Step 3: When an LLC miss event or a prefetch trigger event occurs, multiple prefetchers are invoked to generate candidate prediction addresses in parallel.

[0042] Specifically, when an LLC miss occurs in the processor, the prefetchers BO, SPP, VLDP, and ISB synchronously receive the current memory access information and generate corresponding candidate predicted addresses according to their own prefetch mechanisms: Addr(add BO add SPP add VLDP add ISB ={8ad330167480, 4741a650c280, 98f569043100, 8ad330143c00}. In Addr, each prefetcher currently generates a predicted address.

[0043] Step 4: The perceptron extracts the current memory access context features and performs data preprocessing.

[0044] Specifically, the perceptron extracts the program counter (PC), current access address, page number, page offset, and increment feature from the current running state. The extracted feature combination is as follows: Feature (program counter, page, offset, increment) = {47939203, 20, 3, -2}. Then, this invention uses formulas (1) and (2) to preprocess these features, normalizing them to the [0, 1] interval, which is feature (program counter, page, offset, increment) = {0.2235, 0.8945, 0.14, 0.51}.

[0045] Step 5: Use the MLP perceptron to prefetch the current optimal prefetcher.

[0046] Specifically, the normalized features and the addresses prefetched by the prefetcher are input into the MLP model. Figure 2 The workflow of the MLP is described in detail. In this embodiment, the features (program counter, page, offset, increment) = {0.2235, 0.8945, 0.14, 0.51} and the addresses prefetched by the prefetcher (BO address, SPP address, VLDP address, ISB address) = {8ad330167480, 4741a650c280, 98f569043100, 8ad330143c00} are input into the MLP. The MLP then outputs the probability that each predicted address will be accessed, i.e., the probability P (BO probability, SPP probability, VLDP probability, ISB probability) = {0.37, 0.98, 0.66, 0.41}. After obtaining the probabilities P (BO probability, SPP probability, VLDP probability, ISB probability), this invention selects the prefetch SPP probability with the highest probability = 0.98, and uses SPP as the control strategy for the current stage. At the same time, the corresponding candidate SPP address = {4741a650c280} is sent to the cache as the final prefetch request.

[0047] Step 6: Perform real-time revenue estimation for prefetch requests.

[0048] Specifically, before the actual cache hit result is returned, a quick estimate is made based on the current context to determine whether the predicted SPP address = {4741a650c280} may generate a benefit. This involves calculating the SPP address score according to formula (5), which includes the spatial information score. space Pattern Consistency Score pattern Substituting the SPP address = {4741a650c280} into formula (5), we get Score = 0.816.

[0049] Step 7: Update the perceptron model using the immediate evaluation results.

[0050] Specifically, the Score=0.816 result generated by the evaluator is used as a pseudo-label input to the perceptron component to update the MLP model parameters in advance.

[0051] Step 8: Implement the delay correction mechanism.

[0052] Specifically, when a subsequent actual memory access arrives, if the access address matches the historical predicted address, the Score=0.816 is changed to the maximum value Score=1, and the corresponding prefetch request is marked as valid. If it misses, the Score is changed to 0, and the corresponding prefetch request is marked as invalid. Finally, the perceptron is retrained and corrected using the marked prefetch request data.

[0053] Step 9: Calculate the system performance indicators and output the experimental results.

[0054] Specifically, this implementation outputs the cache prefetch performance of this invention, which is the prefetch accuracy. At the end of the access process, the ChampSim-ML platform automatically outputs the corresponding prefetch cache accuracy: LLC Accuracy = 0.76.

[0055] This invention achieves adaptive selection and collaborative optimization of the prefetcher by dynamically sensing the access characteristics during the program's execution phase, thereby improving the accuracy of cached data prefetching, enhancing the adaptability to dynamic access patterns, and reducing cache pollution caused by invalid prefetching.

Claims

1. A neural network-based cache data prefetching processing method, characterized by, Includes the following steps: Step 1: When a last-level cache LLC miss event occurs in the processor, multiple different prefetchers are invoked in parallel to generate a set of candidate predicted addresses; Step 2: Obtain the current memory access context characteristics and perform data preprocessing operations; The memory access context features include the program counter (PC), access address, page number, page offset, and window statistics features. The memory access context features are then normalized. Step 3: Input the normalized memory access context features into the perceptron and calculate the probability of all prefetchers in the current state; The normalized features are input into a perceiver; the perceiver adopts a lightweight multi-layer MLP structure, which is composed of an input layer, a hidden layer, and an output layer; then, the input features are forward propagated in the network, and the output of the lth layer is The calculation is as follows: (3) where l = 0, is the input layer, i.e., normalized feature values, , and denote the weight and bias of the l-th layer, respectively, is the activation function; the Softmax function is used in the output layer of the perceptron to calculate the adaptation probability of each prefetcher. (4) wherein, represents the probability that the first i prefetcher is able to produce the optimal prefetch effect in the current memory access state. Step 4: Select a prefetcher based on the perceptron output and generate the final prefetch request; Select the prefetcher with the highest fit probability as the current control strategy, and send the candidate prediction address of this strategy to the cache level as future prefetch data; Step 5: Build an instant evaluation mechanism to estimate the revenue of prefetch requests in real time; Before the actual cache hit feedback is returned, calculate the estimated benefit score of the prefetch candidate prediction address based on the current context information: (5) wherein, and spatial information score and mode consistency score, respectively; denotes estimating the candidate prediction address accuracy score within a t window. Step 6: Calculate the spatial information score; If the candidate predicted address is located on the current page or in a nearby region, the spatial information score is calculated using the following formula: (6) in, Indicates the page number of the predicted address. This indicates the page with the most recent historical address. Indicates page size. Indicates the predicted address. Indicates the most recent historical address; Step 7: Calculate the pattern consistency score; By comparing the consistency between historical address increment patterns and predicted address increment patterns, prefetch requests are evaluated using pattern matching; the pattern consistency score is calculated as follows: (7) in, This represents the increment between the predicted address and the nearest address. This represents the increment of the most recent historical address. Indicates the direction of incremental change; Step 8: Use the real-time evaluation results to train and update the perceptron online; The prefetching results are scored in real time through an instant evaluation mechanism, and the corresponding prefetching score is calculated according to formula (5). Then, the score is used as a pseudo-label of the prefetching accuracy to guide the online updating of the sensor. Step 9: Construct a delay correction mechanism to provide real-time feedback and correction to the immediate evaluation results; Record the prefetch requests that have been sent; when a subsequent processor accesses the corresponding predicted address, mark the prefetch request as 1 to indicate a valid prefetch and recalibrate the perceptron; if a subsequent processor accesses the corresponding predicted address but does not hit it, mark the prefetch request as 0 to indicate an invalid prefetch and recalibrate the perceptron.

2. The method for prefetching cached data based on a neural network according to claim 1, characterized in that, The prefetcher is at least two of the following: a time prefetcher, a spatial prefetcher, and a spatiotemporal prefetcher; each prefetcher outputs a corresponding candidate prediction address according to its own prefetching mechanism.

3. The method for prefetching cached data based on a neural network according to claim 2, characterized in that, The prefetchers are BO, SPP, VLDP, and ISB.

4. The method for prefetching cached data based on a neural network according to claim 1, characterized in that, For the program counter (PC) and the access address, a bit-folded hashing method is used for compressed mapping to obtain normalized feature values: (1) in, Represents the normalized features. Indicates input features, This represents the result of the bit-folded hash function. Indicates the bit width of the hash code; For page numbers, offsets, and window statistics, maximum value normalization is used: (2) in, This represents the theoretical maximum value or preset upper bound of the corresponding feature.

Images