High-precision memory access prediction and cache prefetching method and system based on machine learning
By dynamically identifying significant change points and state-space models in memory access sequences, a combined feature sequence is used to train a neural network model. This solves the problems of prediction accuracy and robustness of irregular memory access patterns in existing technologies, and achieves high-precision cache prefetching.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUAZHONG UNIV OF SCI & TECH
- Filing Date
- 2026-01-24
- Publication Date
- 2026-05-08
AI Technical Summary
Existing machine learning prediction methods have weak generalization ability and insufficient robustness when faced with memory access patterns of irregular strides and complex indexes. They also struggle to deeply understand the underlying mechanisms of memory access behavior, leading to a decline in prediction accuracy.
By monitoring memory access addresses in real time and dynamically identifying significant and irregular change points, the feature values of hidden patterns in memory access behavior are extracted using a state-space model and online recursive estimation. Combined feature sequences are constructed and a neural network model is trained. Combined with a selective architecture, a prefetching strategy is dynamically selected to achieve high-precision prediction of irregular access patterns.
It improves the hit rate, bandwidth utilization efficiency, and overall robustness of the cache prefetching system under complex access patterns, adapting to changes in program behavior and continuously evolving.
Smart Images
Figure CN121996576A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of cache prefetching optimization technology, and in particular to a high-precision memory access prediction and cache prefetching method and system based on machine learning. Background Technology
[0002] In the field of memory access prediction and cache prefetching, in order to alleviate the performance bottleneck between the processor and main memory, a prefetcher is usually used to predict the addresses that the program may access in the future and load the relevant data into the cache in advance. In recent years, intelligent prediction methods based on machine learning have been introduced into this field, aiming to improve the ability to predict nonlinear and irregular memory access behavior by learning complex patterns in historical memory access sequences, thereby optimizing the cache hit rate.
[0003] However, the limitations of existing machine learning prediction methods become particularly prominent when faced with the special and critical memory access pattern of irregular strides and complex indexes. In such scenarios, the offset of the memory access address exhibits dynamic, non-fixed, and seemingly disordered characteristics, causing traditional prefetchers based on fixed strides to completely fail. While existing data-driven learning methods can fit the sequence presented by specific training data, they are unable to fundamentally understand the hidden mapping rules driving this access pattern. This results in the model being heavily dependent on the specific loop structure or data distribution during training. Once the data distribution of the target program is fine-tuned or the execution phase is switched, the prediction accuracy of the model will decrease significantly. The problems of weak generalization ability and lack of robustness become the main obstacles in actual deployment.
[0004] Therefore, the current field of memory access prediction urgently needs a new cache prefetching method that can deeply understand the underlying mechanisms of irregular memory access behavior and maintain stable prediction performance when the operating environment changes. This method needs to not only be able to identify surface patterns, but also adaptively capture and track the dynamic changes behind memory access behavior in order to achieve high-precision and high-reliability prefetching for irregular stepping and complex indexing scenarios. Summary of the Invention
[0005] To overcome the shortcomings of weak generalization ability and insufficient robustness of prefetchers, this invention provides a high-precision memory access prediction and cache prefetching method and system based on machine learning.
[0006] The technical implementation of this invention is a high-precision memory access prediction and cache prefetching method based on machine learning, comprising the following steps:
[0007] S1: Monitor and record the processor's memory access addresses and the identifier sequence corresponding to the memory access instructions in real time; calculate the difference between adjacent memory access addresses as the actual change in memory access addresses; dynamically identify and mark significant non-standard change points in the address sequence; and extract feature values reflecting the hidden patterns of memory access behavior based on the values of the significant non-standard change points through online recursive estimation.
[0008] S2: The feature value reflecting the hidden pattern of memory access behavior is combined with the current memory access address, the change in the current real memory access address, and the identifier sequence corresponding to the memory access instruction to form a combined feature sequence, which is used as the input of the neural network model.
[0009] S3: Train a neural network model using a training method that focuses on predicting unconventional change points. The neural network model is used to predict future memory access address change trends based on combined feature sequences and output the corresponding memory access address prediction results.
[0010] S4: Deploy the trained neural network model in a selective architecture that includes a prefetching mechanism. The selective architecture determines the memory access behavior pattern based on the combined feature sequence extracted in real time and selects to enable the neural network model or generate a prefetch address based on a prefetcher with fixed rules.
[0011] S5: Based on the time series formed by the continuously extracted feature values reflecting the hidden patterns of memory access behavior, when a change is detected in the program execution stage, the trained neural network model deployed in the selective architecture containing multiple prefetching mechanisms is adjusted using the memory address sequence of the initial stage after the change and the identifier sequence corresponding to the memory access instruction.
[0012] S6: During operation, monitor the memory access address prediction results and the processor memory access address sequence that is monitored and recorded in real time, and optimize the neural network model deployed on a selective architecture containing multiple prefetching mechanisms based on the monitoring data.
[0013] Preferably, the dynamic identification and marking of significant unconventional change points in the address sequence includes:
[0014] Calculate the sequence of differences between adjacent memory access addresses, i.e., the sequence of actual memory access address changes;
[0015] Calculate the statistical distribution characteristics of recent difference sequences, and dynamically calculate the numerical limits used to distinguish between regular and non-regular differences based on the statistical distribution characteristics;
[0016] Non-zero difference points that fall outside the stated numerical limits are marked as significant unconventional change points.
[0017] Preferably, the step of extracting feature values reflecting hidden patterns in memory access behavior based on the numerical values at the change points through online recursive estimation includes:
[0018] The numerical values of the marked significant unconventional change points, the first parameter to be determined, the second parameter to be determined, and the assumed changes in internal variables are associated to form a state-space model, which consists of state transition equations and observation equations.
[0019] The state transition equation is: ,in, express The state vector at time t, express The state vector at time t includes the assumed changes in the internal variables, the first parameter to be determined, and the second parameter to be determined; The state transition matrix is a matrix with diagonal elements close to 1 and off-diagonal elements of 0, used to characterize the smooth evolution of the state vector between consecutive time steps. This is the process noise vector;
[0020] The observation equation is: ,in, for The observed value at a given time, i.e., the numerical value at the point of change; for The observation matrix at each time point has a structure determined by the linear observation relationship. To observe noise;
[0021] Based on the state-space model, a recursive estimation problem for the state vector is constructed; an online recursive estimation algorithm is used to solve the recursive estimation problem for the state vector, and the state vector is updated in real time. The estimated value;
[0022] The first and second parameter values corresponding to the state vector obtained at the current moment, as well as the statistical characteristic values of the changes in internal variables over a period of time, are combined to form the feature value that reflects the hidden pattern of memory access behavior.
[0023] Preferably, the step of merging the feature value reflecting the hidden pattern of memory access behavior with the current memory access address, the change in the current actual memory access address, and the identifier sequence corresponding to the memory access instruction to form a combined feature sequence includes:
[0024] The current memory access address, the change in the current actual memory access address, and the feature value reflecting the hidden pattern of memory access behavior, which is composed of the first parameter value to be determined, the second parameter value to be determined, and the statistical feature value of the change in internal variables, are concatenated; the identifier corresponding to the current memory access instruction is converted into a vector representation.
[0025] The concatenated features are connected to the vector representation of the identifier corresponding to the current memory access instruction. The result of the connection is standardized to form a combined feature sequence as input to the neural network model.
[0026] Preferably, training the neural network model using a training method that focuses on predicting unconventional change points includes:
[0027] Obtain a training dataset, which includes a sequence of historical memory access addresses and a corresponding sequence of historical memory access instruction identifiers; calculate the historical real memory access address changes as the training target based on the historical memory access address sequence; identify the historical real memory access address changes and mark significant non-standard change points.
[0028] When training the neural network model, a weighted loss function is used to evaluate the error between the future memory access address change trend predicted by the neural network model and the historical actual memory access address change; the weighted loss function is: Wherein, the prediction error is the difference between the future memory access address change trend predicted by the neural network model and the historical actual memory access address change; the importance coefficient assigned to the training samples is dynamically variable, and when the historical actual memory access address change corresponding to the training sample is not marked as a significant abnormal change point, the importance coefficient is a base value. When the historical real memory access address changes corresponding to the training samples have been marked as significant unconventional change points, the importance coefficient is... The calculation formula is: ;in, This represents the historical real memory access address changes corresponding to the training samples. The absolute value of the historical real memory address changes for the training samples. It is a positive coefficient.
[0029] Preferably, the step of deploying the trained neural network model in a selective architecture containing multiple prefetching mechanisms, wherein the selective architecture determines the memory access behavior pattern based on the real-time extracted combined feature sequence, and selects to enable the neural network model or a prefetcher based on fixed rules to generate prefetch addresses, includes: deploying a selection unit based on judgment logic, wherein the input of the selection unit is a feature value extracted in real time that reflects the hidden pattern of memory access behavior; the selection unit evaluates the regularity of the current memory access behavior pattern based on the statistical feature value of the change in internal variables contained in the feature value; when the statistical feature value of the change in internal variables indicates that the memory access pattern is highly regularized, the prefetcher based on a fixed step pattern is enabled; when the statistical feature value of the change in internal variables indicates that the memory access pattern is irregular, the neural network model is enabled for prediction.
[0030] Preferably, the detection of a change in the program execution stage based on the time series formed by the continuously extracted feature values reflecting the hidden patterns of memory access behavior includes: maintaining a sliding window containing the feature values at M time points in real time; calculating the components of the feature values within the sliding window and forming a statistical vector, wherein the components include the first parameter value to be determined, the second parameter value to be determined, and the mean of the statistical feature values of the internal variable change; comparing the statistical vector with a reference statistical vector and calculating the Euclidean distance as a difference measure; determining that a change in the program execution stage has been detected when the difference measure exceeds a dynamic threshold; calculating the statistical distribution characteristics of the statistical vector sequence, wherein the dynamic threshold is calculated based on the reference statistical vector and the statistical distribution characteristics of the statistical vector sequence, and is used to distinguish between normal fluctuations within a stage and significant changes between stages; wherein the reference statistical vector is statistically determined by the historical feature values corresponding to the program execution stage before the change.
[0031] Preferably, adjusting the trained neural network model deployed in a selective architecture containing multiple prefetching mechanisms using the memory access address sequence and the identifier sequence corresponding to the memory access instructions in the initial stage after the change includes:
[0032] For the detected changed program execution phase, allocate a temporary storage area of fixed capacity to store the initial N combined feature sequences of the changed program execution phase and the corresponding actual memory access address changes.
[0033] In a background processing core independent of the processor, the network layer parameters of the neural network model are frozen at a first preset ratio, and the remaining network layers of the neural network model are updated with gradients a limited number of times using the data stored in the temporary storage area.
[0034] Preferably, during the operation, the memory access address prediction result and the processor memory access address sequence monitored and recorded in real time are monitored, and the neural network model deployed on a selective architecture containing multiple prefetching mechanisms is optimized based on the monitoring data, including: continuously calculating the difference between the memory access address prediction result and the processor memory access address sequence monitored and recorded in real time, storing the difference data in a long-term dataset; when the amount of data in the long-term dataset reaches a preset threshold, updating the parameters of the neural network model using the data in the dataset.
[0035] Preferably, the high-precision memory access prediction and cache prefetching system based on machine learning further includes:
[0036] The memory access monitoring and feature extraction module monitors and records the processor's memory access addresses and the identifier sequence corresponding to memory access instructions in real time; calculates the difference between adjacent memory access addresses as the actual change in memory access addresses; dynamically identifies and marks significant non-standard change points in the address sequence; and extracts feature values reflecting hidden patterns in memory access behavior based on the values of the significant non-standard change points through online recursive estimation.
[0037] The combined feature construction module merges the feature value reflecting the hidden pattern of memory access behavior with the current memory access address, the change in the current real memory access address, and the identifier sequence corresponding to the memory access instruction to form a combined feature sequence, which is used as the input of the neural network model.
[0038] The neural network prediction model module trains the neural network model using a training method that focuses on predicting unconventional change points. The neural network model is used to predict future memory access address change trends based on combined feature sequences and outputs the corresponding memory access address prediction results.
[0039] The prefetch decision and execution module deploys the trained neural network model in a selective architecture that includes a prefetch mechanism. The selective architecture determines the memory access behavior pattern based on the real-time extracted combined feature sequence and selects to enable the neural network model or generate a prefetch address based on a prefetcher with fixed rules.
[0040] The model fast adaptive module, based on the time series formed by the continuously extracted feature values reflecting the hidden patterns of memory access behavior, when it detects a change in the program execution stage, adjusts the trained neural network model deployed in a selective architecture containing multiple prefetching mechanisms using the memory address sequence and the identifier sequence corresponding to the memory access instruction in the initial stage after the change.
[0041] The online model optimization module monitors the memory access address prediction results and the processor memory access address sequence in real time during operation, and optimizes the neural network model deployed on a selective architecture containing multiple prefetching mechanisms based on the monitoring data.
[0042] Beneficial effects: This invention dynamically identifies significant irregular change points in memory access sequences and extracts quantifiable feature values that characterize the inherent laws of memory access behavior based on a state-space model and online recursive estimation; then, it constructs a fused feature model and trains a neural network prediction model using a dynamic weighted loss function oriented towards key jump points, thereby improving the prediction accuracy and generalization ability of irregular access patterns.
[0043] This invention integrates the trained neural network model with a traditional prefetcher into a selective architecture, and dynamically selects the prefetching strategy based on the features extracted in real time, achieving a balance between accuracy and efficiency; further, it introduces a feature sequence-based pattern change detection and fast fine-tuning mechanism, as well as a long-term online optimization closed loop, enabling the system to adapt to changes in program behavior and continuously evolve.
[0044] This invention effectively alleviates the problem of insufficient generalization ability of existing machine learning methods in irregular memory access prediction. Through the collaborative design of the entire process from feature extraction, training optimization to runtime decision-making and adjustment, it improves the hit rate, bandwidth utilization efficiency and overall robustness of the cache prefetching system under complex access patterns. Attached Figure Description
[0045] Figure 1 This is a flowchart of the high-precision memory access prediction and cache prefetching method based on machine learning of the present invention;
[0046] Figure 2 This is a structural diagram of the high-precision memory access prediction and cache prefetching system based on machine learning, as presented in this invention. Detailed Implementation
[0047] The present invention will be further described below with reference to specific embodiments. The illustrative embodiments and descriptions herein are used to explain the present invention, but are not intended to limit the present invention.
[0048] Example 1: A high-precision memory access prediction and cache prefetching method based on machine learning, such as Figure 1 As shown, it includes the following steps:
[0049] S1: Monitor and record the processor's memory access addresses and the identifier sequence corresponding to the memory access instructions in real time; calculate the difference between adjacent memory access addresses as the actual change in memory access addresses; dynamically identify and mark significant non-standard change points in the address sequence; and extract feature values reflecting the hidden patterns of memory access behavior based on the values of the significant non-standard change points through online recursive estimation.
[0050] S2: The feature value reflecting the hidden pattern of memory access behavior is combined with the current memory access address, the change in the current real memory access address, and the identifier sequence corresponding to the memory access instruction to form a combined feature sequence, which is used as the input of the neural network model.
[0051] S3: Train a neural network model using a training method that focuses on predicting unconventional change points. The neural network model is used to predict future memory access address change trends based on combined feature sequences and output the corresponding memory access address prediction results.
[0052] S4: Deploy the trained neural network model in a selective architecture that includes a prefetching mechanism. The selective architecture determines the memory access behavior pattern based on the combined feature sequence extracted in real time and selects to enable the neural network model or generate a prefetch address based on a prefetcher with fixed rules.
[0053] S5: Based on the time series formed by the continuously extracted feature values reflecting the hidden patterns of memory access behavior, when a change is detected in the program execution stage, the trained neural network model deployed in the selective architecture containing multiple prefetching mechanisms is adjusted using the memory address sequence of the initial stage after the change and the identifier sequence corresponding to the memory access instruction.
[0054] S6: During operation, monitor the memory access address prediction results and the processor memory access address sequence that is monitored and recorded in real time, and optimize the neural network model deployed on a selective architecture containing multiple prefetching mechanisms based on the monitoring data.
[0055] Calculate the sequence of differences between adjacent memory access addresses, i.e., the sequence of actual memory access address changes;
[0056] Calculate the statistical distribution characteristics of the difference sequence, and dynamically calculate the numerical limits used to distinguish between regular and non-regular differences based on the statistical distribution characteristics;
[0057] Non-zero difference points that fall outside the stated numerical limits are marked as significant unconventional change points.
[0058] It should be explained that the statistical distribution characteristics of the recent difference sequence are used to dynamically calculate the numerical boundary used to distinguish between regular and irregular differences. The purpose is to establish an adaptive threshold mechanism to separate regular memory access spans from truly irregular access jumps. The mean of the actual memory address change sequence is estimated online and recursively using the exponentially weighted moving average method. with standard deviation The calculation formula is: , ;in for Historical mean estimate at time 10:00 for Historical variance estimates at time points; The forgetting factor, between 0 and 1, is obtained through offline analysis of the statistical characteristics of typical loads and optimized calibration. For example, a value of 0.95 is used to control the weight of historical data and achieve smooth tracking of recent series volatility. The dynamic calculation formula for the numerical limit is: ,in It is a dynamic lower limit. The upper limit is dynamic, and together they constitute a dynamic confidence interval; A preset positive coefficient is obtained through offline analysis and calibration based on statistical distribution characteristics; for example, a value of 2 is used, and the numerical limits constitute a dynamic confidence interval. Non-zero difference points falling outside the numerical limits are marked as points of significant abnormal change. satisfy And fall When the value is outside the interval, it is identified and output as a significant unconventional change point, which serves as the input for the subsequent core feature extraction step.
[0059] The numerical values of the marked significant unconventional change points, the first parameter to be determined, the second parameter to be determined, and the assumed changes in internal variables are associated to form a state-space model, which consists of state transition equations and observation equations.
[0060] The state transition equation is: ,in, express The state vector at time t, express The state vector at time t includes the assumed changes in the internal variables, the first parameter to be determined, and the second parameter to be determined; The state transition matrix is a matrix with diagonal elements close to 1 and off-diagonal elements of 0, used to characterize the smooth evolution of the state vector between consecutive time steps. This is the process noise vector;
[0061] The observation equation is: ,in, for The observed value at a given time, i.e., the numerical value at the point of change; for The observation matrix at each time point has a structure determined by the linear observation relationship. To observe noise;
[0062] Based on the state-space model, a recursive estimation problem for the state vector is constructed; an online recursive estimation algorithm is used to solve the recursive estimation problem for the state vector, and the state vector is updated in real time. The estimated value;
[0063] The first and second parameter values corresponding to the state vector obtained at the current moment, as well as the statistical characteristic values of the changes in internal variables over a period of time, are combined to form the feature value that reflects the hidden pattern of memory access behavior.
[0064] It should be explained that the state-space model aims to model a discrete, disordered numerical sequence of significantly unconventional changes as a program driven by internal hidden states and supporting continuous observation; the state vector ,in The assumed change in internal variables represents the strength of the root cause that drives the jump in memory access address; The first parameter to be determined represents the scaling factor for the transition from the driving strength to the actual address. The second parameter to be determined represents a fixed address offset; the state transition equation The state transition matrix in Set as a diagonal matrix ,in, , and In the state vector , and The autocorrelation coefficients of the three components, This represents a diagonal matrix with these three coefficients as its main diagonal elements; each diagonal element... A constant between 0.9 and 0.999 is used to characterize the high autocorrelation and smooth evolution of the components of the state vector across consecutive time points, assuming that the hidden patterns driving memory access behavior do not change drastically; process noise. Used to model small uncertainties beyond the smoothing assumption; the observation equation A bridge has been established between the hidden state and the observed values; That is, the numerical values of the marked significant unconventional change points; observation matrix The specific structure is based on the linear observation relationship. Determined, linearized to ,in for One-step prediction of state at time step; observation noise Errors used to model linear relationships; the recursive estimation problem of the state vector is given an observation sequence. Under the given conditions, optimally estimate the state sequence The online recursive estimation algorithm employs the recursive least squares method, at each time step... The process is performed recursively in two steps: first, predict the current state based on the state transition equation; then, utilize the latest observations. By comparing the predicted observations, the gain is calculated and the state estimate is updated. Finally, the optimal estimate of the current state vector is output. The calculation formula is as follows: ;in, , and Based on the cutoff date All observation data at time, for The optimal estimates of the changes in internal variables at time t, the first undetermined parameter, and the second undetermined parameter; the combination of the first undetermined parameter value, the second undetermined parameter value corresponding to the state vector obtained at the current time, and the statistical characteristic value of the changes in internal variables over a period of time, to extract the currently estimated... and As an instantaneous feature; simultaneously, within a sliding window of length L, the calculation is performed. mean of the estimated values and standard deviation As a statistical characteristic; ultimately, , , , The features are combined into a feature vector, which is the feature value that reflects the hidden pattern of memory access behavior. The feature value describes the access pattern and intensity of the current memory access instruction from different dimensions.
[0065] The current memory access address, the change in the current actual memory access address, and the feature value reflecting the hidden pattern of memory access behavior, which is composed of the first parameter value to be determined, the second parameter value to be determined, and the statistical feature value of the change in internal variables, are concatenated; the identifier corresponding to the current memory access instruction is converted into a vector representation.
[0066] The concatenated features are connected to the vector representation of the identifier corresponding to the current memory access instruction. The result of the connection is standardized to form a combined feature sequence as input to the neural network model.
[0067] It needs to be explained that the current memory access address, the change in the current actual memory access address, and the statistical characteristic values of the first parameter to be determined, the second parameter to be determined, and the changes in internal variables are combined in a predetermined order to form a one-dimensional original feature vector, for example... ,in This is the current memory access address. This represents the current actual change in memory access addresses. and The values of the first and second parameters to be determined are obtained in real time from the state-space model. and The mean and standard deviation of the internal variable changes within the recent window are statistically characteristic values. An embedded lookup table technique is used to assign a dense vector of fixed dimensions to the memory access instruction identifier. Through a trainable embedding layer, the original discrete high-dimensional memory access instruction identifier is mapped to a low-dimensional continuous real-number vector, transforming the program semantic information into a form suitable for direct processing by the neural network model, i.e., the vector representation of the identifier. The original feature vector and the vector representation of the identifier are concatenated dimensionally to form a comprehensive feature vector that integrates numerical and semantic features. The standardization process uses Z-score standardization, calculating the mean and standard deviation of the numerical dimensions of the comprehensive feature vector on the training dataset. During training and inference, the original value of each dimension is subtracted from the dimension mean and then divided by the standard deviation, resulting in a mean of 0 and a variance of 1 for each dimension.
[0068] Obtain a training dataset, which includes a sequence of historical memory access addresses and a corresponding sequence of historical memory access instruction identifiers; calculate the historical real memory access address changes as the training target based on the historical memory access address sequence; identify the historical real memory access address changes and mark significant non-standard change points.
[0069] When training the neural network model, a weighted loss function is used to evaluate the error between the future memory access address change trend predicted by the neural network model and the historical actual memory access address change; the weighted loss function is: Wherein, the prediction error is the difference between the future memory access address change trend predicted by the neural network model and the historical actual memory access address change; the importance coefficient assigned to the training samples is dynamically variable, and when the historical actual memory access address change corresponding to the training sample is not marked as a significant abnormal change point, the importance coefficient is a base value. When the historical real memory access address changes corresponding to the training samples have been marked as significant unconventional change points, the importance coefficient is... The calculation formula is: ;in, This represents the historical real memory access address changes corresponding to the training samples. The absolute value of the historical real memory address changes for the training samples. It is a positive coefficient.
[0070] It should be explained that the original training dataset consists of complete historical memory access address sequences and corresponding historical memory access instruction identifier sequences generated during the execution of the workload; the differences between adjacent addresses in the historical address sequences are calculated to obtain the sequence. As a neural network model, it needs to learn and predict target labels; for The sequence is scanned and analyzed, for each The samples are labeled with Boolean tags representing points of significant, unconventional change; the weighted loss function aims to guide the neural network model to give different levels of attention to different types of samples during the learning process. The calculation is performed for a training batch, summing over all samples within the batch. ; The prediction error is the error in the neural network model's prediction of the sample. The predicted future memory access address change trend value and the actual The difference between them; the rule for the dynamically variable importance coefficient is that, for training samples in a batch, the real value corresponding to the training sample is checked. The states marked as significant non-standard changes during the preprocessing stage; the importance coefficient of the training samples when they are checked as unlabeled. The basic value is uniformly assigned. ,For example = 1.0; The importance coefficient of the training sample when it is checked that it has been labeled. According to the formula Calculate; where, A pre-defined positive coefficient is used to control the strength of the impact of the jump magnitude on the weights, for example, set between 0.01 and 0.1 through cross-validation; The absolute value of the change at significant unconventional points is used to quantify the magnitude of memory access jumps; in the weighting formula, the larger the jump magnitude, the greater the importance coefficient of the training sample during training; in the backpropagation process of neural network model training, the optimization algorithm minimizes the weighted loss. It passively allocates more model capacity to learn and predict high-weight, irregular key samples.
[0071] A selection unit based on judgment logic is deployed. The input of the selection unit is a feature value extracted in real time that reflects the hidden pattern of memory access behavior. The selection unit evaluates the degree of regularity of the current memory access behavior pattern based on the statistical feature value of the change in internal variables contained in the feature value. When the statistical feature value of the change in internal variables indicates that the memory access pattern is highly regular, a prefetcher based on a fixed step pattern is activated. When the statistical feature value of the change in internal variables indicates that the memory access pattern is irregular, the neural network model is activated for prediction.
[0072] It should be explained that the selection unit based on judgment logic is a lightweight microcode control unit that receives the feature value stream reflecting the hidden rules of memory access behavior from the real-time output of feature extraction; and extracts statistical features from the feature values that characterize the fluctuation intensity of internal variable changes, such as those within the recent window. Standard deviation of the estimated value The assessment of the degree of regularization is simplified to comparing statistical feature values with preset regularity thresholds. Compare; when Determining the current memory access pattern to be highly regular indicates that the latent variables driving memory access are changing steadily; when The pattern is classified as irregular, indicating that the latent variables change drastically and lack a stable pattern; threshold Calibration is performed by analyzing the statistical characteristics of typical loads offline; a prefetcher based on a fixed step pattern is activated after a regular pattern is determined, and the selection unit switches the memory access address prediction task to a low-hardware-overhead prefetcher based on fixed rules, such as the step prefetcher Next-Line; the step prefetcher generates prefetch addresses based on fixed differences in memory access addresses, handling continuous or fixed-step accesses according to rules; the neural network model is activated after an irregular pattern is determined, and the selection unit inputs the combined feature sequence at the current moment into the neural network prediction model, and uses the memory access address prediction results output by the neural network prediction model to generate prefetch addresses.
[0073] A sliding window containing the feature values at M time points is maintained in real time; the components of the feature values within the sliding window are calculated and formed into a statistical vector, the components including the first parameter value to be determined, the second parameter value to be determined, and the mean of the statistical feature values of the internal variable change; the statistical vector is compared with a reference statistical vector, and the Euclidean distance is calculated as a difference measure; when the difference measure exceeds a dynamic threshold, it is determined that a change in the program execution stage has been detected; the statistical distribution characteristics of the statistical vector sequence are calculated, the dynamic threshold is calculated based on the reference statistical vector and the statistical distribution characteristics of the statistical vector sequence, and is used to distinguish between normal fluctuations within a stage and significant changes between stages; wherein, the reference statistical vector is statistically determined by the historical feature values corresponding to the program execution stage before the change.
[0074] It needs to be explained that a fixed-length, first-in-first-out (FIFO) queue of length M is allocated in memory, for example, M=64, to continuously store the feature value vectors reflecting the hidden patterns of memory access behavior extracted at the most recent M time points; the arithmetic mean of the three core components of the feature value vector is independently calculated, and the calculation... The average value is denoted as ;calculate The average value is denoted as ;calculate The average value is denoted as A vector consisting of three means This is the statistical vector, representing the macroscopic average state of the memory access hiding pattern within the current window period; the calculated current statistical vector... The reference statistical vector representing the baseline state of the previous stable phase Comparison, Difference Measurement Calculated using the Euclidean distance formula: ; where subscript Indicates that it is taken from the current statistical vector The quantity, subscript Indicates that it is taken from the reference statistical vector The components, the difference is used to measure the offset of the state in each dimension; distance The overall deviation of the current macroscopic state from the historical baseline state is quantified; the dynamic threshold defines the upper limit of the normal fluctuation range, serving as the basis for determining a significant change in memory access behavior patterns, i.e., a switch in the program execution phase. During a stable phase, statistical vectors are continuously calculated and saved to form a reference sequence, and each component in the sequence is calculated. The standard deviation of is denoted as . The dynamic threshold Calculated based on the statistical distribution characteristics of the reference sequence, and set as follows: ,in Amplification factor, dynamic threshold Defines an upper limit to the normal fluctuation range allowed by historical volatility; continuously compares current difference measures. With dynamic threshold ,satisfy A significant change in memory access behavior pattern, exceeding the normal fluctuations within the phase, is detected as a switch in the program execution phase. Upon detecting this switch, the current statistical vector is updated to the new reference statistical vector. This is used for subsequent comparisons and triggers a rapid adaptive process for the neural network model.
[0075] For the detected changed program execution phase, allocate a temporary storage area of fixed capacity to store the initial N combined feature sequences of the changed program execution phase and the corresponding actual memory access address changes.
[0076] In a background processing core independent of the processor, the network layer parameters of the neural network model are frozen at a first preset ratio, and the remaining network layers of the neural network model are updated with gradients a limited number of times using the data stored in the temporary storage area.
[0077] It should be explained that when a change is detected in the program execution phase, a buffer with a fixed capacity of N records (e.g., N=64) is allocated in memory to temporarily store the initial N combined feature sequences of the changed program execution phase and the corresponding actual changes in memory access addresses. Each record contains a standardized combined feature sequence and the corresponding actual change in memory access address, which serves as a monitoring label. The cache operates on a first-in, first-out (FIFO) logic, stopping collection once it is full of N records, forming a dataset for rapid adaptation of the neural network model. The neural network model is then scheduled to a low-priority background computing unit, such as an E-core in a processor. The first preset ratio is the top 50%, determined through offline experiments. The neural network model is run on a representative workload to test the prediction accuracy after rapid adaptation under different freezing ratios, and the ratio with the highest prediction accuracy is selected as the first preset ratio. The parameters of the top 50% of the shallow and intermediate layers responsible for basic feature extraction in the neural network model are frozen to preserve the general knowledge gained during model training. Only the last few layers of the model are unlocked, such as the last fully connected layer, which is responsible for making the final prediction based on the abstract features extracted by the previous layers of the neural network model. N samples from the temporary storage area are used as training data, with a small learning rate, such as 0.001, to execute... Gradient descent optimization is used to limit the number of iterations, for example, 3 to 5 iterations.
[0078] The difference between the memory access address prediction result and the processor memory access address sequence monitored and recorded in real time is continuously calculated, and the difference data is stored in a long-term dataset; when the amount of data in the long-term dataset reaches a preset threshold, the parameters of the neural network model are updated using the data in the dataset.
[0079] It should be explained that when the neural network prediction model outputs a predicted value for the future trend of memory access address changes... At the same time, it waits for and captures the actual changes in the corresponding real memory access addresses. The difference between the predicted value and the corresponding actual change in memory access address is the error between the two. The calculation formula is: The error value, along with the context information of the time when the prediction was generated, is recorded as difference data. This continuously generated difference data... The data is stored in a long-term database maintained by non-volatile storage, with a capacity much larger than the temporary storage area. A long-term dataset is constructed to store the differential data, organized in a circular queue, covering program behavior within a long time window, and its capacity is determined by the preset threshold Q. When the amount of data in the long-term dataset reaches the preset threshold, it constitutes a condition judgment mechanism to trigger periodic optimization of the model. The preset threshold Q is determined by balancing the model update frequency and the statistical significance of the data. It is determined through offline analysis, run on a representative load, and observe the convergence of the model prediction error. Q is set as the minimum number of samples required for the statistical characteristics of the model error to stabilize. When the number of accumulated differential data reaches Q, it is determined that statistical samples sufficient to reflect the recent general performance deviation of the model have been collected. A batch of data is sampled from the long-term dataset, including the differential values and context information. When the load is low, a background optimization process is started, fixing most of the layers of the neural network model, including the bottom feature extraction layer, and unlocking some of the top layers. Using the sampled data, with a small learning rate such as 0.0001, the Adam optimizer is used to run a few rounds, such as gradient descent optimization for 1-3 epochs. After the update is completed, the long-term dataset is cleared.
[0080] Example 2: Based on Example 1, a high-precision memory access prediction and cache prefetching system based on machine learning, such as... Figure 2 As shown, it includes:
[0081] The memory access monitoring and feature extraction module monitors and records the processor's memory access addresses and the identifier sequence corresponding to memory access instructions in real time; calculates the difference between adjacent memory access addresses as the actual change in memory access addresses; dynamically identifies and marks significant non-standard change points in the address sequence; and extracts feature values reflecting hidden patterns in memory access behavior based on the values of the significant non-standard change points through online recursive estimation.
[0082] The combined feature construction module merges the feature value reflecting the hidden pattern of memory access behavior with the current memory access address, the change in the current real memory access address, and the identifier sequence corresponding to the memory access instruction to form a combined feature sequence, which is used as the input of the neural network model.
[0083] The neural network prediction model module trains the neural network model using a training method that focuses on predicting unconventional change points. The neural network model is used to predict future memory access address change trends based on combined feature sequences and outputs the corresponding memory access address prediction results.
[0084] The prefetch decision and execution module deploys the trained neural network model in a selective architecture that includes a prefetch mechanism. The selective architecture determines the memory access behavior pattern based on the real-time extracted combined feature sequence and selects to enable the neural network model or generate a prefetch address based on a prefetcher with fixed rules.
[0085] The model fast adaptive module, based on the time series formed by the continuously extracted feature values reflecting the hidden patterns of memory access behavior, when it detects a change in the program execution stage, adjusts the trained neural network model deployed in a selective architecture containing multiple prefetching mechanisms using the memory address sequence and the identifier sequence corresponding to the memory access instruction in the initial stage after the change.
[0086] The online model optimization module monitors the memory access address prediction results and the processor memory access address sequence in real time during operation, and optimizes the neural network model deployed on a selective architecture containing multiple prefetching mechanisms based on the monitoring data.
[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A high-precision memory access prediction and cache prefetching method based on machine learning, characterized in that, Includes the following steps: S1: Monitor and record the processor's memory access addresses and the identifier sequence corresponding to the memory access instructions in real time; calculate the difference between adjacent memory access addresses as the actual change in memory access addresses; dynamically identify and mark significant non-standard change points in the address sequence; and extract feature values reflecting the hidden patterns of memory access behavior based on the values of the significant non-standard change points through online recursive estimation. S2: The feature value reflecting the hidden pattern of memory access behavior is combined with the current memory access address, the change in the current real memory access address, and the identifier sequence corresponding to the memory access instruction to form a combined feature sequence, which is used as the input of the neural network model. S3: Train a neural network model using a training method that focuses on predicting unconventional change points. The neural network model is used to predict future memory access address change trends based on combined feature sequences and output the corresponding memory access address prediction results. S4: Deploy the trained neural network model in a selective architecture that includes a prefetching mechanism. The selective architecture determines the memory access behavior pattern based on the combined feature sequence extracted in real time and selects to enable the neural network model or generate a prefetch address based on a prefetcher with fixed rules. S5: Based on the time series formed by the continuously extracted feature values reflecting the hidden patterns of memory access behavior, when a change is detected in the program execution stage, the trained neural network model deployed in the selective architecture containing multiple prefetching mechanisms is adjusted using the memory address sequence of the initial stage after the change and the identifier sequence corresponding to the memory access instruction. S6: During operation, monitor the memory access address prediction results and the processor memory access address sequence that is monitored and recorded in real time, and optimize the neural network model deployed on a selective architecture containing multiple prefetching mechanisms based on the monitoring data.
2. The high-precision memory access prediction and cache prefetching method based on machine learning according to claim 1, characterized in that, The dynamic identification and marking of significant unconventional changes in the address sequence includes: Calculate the sequence of differences between adjacent memory access addresses, i.e., the sequence of actual memory access address changes; Calculate the statistical distribution characteristics of the difference sequence, and dynamically calculate the numerical limits used to distinguish between regular and non-regular differences based on the statistical distribution characteristics; Non-zero difference points that fall outside the stated numerical limits are marked as significant unconventional change points.
3. The high-precision memory access prediction and cache prefetching method based on machine learning according to claim 1, characterized in that, The feature values reflecting hidden patterns in memory access behavior are extracted based on the numerical values at the change points through online recursive estimation, including: The numerical values of the marked significant unconventional change points, the first parameter to be determined, the second parameter to be determined, and the assumed changes in internal variables are associated to form a state-space model, which consists of state transition equations and observation equations. The state transition equation is: ,in, express The state vector at time t, express The state vector at time t includes the assumed changes in the internal variables, the first parameter to be determined, and the second parameter to be determined; The state transition matrix is a matrix with diagonal elements close to 1 and off-diagonal elements of 0, used to characterize the smooth evolution of the state vector between consecutive time steps. This is the process noise vector; The observation equation is: ,in, for The observed value at a given time, i.e., the numerical value at the point of change; for The observation matrix at each time point has a structure determined by the linear observation relationship. To observe noise; Based on the state-space model, a recursive estimation problem for the state vector is constructed; an online recursive estimation algorithm is used to solve the recursive estimation problem for the state vector, and the state vector is updated in real time. The estimated value; The first and second parameter values corresponding to the state vector obtained at the current moment, as well as the statistical characteristic values of the changes in internal variables over a period of time, are combined to form the feature value that reflects the hidden pattern of memory access behavior.
4. The high-precision memory access prediction and cache prefetching method based on machine learning according to claim 1, characterized in that, The step of merging the feature value reflecting the hidden pattern of memory access behavior with the current memory access address, the change in the current actual memory access address, and the identifier sequence corresponding to the memory access instruction to form a combined feature sequence includes: The current memory access address, the change in the current actual memory access address, and the feature value reflecting the hidden pattern of memory access behavior, which is composed of the first parameter value to be determined, the second parameter value to be determined, and the statistical feature value of the change in internal variables, are concatenated; the identifier corresponding to the current memory access instruction is converted into a vector representation. The concatenated features are connected to the vector representation of the identifier corresponding to the current memory access instruction. The result of the connection is standardized to form a combined feature sequence as input to the neural network model.
5. The high-precision memory access prediction and cache prefetching method based on machine learning according to claim 1, characterized in that, The training of the neural network model using a training method that focuses on predicting unconventional change points includes: Obtain a training dataset, which includes a sequence of historical memory access addresses and a corresponding sequence of historical memory access instruction identifiers; calculate the historical real memory access address changes as the training target based on the historical memory access address sequence; identify the historical real memory access address changes and mark significant non-standard change points. When training the neural network model, a weighted loss function is used to evaluate the error between the future memory access address change trend predicted by the neural network model and the historical actual memory access address change; the weighted loss function is: Wherein, the prediction error is the difference between the future memory access address change trend predicted by the neural network model and the historical actual memory access address change; the importance coefficient assigned to the training samples is dynamically variable, and when the historical actual memory access address change corresponding to the training sample is not marked as a significant abnormal change point, the importance coefficient is a base value. When the historical real memory access address changes corresponding to the training samples have been marked as significant unconventional change points, the importance coefficient is... The calculation formula is: ;in, This represents the historical real memory access address changes corresponding to the training samples. The absolute value of the historical real memory address changes for the training samples. It is a positive coefficient.
6. The high-precision memory access prediction and cache prefetching method based on machine learning according to claim 1, characterized in that, The step involves deploying the trained neural network model in a selective architecture that includes multiple prefetching mechanisms. The selective architecture determines memory access behavior patterns based on real-time extracted combined feature sequences and selects to enable either the neural network model or a prefetcher based on fixed rules to generate prefetched addresses. This includes: deploying a selection unit based on judgment logic; the input of the selection unit is real-time extracted feature values reflecting hidden patterns in memory access behavior; the selection unit evaluates the regularity of the current memory access behavior pattern based on statistical feature values of internal variable changes contained in the feature values; when the statistical feature values of internal variable changes indicate that the memory access pattern is highly regularized, a prefetcher based on fixed step patterns is enabled; when the statistical feature values of internal variable changes indicate that the memory access pattern is irregular, the neural network model is enabled for prediction.
7. The high-precision memory access prediction and cache prefetching method based on machine learning according to claim 6, characterized in that, The time series formed by continuously extracted feature values reflecting hidden patterns in memory access behavior, when detecting changes in the program execution phase, includes: maintaining a sliding window containing the feature values at M time points in real time; calculating the components of the feature values within the sliding window and constructing a statistical vector, wherein the components include the first undetermined parameter value, the second undetermined parameter value, and the mean of the statistical feature values of the internal variable change; comparing the statistical vector with a reference statistical vector and calculating the Euclidean distance as a difference measure; determining that a change in the program execution phase has been detected when the difference measure exceeds a dynamic threshold; calculating the statistical distribution characteristics of the statistical vector sequence, wherein the dynamic threshold is calculated based on the reference statistical vector and the statistical distribution characteristics of the statistical vector sequence, used to distinguish between normal fluctuations within a phase and significant changes between phases; wherein the reference statistical vector is statistically determined by the historical feature values corresponding to the program execution phase before the change.
8. The high-precision memory access prediction and cache prefetching method based on machine learning according to claim 1, characterized in that, The adjustment of the trained neural network model deployed in a selective architecture containing multiple prefetching mechanisms by utilizing the memory access address sequence and the identifier sequence corresponding to the memory access instructions in the initial stage after the change includes: For the detected changed program execution phase, allocate a temporary storage area of fixed capacity to store the initial N combined feature sequences of the changed program execution phase and the corresponding actual memory access address changes. In a background processing core independent of the processor, the network layer parameters of the neural network model are frozen at a first preset ratio, and the remaining network layers of the neural network model are updated with gradients a limited number of times using the data stored in the temporary storage area.
9. The high-precision memory access prediction and cache prefetching method based on machine learning according to claim 1, characterized in that, During operation, the memory access address prediction result and the processor memory access address sequence monitored and recorded in real time are monitored. Based on the monitoring data, the neural network model deployed on a selective architecture containing multiple prefetching mechanisms is optimized, including: continuously calculating the difference between the memory access address prediction result and the processor memory access address sequence monitored and recorded in real time, and storing the difference data in a long-term dataset; when the amount of data in the long-term dataset reaches a preset threshold, the parameters of the neural network model are updated using the data in the dataset.
10. A high-precision memory access prediction and cache prefetching system based on machine learning, used to implement the high-precision memory access prediction and cache prefetching method based on machine learning as described in any one of claims 1-9, characterized in that, The above also includes: The memory access monitoring and feature extraction module monitors and records the processor's memory access addresses and the identifier sequence corresponding to memory access instructions in real time; calculates the difference between adjacent memory access addresses as the actual change in memory access addresses; dynamically identifies and marks significant non-standard change points in the address sequence; and extracts feature values reflecting hidden patterns in memory access behavior based on the values of the significant non-standard change points through online recursive estimation. The combined feature construction module merges the feature value reflecting the hidden pattern of memory access behavior with the current memory access address, the change in the current real memory access address, and the identifier sequence corresponding to the memory access instruction to form a combined feature sequence, which is used as the input of the neural network model. The neural network prediction model module trains the neural network model using a training method that focuses on predicting unconventional change points. The neural network model is used to predict future memory access address change trends based on combined feature sequences and outputs the corresponding memory access address prediction results. The prefetch decision and execution module deploys the trained neural network model in a selective architecture that includes a prefetch mechanism. The selective architecture determines the memory access behavior pattern based on the real-time extracted combined feature sequence and selects to enable the neural network model or generate a prefetch address based on a prefetcher with fixed rules. The model fast adaptive module, based on the time series formed by the continuously extracted feature values reflecting the hidden patterns of memory access behavior, when it detects a change in the program execution stage, adjusts the trained neural network model deployed in a selective architecture containing multiple prefetching mechanisms using the memory address sequence and the identifier sequence corresponding to the memory access instruction in the initial stage after the change. The online model optimization module monitors the memory access address prediction results and the processor memory access address sequence in real time during operation, and optimizes the neural network model deployed on a selective architecture containing multiple prefetching mechanisms based on the monitoring data.