Data prefetching method and device based on neural network dynamic parameter adjustment, equipment and medium

Abstract

This invention discloses a data prefetching method, apparatus, device, and medium based on dynamic parameter tuning of a neural network, relating to the field of data prefetching technology. The method includes: obtaining the current access address of the CPU's L2 cache at the current moment when a prefetch hit or miss occurs using a preset optimal offset prefetcher; determining the current prefetch address based on the current access address and the current optimal offset; the current optimal offset being 0 or an offset generated at the end of the previous training cycle; inputting the access address of the previous training cycle, the current prefetch address, the current optimal offset, and a preset bias term into the target neural network to obtain a target output value; determining a target threshold based on the target output value; determining a new optimal offset based on the target threshold using the preset optimal offset prefetcher; determining a new prefetch address based on the new optimal offset; and performing data prefetching of the L2 cache using the new prefetch address. Therefore, this invention can effectively improve data prefetching accuracy and coverage.

Description

Technical Field

[0001] This invention relates to the field of data prefetching technology, and in particular to data prefetching methods, apparatus, devices and media based on dynamic parameter tuning of neural networks. Background Technology

[0002] With the rapid development of information technology, the processing speed of CPUs has increased rapidly, while the speed of memory (mainly RAM) has increased relatively slowly, leading to a speed difference between memory and CPU, also known as the memory wall. For example, based on statistics from 2001 to 2005, processor clock frequencies increased by 55% annually, while main memory speeds increased by only 7% annually, a difference of approximately 48% per year. Due to this speed difference, when the processor needs to access data in memory, it may have to wait due to memory access latency, resulting in a performance bottleneck. This performance bottleneck is particularly pronounced under high load or large data processing scenarios, severely impacting the overall performance of the computer system. To alleviate the speed difference between processor and memory, modern computer systems have proposed a cache structure based on the principle of locality of memory access. Cache is a hidden, small-capacity, fast SRAM (Static Random-Access Memory) added between the CPU (Central Processing Unit) and main memory based on the principle of locality of reference. By keeping the most useful data close to the processor, retaining as much recently used data as possible, and replacing as little recently used data as possible when the capacity is insufficient, most data access can be performed in the fast SRAM, thereby improving system performance.

[0003] Data prefetching refers to loading necessary data into the cache before the program accesses it, thereby avoiding cache misses and improving cache performance. Compared to the processor accessing data directly from main memory, prefetching data and accessing data from the cache is obviously much faster. By predicting subsequent memory accesses, it can effectively imply potential long-latency behaviors, thus hiding memory access latency. The advantage of this method is that it can reduce the time the program waits for data to be read, thereby improving the program's running speed. To address the timeliness issue of prefetching, the best offset prefetcher (BOP) is mainly used to find the accurate and timely offset of the prefetch request. The training result of the optimal offset of the BOP algorithm is determined based on the offset score table, and the two most important parameters of the offset score table are the best score and the bad score. However, the current bestscore and badscore of BOP are manually set, but the characteristics of program flow and data access sequence are constantly changing. Fixed bestscore and badscore settings will limit the capabilities of BOP prefetcher. For example, if the badscore is set very low, BOP prefetching will become very aggressive, and if the bestscore is set very high, BOP prefetching will become very conservative. Therefore, the fixed parameter design makes it difficult for BOP to achieve good accuracy and coverage when facing complex and diverse program loads and data flows.

[0004] It is evident that achieving good prefetch accuracy and coverage in the face of varied and complex access sequences is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a data prefetching method, apparatus, device, and medium based on dynamic parameter tuning of a neural network. This method replaces static thresholds with dynamic parameter generation, enabling the BOP prefetcher to have adaptive decision-making capabilities and achieve real-time dynamic generation of the target threshold. This allows for good prefetching accuracy and coverage even when facing complex and variable access sequences. The specific solution is as follows: In a first aspect, the present invention discloses a data prefetching method based on dynamic parameter tuning of a neural network, comprising: The current access address when a prefetch hit or a prefetch miss occurs in the CPU's L2 cache at the current moment is obtained by a preset optimal offset prefetcher. The current prefetch address is determined based on the current access address and the current optimal offset. The current optimal offset is 0 or the offset generated at the end of the previous training cycle. Input the access address of the previous training cycle, the current prefetch address, the current optimal offset, and the preset bias term into the target neural network to obtain the target output value; The target threshold is determined based on the target output value, and a new optimal offset is determined based on the target threshold by a preset optimal offset prefetcher. A new prefetch address is determined based on the new optimal offset, and data prefetching of the second-level cache is performed using the new prefetch address.

[0006] Optionally, the current prefetch address is determined based on the current access address and the current optimal offset, including: The sum of the current access address and the current optimal offset is used to determine the current prefetch address.

[0007] Optionally, before inputting the access address from the previous training cycle, the current prefetch address, the current optimal offset, and the preset bias term into the target neural network, the following may also be included: Determine the target difference between the current system target address, the previous training cycle's access address, and the current optimal offset; Determine the target product between the square of the target difference and the target value; the target value is 0.5. The target product is defined as the target error; Based on the target error, preset momentum factor, and preset learning rate, backpropagation is performed using gradient descent to adjust the weights of each connection in the preset neural network in order to obtain the corresponding target neural network. The target neural network is a three-layer backpropagation neural network, which includes 4 input layer nodes, 2 output layer neuron nodes, and 5 hidden layer neuron nodes.

[0008] Optionally, the target threshold can be determined based on the target output value, including: The target threshold is determined based on the product of the target output value and the target amplification factor.

[0009] Optionally, the target threshold includes a first threshold and a second threshold; Accordingly, a new optimal offset is determined based on the target threshold using a preset optimal offset prefetcher, including: When the preset optimal offset prefetcher is trained based on a preset number of training rounds, if there is an offset score corresponding to a target offset that is equal to the first threshold, then the target offset is directly determined as the new optimal offset; the offset score is a score used to characterize the number of times the offset is effective; After the number of training rounds of the preset optimal offset prefetcher is greater than the preset number of training rounds, if the first offset score corresponding to each offset is less than the first threshold and greater than the second threshold, then the largest offset score among the first offset scores is determined. The offset corresponding to the largest offset score is determined as the new optimal offset.

[0010] Optionally, data prefetching methods based on neural network dynamic parameter tuning also include: After the number of training rounds of the preset optimal offset prefetcher is equal to the preset number of training rounds, if the second offset score corresponding to each offset is less than the second threshold, the preset optimal offset prefetcher is judged to have failed to train and no data prefetching is performed.

[0011] Optionally, a new prefetch address is determined based on the new optimal offset, including: The sum of the new optimal offset and the current access address is used to determine the new prefetch address.

[0012] Secondly, the present invention discloses a data prefetching device based on dynamic parameter tuning of a neural network, comprising: The current prefetch address determination module is used to obtain the current access address of the CPU's L2 cache when a prefetch hit or a prefetch miss occurs at the current moment through a preset optimal offset prefetcher, and determine the current prefetch address based on the current access address and the current optimal offset; the current optimal offset is 0 or the offset generated at the end of the previous training cycle; The target output value determination module is used to input the access address of the previous training cycle, the current prefetch address, the current optimal offset, and the preset bias term into the target neural network to obtain the target output value. The new optimal offset determination module is used to determine the target threshold based on the target output value, and to determine the new optimal offset based on the target threshold through the preset optimal offset prefetcher; The data prefetch module is used to determine a new prefetch address based on the new optimal offset, and to perform data prefetching in the second-level cache using the new prefetch address.

[0013] Thirdly, the present invention discloses an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute computer programs to implement data prefetching methods based on neural network dynamic parameter tuning, as described above.

[0014] Fourthly, the present invention discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned data prefetching method based on dynamic parameter tuning of a neural network.

[0015] In this invention, the current access address of the CPU's L2 cache at the current moment, when a prefetch hit or prefetch miss occurs, is first obtained through a preset optimal offset prefetcher. The current prefetch address is then determined based on the current access address and the current optimal offset. The current optimal offset is either 0 or the offset generated at the end of the previous training cycle. Then, the access address of the previous training cycle, the current prefetch address, the current optimal offset, and a preset bias term are input into the target neural network to obtain the target output value. Next, a target threshold is determined based on the target output value, and a new optimal offset is determined based on the target threshold using the preset optimal offset prefetcher. Finally, a new prefetch address is determined based on the new optimal offset, and data prefetching of the L2 cache is performed using the new prefetch address.

[0016] Beneficial effects: This invention deeply integrates a neural network model with the classic BOP prefetcher, and innovatively replaces the static threshold with dynamic parameter generation, enabling the BOP prefetcher to have adaptive decision-making capabilities, realize the real-time dynamic generation of the target threshold, and thus control the BOP to train a suitable optimal offset according to the target threshold, maintaining a balance between aggressive and conservative strategies, thereby achieving good prefetch accuracy and coverage even when facing varied and complex access sequences. Attached Figure Description

[0017] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a data prefetching method based on dynamic parameter tuning of a neural network is provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a BOP algorithm prefetching method for L2cache provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a three-layer neural network topology provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a BOP training approach provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a three-layer neural network training process provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of a data prefetching device based on dynamic parameter tuning of a neural network, provided in an embodiment of the present invention. Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0020] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0021] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] In some techniques, the optimal training offset for the BOP algorithm is determined based on an offset score table. The two most important parameters of this table are the best score and the bad score, which are categorized as follows: 1. If the best score is reached within the training epochs (before reaching the maximum number of training epochs), training ends and the offset corresponding to that best score is directly selected as the optimal offset. BOP then uses this offset as the actual prefetch offset for prefetching operations. 2. After reaching the maximum number of training epochs, if the best score is reached, training ends and the offset corresponding to that best score is selected as the optimal offset. If any of the corresponding offset scores fall between the best score and the bad score, the largest score among these scores is selected, and its corresponding offset becomes the offset for subsequent prefetching. 3. After reaching the maximum number of training rounds, if the offset scores corresponding to each offset are all below the bad score, then the training round is considered to have failed. This means that BOP cannot capture a valid data access pattern during this period, so no offset is generated, and prefetching will not occur for a period of time. However, training will continue until a matching offset score appears again in subsequent training, at which point prefetching will continue. However, the bestscore and badscore of the BOP mentioned above are manually set, but the characteristics of program flow and data access sequence are constantly changing. Fixed bestscore and badscore settings will limit the capabilities of the BOP prefetcher. For example, a very low badscore will make BOP prefetching too aggressive, while a very high bestscore will make BOP prefetching too conservative. Therefore, the fixed parameter design makes it difficult for BOP to achieve good accuracy and coverage when facing complex and diverse program loads and data flows. To solve the above technical problems, this invention discloses a data prefetching method, apparatus, device, and medium based on dynamic parameter tuning of neural networks. It can replace static thresholds with dynamic parameter generation, enabling the BOP prefetcher to have adaptive decision-making capabilities and realize real-time dynamic generation of target thresholds, thereby achieving good prefetching accuracy and coverage even when facing complex and varied access sequences.

[0023] See Figure 1 As shown, this embodiment of the invention provides a data prefetching method based on dynamic parameter tuning of a neural network, including: Step S11: Obtain the current access address of the CPU's L2 cache when a prefetch hit or a prefetch miss occurs at the current moment through the preset optimal offset prefetcher, and determine the current prefetch address based on the current access address and the current optimal offset; the current optimal offset is 0 or the offset generated at the end of the previous training cycle.

[0024] In this embodiment of the invention, in the BOP algorithm implementation, D is the current prefetch offset, which is dynamically adjusted according to the application's behavior. The optimal offset learning algorithm finds the optimal prefetch offset by testing several different offset values, and also includes a list of offsets and scores. The Recent Request (RR) table records the base addresses of completed prefetch requests; the base address refers to the address used to trigger the prefetch request. Compared to sandbox prefetching, BOP uses a selection mechanism unique to offset prefetching, where the RR table is updated using the actual prefetched offset, rather than the result of a hypothetical simulation.

[0025] Specifically, when a read request from line X in the L1-Cache accesses the L2-Cache, if a cache miss or prefetch hit occurs and line X+D is in the same memory page as line X, the prefetch request for line X+D is sent to the L3-Cache. When a miss or prefetch hit occurs in line X, the offset d in the offset list is also tested. If Xd matches a value in the RR table, it indicates that the most recently issued prefetch request for line X-d+D has been completed, and the score of the corresponding offset d will increase. Each offset in the list is tested in turn during one round of testing. After the learning phase ends, the best offset with the highest score is used as the new prefetch offset D, and a new learning phase begins. However, if the best score is not greater than the set BADSCORE threshold, prefetching is disabled, and best offset learning continues. When prefetching is enabled, for each prefetch line Y inserted from L3 into L2, YD is written to the RR table.

[0026] The optimal offset prefetcher categorizes all prefetch offsets into two types: optimal offsets and unqualified offsets. Unqualified offsets are not allowed to be used to issue any prefetch requests and are discarded. Only optimal offset values ​​are retained.

[0027] The BOP prefetcher predicts future memory accesses by dynamically learning memory access patterns. It uses an RR table to track recent prefetch requests and updates the offset score based on the access pattern. Through a learning phase, the BOP prefetcher determines the optimal offset and generates prefetch requests accordingly. Furthermore, the BOP prefetcher uses a delay queue to control the timing of prefetch requests, ensuring timeliness. Initialization: During initialization, the BOP prefetcher sets some basic parameters, such as the maximum score (score_max), the maximum number of rounds (round_max), the score for disabling hardware prefetching (bad_score), and the number of entries per RR bank (rr_size). Listening for Memory Accesses: The BOP prefetcher receives memory access events through a ProbeListener. For each memory access, the prefetcher generates a prefetch request based on the current learning phase and the offset list. Updating the RR Table and Delay Queue: When a memory access occurs, the BOP prefetcher checks if the accessed address is in the recent request table (RR table). If so, the corresponding offset score is increased. Simultaneously, if a deferred queue is enabled, the BOP prefetcher inserts the address into the deferred queue and processes it after a specified deferred period. Learning the optimal offset: The BOP prefetcher maintains a list of offsets and determines the optimal offset through learning phases. In each learning phase, the prefetcher tests the offsets in the list and updates the score based on the access pattern. When the score reaches the maximum score or the number of rounds reaches the maximum number of rounds, the learning phase ends, and the optimal offset is selected as the new prefetch offset. Generating prefetch requests: Once the optimal offset is determined, the BOP prefetcher generates prefetch requests based on this offset. If prefetch requests are enabled, the prefetcher calculates the prefetch address and adds it to the prefetch request list. Updating the RR table: When a prefetched row is inserted into the cache, the BOP prefetcher updates the RR table so that this offset can be used for prefetching in future accesses. Processing the deferred queue: The BOP prefetcher periodically checks the deferred queue and inserts the addresses in the queue into the RR table after a specified deferred period. This helps the prefetcher issue prefetch requests at the appropriate time.

[0028] In embodiments of the present invention, such as Figure 2 As shown, after an L2 cache miss or prefetch hit occurs at the current moment, the BOP obtains the address Xi. Assuming it's the i-th training iteration in a certain round of the current training cycle, and the current optimal offset D comes from the offset generated at the end of the previous training cycle, then the BOP simultaneously calculates the current prefetch address Y = Xi + D. In other words, after obtaining the current access address when a prefetch hit or miss occurs in the CPU's L2 cache using the preset optimal offset prefetcher, the sum of the current access address and the current optimal offset is determined as the current prefetch address.

[0029] Step S12: Input the access address of the previous training cycle, the current prefetch address, the current optimal offset, and the preset bias term into the target neural network to obtain the target output value.

[0030] In this embodiment of the invention, the training result of the optimal offset of the BOP algorithm is determined based on the offset score table, and the two most important parameters of the offset score table are the best score and the bad score. Therefore, this application uses a three-layer BP neural network (Back Propagation Neural Network, BPNN) to train the bad score and best score in real time online, thereby training the optimal bad score and best score at the current time, thus optimizing the selection strategy of the optimal offset D, and ultimately improving the prefetch accuracy and coverage of the BOP prefetcher. The topology of the three-layer BPNN is shown below. Figure 3 Backpropagation Neural Network (BPNN) is a multi-layered feedforward neural network trained using the backpropagation algorithm. The network consists of an input layer, hidden layers, and an output layer. The input signal is processed by the hidden layers and then passed to the output layer. The state of each neuron in each layer is only affected by the neurons in the previous layer. When the output layer fails to generate the desired output, the error signal is backpropagated, correcting itself layer by layer along the network path. By repeatedly adjusting the connection weights of each neuron, the error gradually decreases along the gradient, ultimately minimizing the difference between the actual output and the target output.

[0031] The 3-layer BP-NN in this invention is constructed as follows: the input layer has 4 nodes, namely the trigger address of the previous time step (i.e., the address of the previous miss / prefetch hit), the current prefetch address Y, the current best offset D, and the bias term 1. The bias term is the output value of the neuron when the system has no input, which is beneficial to the stability and convergence of the neural network; the output layer has 2 neuron nodes, namely the two threshold parameters badscore and bestscore of the offset score; finally, the hidden layer has 5 neuron nodes, which are set through debugging experience. Therefore, before inputting the access address of the previous training cycle, the current prefetch address, the current optimal offset, and the preset bias term into the target neural network, the target difference between the access address of the current system target time, the access address of the previous training cycle, and the current optimal offset is determined; the target product between the square of the target difference and the target value is determined; the target value is 0.5; the target product is determined as the target error; based on the target error, the preset momentum factor, and the preset learning rate, backpropagation is performed using gradient descent to adjust the weights of each connection in the preset neural network to obtain the corresponding target neural network; wherein, the target neural network is a three-layer backpropagation neural network; the target neural network includes 4 input layer nodes, 2 output layer neuron nodes, and 5 hidden layer neuron nodes. Among them, the hidden layer includes, but is not limited to, 5 neurons. This number is set according to "tuning experience" and is the result of balancing model complexity and expressive power. The hidden layer uses tanh as the activation function, which can output positive and negative values ​​and handle nonlinear relationships. Output layer: 2 neurons, which are the two dynamic parameters required by this invention: L=1: the output value, which, after scaling, is the badscore. L=2: The output value, after scaling, is the bestscore. The output layer uses the Sigmoid function to compress the output value to the range (0, 1), which is convenient for subsequent mapping to the actual score threshold range (such as 10-32).

[0032] Specifically, this invention designs an error function: ; Where E(k) is the square root of the error; The actual memory address accessed at time k; This is the address (previous address) at time k-1 recorded in the RR table; Let D(k) be the optimal offset used by the BOP at time k. This error measures the difference between the theoretical prediction of the BOP and the actual memory access value. Ideally, if D(k) is a perfect offset, then the current memory access address Xi(k) should be exactly equal to the previous address X(k) plus this offset, i.e., Xi(k) = X(k) + D(k). In this case, the error e(k) = 0. If e(k) is large, it means that the current D(k) prediction is inaccurate. This error function directly links the output of the neural network (badscore / bestscore) to the prefetch accuracy of the BOP. The training goal of the neural network is to adjust the weights so that the D(k) selected by the badscore / bestscore generated by it makes this error E(k) as small as possible. In other words, it teaches the neural network to set a threshold to select the offset that can most accurately predict the future address.

[0033] Then, based on the calculated error E(k), the weights (w) of each connection in the network are adjusted layer by layer using backpropagation via gradient descent. Momentum factor α: accelerates convergence and prevents the training process from oscillating violently around local optima. Learning rate η: controls the step size of each weight update.

[0034] In this way, the four input features—the previous access address, the currently calculated prefetch address, the currently used optimal offset, and the bias term—are fed into the BP neural network. After calculation, two output values ​​are obtained. Then, using the calculated error E(k), all weights w of the neural network are updated through backpropagation. In this way, the neural network "learns" from the prediction result. This process continues online. The neural network acts like a real-time online "strategy optimizer," constantly observing the performance of the BOP (through error) and dynamically fine-tuning its decision threshold (badscore / bestscore) to guide the BOP to choose a better offset D, thus forming a virtuous cycle of "better performance—better threshold—better performance."

[0035] Step S13: Determine the target threshold based on the target output value, and determine a new optimal offset based on the target threshold using a preset optimal offset prefetcher.

[0036] In this embodiment of the invention, a target threshold is determined based on the product of the target output value and the target amplification factor. The target threshold includes a first threshold and a second threshold. The output value of the neural network (between 0 and 1) is mapped to the actual threshold range through an "amplification factor" to obtain the real-time badscore(k) (second threshold) and bestscore(k) (first threshold) at the current time k. The BOP immediately uses these two new thresholds for subsequent offset score evaluation and selection.

[0037] Subsequently, when the preset optimal offset prefetcher is trained based on the preset number of training epochs, if there exists an offset score corresponding to a target offset that is equal to the first threshold, then the target offset is directly determined as the new optimal offset; the offset score is a score used to characterize the number of times the offset is effective; after the number of training epochs of the preset optimal offset prefetcher is greater than the preset number of training epochs, if the first offset score corresponding to each offset is less than the first threshold and greater than the second threshold, then the largest offset score among the first offset scores is determined; the offset corresponding to the largest offset score is determined as the new optimal offset.

[0038] Specifically, BOP simultaneously calculates the current prefetch address Y = Xi + D, and the difference between address Xi and the corresponding offset di in the offset table, i.e., Xi - di. This difference is compared with the previous address X in the recent request table (RR table). If they are equal, it means address Xi is obtained by adding the previous optimal offset to the previous address X, i.e., Xi = X + D. This indicates that di = D, meaning Xi is a valid prefetch address. In this case, BOP increments the offset score di_score corresponding to offset di by 1. Otherwise, if they are not equal, di_score remains unchanged, and the next training iteration begins directly, i.e., X(i+1) corresponds to the training of d(i+1). Simultaneously, Xi is sent to a FIFO (First Input First Output) delay queue, waits for a period of time, and then is placed into the RR table. The prefetch queue simulates the delay caused by the prefetch operation. Thus, when accessing the next cache address X(i+1), if the RR table... If Xi is not in the table class, it means that the prefetch operation issued by Xi has not been completed, so the offset score corresponding to offset d(i+1) will not be increased, thus ensuring the authenticity of training.

[0039] In addition, regarding parameter settings, the RR table has 64 entries, meaning it can store the 64 most recent addresses; both the offset table and the offset fraction table are set to 46 entries, with each entry corresponding to an offset. From entry0 to entry45, the values ​​are set as {1, -1, 2, -2, 3, -3, 4, -4, 5, -5, 6, -6, 8, -8, 9, -9, 10, -10, 12, -12, 15, - 15, 16, -16, 18, -18, 20, -20, 24, -24, 25, -25, 27, -27, 30, -30, 32, -32, 36, -36, 40, -40, 45, -45, 48, -48}, where a negative offset indicates forward prefetching; each corresponding offset score entry is initially set to 0, the maximum number of training rounds is set to 100 rounds, and each round is trained 46 times, that is, each offset is trained once.

[0040] When training BOP, the training approach is as follows: Figure 4 As shown, if a score corresponding to a certain offset reaches the best score ahead of schedule during 100 training rounds, the training ends and the offset corresponding to the best score is directly selected as the optimal offset. The BOP then uses this offset as the actual prefetch offset for prefetching operations. After 100 training rounds, if there are scores between the best score and the bad score among the offset scores corresponding to each offset, the largest score among these scores is selected, and its corresponding offset is the offset for prefetching by the subsequent prefetcher.

[0041] Furthermore, after the number of training epochs for the preset optimal offset prefetcher equals the preset number of training epochs, if the second offset score corresponding to each offset is less than the second threshold, the preset optimal offset prefetcher is considered to have failed, and data prefetching is not performed. In a specific embodiment, after reaching 100 training epochs, if the offset score corresponding to each offset is below the badscore, the training for that epoch is considered to have failed. This means that the BOP cannot capture a valid data access pattern during that period, and therefore no offset is generated. In other words, prefetching will not occur for a period of time, but training will continue until a matching offset score appears again in subsequent training, at which point prefetching will continue.

[0042] Step S14: Determine a new prefetch address based on the new optimal offset, and perform data prefetching in the second-level cache using the new prefetch address.

[0043] In this embodiment of the invention, the sum of the new optimal offset and the current access address is determined as the new prefetch address. Simultaneously, the current prefetch address Y=Xi+D is sent to a FIFO-style prefetch queue, issuing a prefetch request to the bus. This request retrieves data from the lower-level storage LLC (Last Level Cache) via the bus. Afterwards, if the prefetch request is allowed, the prefetch address and data are returned to the L2 cache, completing the prefetch operation.

[0044] In this way, by constructing a neural network model with a real-time feedback mechanism, the activation values ​​of the L1 / L2 neurons in the output layer are synchronously calculated in each clock cycle k, and multiplied by the dynamic method coefficients respectively to generate adaptive badscore and bestscore in real time. This dynamic parameter generation mechanism enables the prefetcher to autonomously adjust the training strategy of the optimal offset based on the spatiotemporal locality characteristics of the current data flow: when a regular access pattern is detected, the prefetching initiative is enhanced by increasing the bestscore; when encountering irregular data flow, the badscore is increased to suppress over-prefetching. This neural network-based dynamic parameter balancing mechanism enables the BOP prefetcher to achieve an adaptive trade-off between a conservative strategy (reducing cache pollution) and an aggressive strategy (improving coverage), ultimately achieving both high prefetch accuracy and memory access coverage in complex and variable program load environments.

[0045] Furthermore, this invention is applicable to various prefetcher optimizations. For existing fixed-parameter prefetchers (such as fixed cache thresholds and static prefetch step sizes), their parameters can be replaced with aggressive / conservative coefficients dynamically generated by a neural network. For example, the prefetch range can be adjusted according to the real-time data access pattern, or the prefetch depth or priority can be dynamically changed in combination with the data flow. Only the neural network adaptive training module of this solution needs to be embedded in the original prefetch logic to achieve the upgrade from a fixed strategy to an adaptive strategy without changing the original prefetcher core architecture.

[0046] Beneficial effects: This invention deeply integrates a neural network model with the classic BOP prefetcher, and innovatively replaces the static threshold with dynamic parameter generation, enabling the BOP prefetcher to have adaptive decision-making capabilities, realize the real-time dynamic generation of the target threshold, and thus control the BOP to train a suitable optimal offset according to the target threshold, maintaining a balance between aggressive and conservative strategies, thereby achieving good prefetch accuracy and coverage even when facing varied and complex access sequences.

[0047] As described in the previous embodiment, this invention discloses a three-layer BP neural network. The construction of the three-layer BP-NN will now be explained in detail.

[0048] In a three-layer backpropagation (BP) neural network, four nodes are selected in the input layer: the trigger address of the previous time step (i.e., the address of the previous miss / prefetch hit), the current prefetch address Y, the current best offset D, and the bias term 1. The bias term is the output value of the neuron when the system has no input, which is beneficial to the stability and convergence of the neural network. The output layer has two neuron nodes, namely the two threshold parameters badscore and bestscore of the offset score. Finally, the number of hidden layer neuron nodes is set to five based on debugging experience.

[0049] like Figure 3 As shown, J represents the input layer neuron nodes, J=1,2,3,4; M represents the hidden layer neuron nodes, M=1,2,3,4,5; and L represents the output layer neurons, L=1,2. The 3-layer BPNN propagation training algorithm is as follows: The formula for calculating the input layer of a BPNN is: ; In the formula: This represents the output value of the J-th neuron in the input layer. This equation indicates that the input layer does not perform nonlinear transformations and directly transmits data. x is the net network input of the Jth neuron in the input layer; x(J) is the Jth external raw input feature received by the network; the superscript 1 indicates the network input layer.

[0050] The formula for calculating hidden layer neurons is: ; In the formula: g(x) is the code name of the general transformation function used by the network node, called the activation function; tanh(x) is the transformation function that takes positive and negative symmetry, which is used to perform nonlinear mapping and limit the output value to the symmetric interval [-1, 1]; e is the natural constant.

[0051] The formulas for calculating the input and output of the hidden layer of the network are as follows: ; In the formula: the superscript 2 indicates the hidden layer of the network; The coefficients of the weight matrix connecting the J-th node in the input layer to the M-th node in the hidden layer; W represents the network connection weights; This is an accumulation and summation operator used to linearly combine the weighted signals of all predecessor nodes; This is the actual output value of the Mth neuron in the hidden layer after being processed by the activation function g(x). This data will be used as the input source for the next layer (output layer). This represents the weighted sum of inputs received by the Mth neuron in the hidden layer.

[0052] The transformation function for the output layer neurons is the Sigmoid function, obtained by linearly transforming the hyperbolic tangent function. It is used for the nonlinear mapping of the output layer nodes, and its calculation formula is as follows: ; The input and output of the network output layer are: ; In the formula: the superscript 3 indicates the network output layer; The weight parameters are used to connect the hidden layer node M in the previous stage with the output layer node L in the next stage. This is the sum of the weighted signal values ​​received by the output layer; This represents the final response output value of the output layer neuron.

[0053] Using the quadratic form of the output error as a performance metric and optimizing it through backpropagation, its performance evaluation function can be expressed as: ; In the formula: E(k) is the quadratic objective function (mean square error) constructed based on the output error; Xi(k) is the access address of the current system at time k, X(k) is the previous access address, D(k) is the optimal offset at time k; e(k) is the comprehensive error term calculated from the above three system variables.

[0054] To optimize the network weights using gradient descent and suppress potential oscillations and inertia during training, a momentum factor is introduced. Therefore, the weight update formula for the BP-NN output layer can be expressed as: ; In the formula: This is the weight adjustment amount for the current k periods, specifically... For difference operators, it represents the parameter correction value obtained from word iteration calculation. This represents the network connection weight, which is the same physical quantity as W in the previous formula.

[0055] α is the momentum factor, introduced by the weight increment of k-1 from the previous period. This is used to smooth the parameter update trajectory and suppress local oscillations during the optimization process. n is the learning rate, which determines the step size based on the gradient direction. It is the partial derivative of the target loss function with respect to the weights from the hidden layer to the output layer.

[0056] To simplify the expression, take For intermediate variables in the output layer weight update, the formula is: ; in This is a symbolic function used to extract symbolic features from internal algebraic expressions.

[0057] and These represent the system access addresses at time k and time k-1, respectively.

[0058] and This represents the change in the output value of the output layer node at the corresponding discrete time.

[0059] This is the derivative of the output layer transformation function with respect to the net input of the network.

[0060] Therefore, the formula for updating the weights of the output layer is: ; In the formula, This is the response signal transmitted by the previous hidden layer node at time k.

[0061] Similarly, the formula for updating the hidden layer weights is: ; In the formula: This represents the increment of the connection weight from the J-th node in the input layer to the M-th node in the hidden layer during the k-th iteration.

[0062] As an intermediate variable in the hidden layer weight update, its formula is: ; in Perform the accumulation operation on the weighted error signals that are propagated backward to all two nodes of the next level (output layer). This represents the derivative of the hidden layer activation function with respect to the net input of the network.

[0063] The training process for a 3-layer BPNN is as follows: Figure 5 As shown, the backpropagation algorithm of a 3-layer BPNN can calculate the values ​​of the two output layer neuron nodes L=1 and 2 at the current time k in real time. Then, multiplying the neuron output value by a method coefficient, the real-time badscore and bestscore at the current time can be obtained. These parameters can reach full optimality at the current time. Therefore, when facing irregular data streams, the BOP prefetcher can train the optimal offset and offset score more timely and reasonably based on the trained badscore and bestscore, thereby calculating the appropriate prefetch address. This controls BOP to become a balanced prefetcher that is neither conservative nor aggressive, and can meet the requirements of high accuracy and coverage under complex program loads.

[0064] The training mechanism (process as follows) Figure 5(As shown) By suppressing parameter oscillations through the momentum term, the network can achieve continuous address sequences. ( Driven by this mechanism, the weight matrix is ​​continuously optimized, dynamically generating a bestscore / badscore parameter combination that matches the current data flow pattern. When a regular access pattern is detected, the network automatically increases the bestscore to enhance prefetching initiative; when faced with irregular data flows, it increases the badscore to suppress over-prefetching, ultimately achieving a dynamic balance between prefetching aggressiveness and conservatism, effectively improving the accuracy and coverage of the BOP prefetcher under complex loads.

[0065] Beneficial effects: By deeply integrating the BP neural network with the classic BOP prefetcher, this invention innovatively replaces the static threshold with dynamic parameter generation, enabling the BOP prefetcher to have adaptive decision-making capabilities and achieve real-time dynamic generation of badscore / bestscore. Compared with the shortcomings of existing technologies where static parameters require manual tuning and cannot adapt to load changes, the neural network model of this invention can generate adaptive parameter combinations online in each clock cycle k based on the spatiotemporal locality characteristics of the data flow, solving the problem of mismatch between regular and irregular access patterns caused by parameter fixation in traditional methods.

[0066] See Figure 6 As shown, this embodiment of the invention provides a data prefetching device based on dynamic parameter tuning of a neural network, comprising: The current prefetch address determination module 11 is used to obtain the current access address when the CPU's L2 cache has a prefetch hit or a prefetch miss at the current moment through a preset optimal offset prefetcher, and determine the current prefetch address based on the current access address and the current optimal offset; the current optimal offset is 0 or the offset generated at the end of the previous training cycle; The target output value determination module 12 is used to input the access address of the previous training cycle, the current prefetch address, the current optimal offset, and the preset bias term into the target neural network to obtain the target output value; The new optimal offset determination module 13 is used to determine the target threshold based on the target output value, and to determine the new optimal offset based on the target threshold by a preset optimal offset prefetcher. The data prefetch module 14 is used to determine a new prefetch address based on the new optimal offset, and to perform data prefetching of the second-level cache using the new prefetch address.

[0067] Since the embodiments of the device part correspond to the embodiments described above, please refer to the embodiments described in the method part for the embodiments of the device part, and will not be repeated here.

[0068] Beneficial effects: This invention deeply integrates a neural network model with the classic BOP prefetcher, and innovatively replaces the static threshold with dynamic parameter generation, enabling the BOP prefetcher to have adaptive decision-making capabilities, realize the real-time dynamic generation of the target threshold, and thus control the BOP to train a suitable optimal offset according to the target threshold, maintaining a balance between aggressive and conservative strategies, thereby achieving good prefetch accuracy and coverage even when facing varied and complex access sequences.

[0069] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the data prefetching method based on neural network dynamic parameter tuning disclosed in any of the foregoing embodiments. Furthermore, the electronic device in this embodiment may specifically be an electronic computer.

[0070] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0071] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0072] The operating system 221 is used to manage and control the various hardware devices on the electronic device and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the data prefetching method based on neural network dynamic parameter tuning disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0073] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned data prefetching method based on dynamic parameter tuning of a neural network. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0074] Furthermore, this application also discloses a computer program product, including a computer program / instructions; wherein, when the computer program / instructions are executed by a processor, they implement the aforementioned data prefetching method based on dynamic parameter tuning of a neural network. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0075] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0076] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0077] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0078] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0079] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data prefetching method based on dynamic parameter tuning of a neural network, characterized in that, include: The current access address of the CPU's L2 cache at the current moment is obtained by a preset optimal offset prefetcher when a prefetch hit or a prefetch miss occurs. The current prefetch address is determined based on the current access address and the current optimal offset. The current optimal offset is 0 or the offset generated at the end of the previous training cycle. Input the access address of the previous training cycle, the current prefetch address, the current optimal offset, and the preset bias term into the target neural network to obtain the target output value; A target threshold is determined based on the target output value, and a new optimal offset is determined based on the target threshold by the preset optimal offset prefetcher. A new prefetch address is determined based on the new optimal offset, and data prefetching of the second-level cache is performed using the new prefetch address.

2. The data prefetching method based on dynamic parameter tuning of a neural network according to claim 1, characterized in that, The step of determining the current prefetch address based on the current access address and the current optimal offset includes: The sum of the current access address and the current optimal offset is used to determine the current prefetch address.

3. The data prefetching method based on dynamic parameter tuning of a neural network according to claim 1, characterized in that, Before inputting the access address of the previous training cycle, the current prefetch address, the current optimal offset, and the preset bias term into the target neural network, the method further includes: Determine the target difference between the current system target address, the previous training cycle's access address, and the current optimal offset; Determine the target product between the square of the target difference and the target value; the target value is 0.

5. The target product is determined as the target error; Based on the target error, preset momentum factor, and preset learning rate, backpropagation using gradient descent is performed to adjust the weights of each connection in the preset neural network in order to obtain the corresponding target neural network. The target neural network is a three-layer backpropagation neural network; the target neural network includes 4 input layer nodes, 2 output layer neuron nodes, and 5 hidden layer neuron nodes.

4. The data prefetching method based on dynamic parameter tuning of a neural network according to claim 1, characterized in that, Determining the target threshold based on the target output value includes: The target threshold is determined based on the product of the target output value and the target amplification factor.

5. The data prefetching method based on dynamic parameter tuning of a neural network according to claim 1, characterized in that, The target threshold includes a first threshold and a second threshold; Accordingly, determining a new optimal offset based on the target threshold using the preset optimal offset prefetcher includes: When the preset optimal offset prefetcher is trained based on a preset number of training rounds, if there exists an offset score corresponding to a target offset that is equal to the first threshold, then the target offset is directly determined as the new optimal offset; the offset score is a score used to characterize the number of times the offset is effective; After the number of training rounds of the preset optimal offset prefetcher is greater than the preset number of training rounds, if the first offset score corresponding to each offset is less than the first threshold and greater than the second threshold, then the largest offset score among the first offset scores is determined. The offset corresponding to the largest offset score is determined as the new optimal offset.

6. The data prefetching method based on dynamic parameter tuning of a neural network according to claim 5, characterized in that, Also includes: After the number of training rounds of the preset optimal offset prefetcher is equal to the preset number of training rounds, if the second offset score corresponding to each offset is less than the second threshold, then the training of the preset optimal offset prefetcher is determined to be failed, and no data prefetching is performed.

7. The data prefetching method based on dynamic parameter tuning of a neural network according to any one of claims 1 to 6, characterized in that, Determining the new prefetch address based on the new optimal offset includes: The sum of the new optimal offset and the current access address is used to determine the new prefetch address.

8. A data prefetching device based on dynamic parameter tuning of a neural network, characterized in that, include: The current prefetch address determination module is used to obtain the current access address of the CPU's L2 cache when a prefetch hit or a prefetch miss occurs at the current moment through a preset optimal offset prefetcher, and determine the current prefetch address based on the current access address and the current optimal offset; the current optimal offset is 0 or the offset generated at the end of the previous training cycle; The target output value determination module is used to input the access address of the previous training cycle, the current prefetch address, the current optimal offset, and the preset bias term into the target neural network to obtain the target output value. The new optimal offset determination module is used to determine a target threshold based on the target output value, and to determine a new optimal offset based on the target threshold through the preset optimal offset prefetcher; The data prefetch module is used to determine a new prefetch address based on the new optimal offset, and to perform data prefetching of the second-level cache using the new prefetch address.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data prefetching method based on neural network dynamic parameter tuning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data prefetching method based on dynamic parameter tuning of a neural network as described in any one of claims 1 to 7.

Images