A hardware prefetch system

Abstract

The present invention is applicable to the field of processor technology, and in particular relates to a hardware prefetch system. The hardware prefetch system includes a processor, a first-level cache, a second-level cache, a third-level cache, a prefetch module, and a memory; the prefetch module includes a prefetch management unit, a stride prefetch unit, an optimal offset value prefetch unit, and a Markov prefetch unit; the prefetch management unit is used to dynamically select the stride prefetch unit, the optimal offset value prefetch unit, and the Markov prefetch unit; the stride prefetch unit is used to issue a prefetch request based on the stride between the current access address of the processor and the historical access address of the processor; the optimal offset value prefetch unit is used to issue a prefetch request based on the optimal offset value between the current access address and the historical access address. Compared with the prior art, the present invention can select different prefetch units according to the current access address of the processor through the prefetcher management module, thereby improving the accuracy of prefetching.

Description

Technical Field

[0001] The present invention is applicable to the field of processor technology, and in particular relates to a hardware prefetching system. Background Art

[0002] Hardware prefetching is a key feature of modern high-performance processors. Prefetching algorithms can be used to extract data from the next-level cache or memory into the cache in advance, thereby reducing cache miss rates. Modern high-performance processors generally have a three-level cache hierarchy. To address the simple and regular memory access patterns of the first-level cache, many hardware and software prefetching methods have been integrated into modern microprocessors to prefetch instructions and data. However, when it comes to secondary or even higher-level caches, the sources of addresses become multiple. When multiple address sequences are interleaved and sent, the regularity of a single address sequence is disrupted. As a result, the processor is often idle when it needs to request a memory block due to a miss in a higher cache level.

[0003] Conventional stride prefetchers and Markov prefetchers have good prediction rates for single-core address sequences, but have difficulty predicting the patterns of multi-core address sequences. The optimal offset prefetcher finds the optimal offset value by training on a longer historical address sequence, resulting in higher coverage and accuracy in multi-core address sequences and enabling timely prefetching.

[0004] like Figure 1 As shown, Figure 1 The following is a flowchart of the offset prefetch algorithm used in related art. Its core is to determine an address offset value {D} and add it to the current memory access address {X} to generate a prefetch address {X + D}. The offset value {D} is not fixed and can be adaptively changed based on the scenario. The prefetch address is generated by adding the offset value to the required access address. The optimal offset value {D} is automatically found by training a set of offset values ​​{d}.

[0005] However, the training time of the optimal offset prefetcher is too long compared to other prefetchers, such as Figure 2 As shown, Figure 2 This is the offset value training flow chart of the offset value prefetching algorithm in the related art. Each offset value has a corresponding score. At the beginning of the training phase, the scores of all offset values ​​are reset to 0. At each eligible L2 cache read access (miss or prefetch hit), the offset value {d i}. If {X−d i}, then the offset value {d i}'s score increases. In a round, each offset value in the list is tested once: d1 is tested on the first visit of the round, d2 is tested on the next visit, then d3, and so on. When all offset values ​​in the list have been tested, the current round ends and a new round begins, starting with offset d1. When an offset value's score reaches the set threshold, training ends and that offset value is considered the optimal offset value. The training duration is the product of the number of offset values ​​and the threshold.

[0006] While the existing optimal offset value prefetcher can predict multiple intertwined address sequences, its training time is too long. When there is only a single address sequence, the long training time may cause the address sequence pattern to change after training, making the prefetch results inaccurate and causing cache pollution. When there is only a single address sequence, stride prefetcher and Markov prefetcher can more accurately predict the changing pattern of the address sequence, so the prefetcher needs to be able to flexibly and dynamically switch. The training time of the existing optimal offset value prefetcher is dozens of times longer than that of stride prefetcher and Markov prefetcher, which prevents the three prefetchers from being trained simultaneously, making it difficult to achieve the goal of flexible prefetcher switching.

[0007] Therefore, a new hardware prefetching system is urgently needed to solve the above technical problems. Summary of the Invention

[0008] The present invention provides a hardware prefetching system, which can flexibly select one of stride prefetching, Markov prefetching and optimal offset value prefetching for prefetching, thereby improving prediction efficiency and prediction accuracy and reducing broadband resources.

[0009] The present invention provides a hardware prefetching system, which includes a processor, a first-level cache, a second-level cache, a third-level cache, a prefetching module and a memory;

[0010] The processor is used to read and execute instructions in the first-level cache, the second-level cache, the third-level cache and the memory;

[0011] The first-level cache is used to cache instructions;

[0012] The secondary cache is used to cache instructions and serves as an object for the processor to read instructions when the processor reads the primary cache and misses;

[0013] The third-level cache is used to cache instructions, and serves as the object of the processor to read instructions when the processor reads instructions from both the first-level cache and the second-level cache and misses;

[0014] The memory is used to store instructions and serves as an object for the processor to read instructions when the processor reads instructions from the first-level cache, the second-level cache, and the third-level cache and none of them hits;

[0015] The prefetch module is used to issue prefetch requests to the secondary cache, the third-level cache and the memory respectively;

[0016] The prefetch module includes a prefetch management unit, a stride prefetch unit, an optimal offset value prefetch unit, and a Markov prefetch unit;

[0017] The prefetch management unit is used to dynamically select one of the stride prefetch unit, the optimal offset value prefetch unit and the Markov prefetch unit, and issue a prefetch request to the secondary cache, the third level cache and the memory through the selected unit;

[0018] The stride prefetch unit is used to issue a prefetch request to the L2 cache, the L3 cache and the memory according to the stride between the current access address of the processor and the historical access address of the processor;

[0019] The optimal offset value prefetch unit is used to issue a prefetch request to the secondary cache, the third level cache and the memory according to the optimal offset value between the current access address and the historical access address;

[0020] The Markov prefetch unit is used for issuing a prefetch request to the L2 cache, the L3 cache and the memory according to the difference between the current access address and the historical access address.

[0021] Preferably, the optimal offset value pre-fetching unit includes an optimal offset value table, a recent request table, and a first page boundary checking unit;

[0022] The optimal offset value table is used to store a plurality of preset offset values, and select the optimal offset value from the preset offset values ​​according to the current access address and a preset rule and send the optimal offset value to the first page boundary checking unit;

[0023] The recent request table is used to record the historical access addresses;

[0024] The first page boundary check unit is used to calculate an offset value prefetch address based on the received optimal offset value and the current access address, and perform a page boundary check on the offset value prefetch address to determine whether the offset value prefetch address exceeds the address access boundary: if not, the first page boundary check unit issues a prefetch request based on the offset value prefetch address.

[0025] Preferably, the preset rules are:

[0026] When the processor reads a miss: the optimal offset value table performs parallel training on each of the preset offset values ​​according to the current access address based on the offset value training algorithm to obtain a training result corresponding to the preset offset value;

[0027] The recent request table matches each of the training results with the historical access address to obtain a training score corresponding to the training result;

[0028] When a training score corresponding to one of the training results is equal to a first preset threshold, the preset offset value corresponding to the training result is used as the optimal offset value.

[0029] Preferably, the stride prefetch unit includes a first reference prediction table, a tag scanning unit, a stride comparison unit, a first prefetch request generation unit, and a second page boundary checking unit;

[0030] The first reference prediction table is used to store the historical access address, multiple stride values, and stride confidences corresponding to the stride values;

[0031] The label scanning unit is used to check and update the labels of the first reference prediction table;

[0032] The stride comparison unit is configured to calculate a current stride value according to the current access address, and compare the current stride value with all the stride values ​​in the first reference prediction table, and send a match signal to the first prefetch request generation unit if the current stride value matches a stride value in the first reference prediction table and the corresponding stride confidence is greater than or equal to a second preset threshold;

[0033] The first prefetch request generating unit is configured to determine, based on the received matching signal, whether there is a stride value in the first reference prediction table, the stride confidence of which is greater than or equal to a second preset threshold; if so, generate a stride prefetch address, and send the stride prefetch address to the second page boundary checking unit;

[0034] The second page boundary check unit is used to perform a page boundary check on the received stride prefetch address to determine whether the stride prefetch address exceeds the address access boundary: if not, the second page boundary check unit issues a prefetch request according to the stride prefetch address.

[0035] Preferably, the Markov prefetch unit includes a difference calculation unit, a difference shift register, a difference index scanning unit, a second reference prediction table, a confidence check unit, a prediction difference table, a second prefetch request generation unit, and a third page boundary check unit;

[0036] The second reference prediction table is used to store a difference index, a plurality of predicted differences, and difference confidences corresponding to the predicted differences;

[0037] The difference calculation unit is used to calculate a current difference according to the current access address, and send the current difference to the difference shift register;

[0038] The difference shift register is used to store the received current difference value and generate a current difference value index, and send the current difference value index and the current difference value to the difference index scanning unit and the predicted difference value table respectively;

[0039] The difference index scanning unit is configured to check and update the difference index of the second reference prediction table according to the received current difference index;

[0040] The confidence checking unit is configured to update the difference confidences in the second reference prediction table, and send a prediction difference verification signal to the prediction difference table if one of the difference confidences is greater than or equal to a third preset threshold;

[0041] The prediction difference table is used to update the prediction difference value in the second reference prediction table according to the received current difference value, and read the corresponding prediction difference value from the second reference prediction table according to the received prediction difference value verification signal, and output it to the second prefetch request generating unit;

[0042] The second prefetch request generating unit is configured to generate a Markov prefetch address according to the current access address and the received prediction difference value, and send the Markov prefetch address to the third page boundary checking unit;

[0043] The third page boundary check unit is used to perform a page boundary check on the received Markov prefetch address to determine whether the Markov prefetch address exceeds the address access boundary. If not, the third page boundary check unit issues a prefetch request according to the Markov prefetch address.

[0044] Preferably, the prefetch management unit includes a stride prefetch counter, a Markov prefetch counter, an optimal offset value prefetch counter, and a prefetcher status unit;

[0045] The stride prefetch counter is used to record the confidence of the stride prefetch unit in the training mode;

[0046] The Markov prefetch counter is used to record the confidence of the Markov prefetch unit in the training mode;

[0047] The optimal offset value pre-fetch counter is used to record the confidence of the optimal offset value pre-fetch unit in the training mode;

[0048] The prefetcher status unit is used to adjust the working states of the stride prefetch unit, the Markov prefetch unit and the optimal offset value prefetch unit according to the count values ​​of the stride prefetch counter, the Markov prefetch counter and the optimal offset value prefetch counter respectively.

[0049] Preferably, when the count value corresponding to any one of the stride prefetch counter, the Markov prefetch counter and the optimal offset value prefetch counter is greater than or equal to a preset count threshold, the corresponding stride prefetch unit or the Markov prefetch unit or the optimal offset value prefetch unit is enabled to issue a prefetch request, and the remaining units stop training.

[0050] Preferably, when the count values ​​corresponding to at least two of the stride prefetch counter, the Markov prefetch counter and the optimal offset value prefetch counter are simultaneously greater than or equal to the preset count threshold, the selection priority of the prefetch management unit is from first to last: the stride prefetch unit, the Markov prefetch unit, and the optimal offset value prefetch unit.

[0051] Preferably, the prefetch module further includes an output buffer unit; the output buffer unit is used to temporarily store the prefetch request issued by the prefetch module.

[0052] Compared to existing technologies, the prefetch module proposed in this invention is used for prefetching at all levels of cache. For the second and third level caches, if there is only a single address sequence, stride prefetch units and Markov prefetch units can be selected based on a fixed address pattern. If multiple address sequences exist, the optimal offset prefetch unit can find the optimal offset value for prefetching. The prefetcher management module can select different prefetch units based on the processor's current access address, thereby improving prefetch accuracy. Furthermore, the output buffer unit can cancel prefetch requests when they time out or when the prefetch output queue overflows, thereby reducing bandwidth resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be described in detail below with reference to the accompanying drawings. The above and other aspects of the present invention will become clearer and easier to understand through the detailed description made with reference to the following drawings. In the accompanying drawings:

[0054] Figure 1 is a flow chart of an offset value prefetching algorithm in the related art;

[0055] Figure 2 is an offset value training flow chart of an offset value prefetching algorithm in the related art;

[0056] Figure 3 1 is a schematic structural diagram of a hardware prefetching system provided by an embodiment of the present invention;

[0057] Figure 4 1 is a schematic structural diagram of an optimal offset value prefetching unit of a hardware prefetching system provided by an embodiment of the present invention;

[0058] Figure 5 is a flow chart of an offset value training algorithm for a hardware prefetch system provided by an embodiment of the present invention;

[0059] Figure 6 1 is a schematic structural diagram of a stride prefetch unit of a hardware prefetch system provided by an embodiment of the present invention;

[0060] Figure 7 1 is a schematic structural diagram of a Markov prefetch unit of a hardware prefetch system provided by an embodiment of the present invention;

[0061] Figure 8 1 is a schematic structural diagram of a prefetch management unit of a hardware prefetch system provided by an embodiment of the present invention;

[0062] Figure 9 is a logical schematic diagram of a prefetcher state unit of a hardware prefetching system provided by an embodiment of the present invention;

[0063] Figure 10 It is a structural diagram of an output buffer unit of a hardware prefetch system provided by an embodiment of the present invention.

[0064] In the figure, 100, hardware prefetch system; 1, processor; 2, first-level cache; 3, second-level cache; 4, third-level cache; 5, prefetch module; 51, prefetch management unit; 511, stride prefetch counter; 512, Markov prefetch counter; 513, best offset value prefetch counter; 514, prefetcher status unit; 52, stride prefetch unit; 521, first reference prediction table; 522, tag scanning unit; 523, stride comparison unit; 524, first prefetch request generation unit; 525, second page boundary Check unit; 53. Optimal offset value prefetch unit; 531. Optimal offset value table; 532. Most recent request table; 533. First page boundary check unit; 54. Markov prefetch unit; 541. Difference calculation unit; 542. Difference shift register; 543. Second reference prediction table; 544. Confidence check unit; 545. Prediction difference table; 546. Second prefetch request generation unit; 547. Third page boundary check unit; 548. Difference index scanning unit; 55. Output buffer unit; 6. Memory. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0066] Please refer to Figure 3 The present invention provides a hardware prefetch system 100, which includes a processor 1, a first-level cache 2, a second-level cache 3, a third-level cache 4, a prefetch module 5 and a memory 6.

[0067] The processor 1 is used to read and execute instructions in the first-level cache 2, the second-level cache 3, the third-level cache 4 and the memory 6.

[0068] The first-level cache 2 is used to cache instructions.

[0069] The secondary cache 3 is used to cache instructions, and serves as an object for the processor 1 to read instructions when the processor 1 reads the primary cache 2 but misses.

[0070] The L3 cache 4 is used to cache instructions, and serves as the target of instructions read by the processor 1 when the processor 1 reads instructions from both the L1 cache 2 and the L2 cache 3 but misses.

[0071] The memory 6 is used to store instructions and serves as an object for the processor 1 to read instructions when the processor 1 reads instructions from the first-level cache 2, the second-level cache 3 and the third-level cache 4 but none of them hits.

[0072] The prefetch module 5 is used to issue prefetch requests to the L2 cache 3, the L3 cache 4 and the memory 6. Specifically, the prefetch module 5 includes multiple modules respectively arranged between the L1 cache 2, the L2 cache 3, the L3 cache 4 and the memory 6.

[0073] The prefetch module 5 includes a prefetch management unit 51 , a stride prefetch unit 52 , an optimal offset value prefetch unit 53 , and a Markov prefetch unit 54 .

[0074] The prefetch management unit 51 is used to dynamically select one of the stride prefetch unit 52, the optimal offset value prefetch unit 53 and the Markov prefetch unit 54, and issue a prefetch request to the secondary cache 3, the tertiary cache 4 and the memory 6 through the selected unit.

[0075] The stride prefetch unit 52 is configured to issue prefetch requests to the L2 cache 3 , the L3 cache 4 and the memory 6 according to the stride between the current access address of the processor 1 and the historical access address of the processor 1 .

[0076] The optimal offset value prefetch unit 53 is configured to issue a prefetch request to the L2 cache 3 , the L3 cache 4 , and the memory 6 according to the optimal offset value between the current access address and the historical access address.

[0077] The Markov prefetch unit 54 is configured to issue a prefetch request to the L2 cache 3 , the L3 cache 4 and the memory 6 according to the difference between the current access address and the historical access address.

[0078] In the embodiment of the present invention, please refer to Figure 4 , Figure 4 FIG. 5 is a schematic diagram of the structure of the optimal offset value prefetching unit 53 of the hardware prefetching system 100 provided by an embodiment of the present invention. The optimal offset value prefetching unit 53 includes an optimal offset value table 531 , a most recent request table 532 , and a first page boundary checking unit 533 .

[0079] The optimal offset value table 531 is used to store multiple preset offset values, and select the optimal offset value from the preset offset values ​​according to the current access address and preset rules and send it to the first page boundary checking unit 533. The preset rules are:

[0080] When the processor 1 reads a miss, the optimal offset value table 531 performs parallel training on each of the preset offset values ​​according to the current access address based on the offset value training algorithm to obtain a training result corresponding to the preset offset value;

[0081] The recent request table 532 matches each of the training results with the historical access address to obtain a training score corresponding to the training result;

[0082] When a training score corresponding to one of the training results is equal to a first preset threshold, the preset offset value corresponding to the training result is used as the optimal offset value.

[0083] The recent request table 532 is used to record the historical access addresses;

[0084] The first page boundary check unit 533 is used to calculate the offset value prefetch address based on the received optimal offset value and the current access address, and perform a page boundary check on the offset value prefetch address to determine whether the offset value prefetch address exceeds the address access boundary: if not, the first page boundary check unit 533 issues a prefetch request based on the offset value prefetch address.

[0085] Specifically, the steps of issuing a prefetch request through the optimal offset value prefetch unit 53 are as follows:

[0086] When the processor 1 misses the cache or hits the prefetch, the current access address of the processor 1 is subtracted from each preset offset value in the optimal offset value table 531. A test tag is sent to the recent request table 532 based on each subtraction result to determine whether it hits the historical access address in the recent request table 532. If there is a hit, the training score is increased by one. When one of the training scores is equal to the first preset threshold, the corresponding preset offset value is used as the optimal offset value and sent to the first page boundary check unit 533. The first page boundary check unit 533 calculates the offset value prefetch address based on the optimal offset value and the current access address, and performs a page boundary check on the offset value prefetch address. When the offset value prefetch address is within the address access boundary range, a prefetch request is issued. The optimal offset value prefetch unit 53 is used for prefetching the third-level cache 4 and the memory 6. When prefetching the second-level cache 3, it degenerates into a next-row prefetcher (i.e., the offset value is fixed to 1).

[0087] Please refer to Figure 5 , Figure 5 This is a flowchart of the offset value training algorithm of the hardware prefetch system 100 provided by an embodiment of the present invention. Since the optimal offset value prefetch unit 53 proposed in the present invention compares all preset offset values ​​each time during training, it will occupy a large amount of combinational logic. Therefore, the offset value list should not be set too large. Therefore, the optimal offset value table 531 includes twelve offset value lists (i.e., 1, 2, 3, 4, 5, 6, 8, 9, 10, 12, 15, 16). If the training score corresponding to one of the preset offset values ​​reaches the first preset threshold, the preset offset value D will be updated to the offset value with the highest score (i.e., the optimal offset value), all training scores will be reset, and the training sequence will be restarted. If the number of missing values ​​is equal to the predefined missing threshold, it means that this round of training has failed and training will be restarted. The training time of the optimal offset value prefetch unit 53 can be customized according to actual conditions.

[0088] In the embodiment of the present invention, the stride prefetch unit 52 generates a prefetch request by detecting the stride of a continuous address sequence and adding the detected stride to the last observed access address.

[0089] Please refer to Figure 6 , Figure 6 FIG2 is a schematic diagram of the structure of the stride prefetch unit 52 of the hardware prefetch system 100 provided in an embodiment of the present invention. The stride prefetch unit 52 includes a first reference prediction table 521, a tag scanning unit 522, a stride comparison unit 523, a first prefetch request generation unit 524, and a second page boundary check unit 525.

[0090] The first reference prediction table 521 is used to store the historical access address, multiple stride values, and stride confidences corresponding to the stride values. Each entry in the first reference prediction table 521 is assigned a valid bit to indicate whether the content of each entry is filled. The entries in the first reference prediction table 521 are replaced using a PLRU (Pseudo-LRU) algorithm.

[0091] The tag scanning unit 522 is used to check and update the tags in the first reference prediction table 521. The tag scanning unit 522 scans the first reference prediction table 521 and compares the tags with the current access address. In address (PC)-based mode, the tag is the program counter pointer, while in memory address-based mode, the tag is an address within the 4KB range.

[0092] The stride comparison unit 523 is used to calculate the current stride value based on the current access address (obtained by subtracting the current access address from the last access address), and compare the current stride value with all the stride values ​​in the first reference prediction table 521. If the current stride value matches one of the stride values ​​in the first reference prediction table 521 and the corresponding stride confidence is greater than or equal to a second preset threshold, a matching signal is sent to the first prefetch request generation unit 524.

[0093] The first prefetch request generating unit 524 is used to determine whether the stride value exists in the first reference prediction table 521 based on the received matching signal, and the corresponding stride confidence is greater than or equal to the second preset threshold. If so, a stride prefetch address is generated and the stride prefetch address is sent to the second page boundary checking unit 525.

[0094] The second page boundary checking unit 525 is used to perform a page boundary check on the received stride prefetch address to determine whether the stride prefetch address exceeds the address access boundary: if not, the second page boundary checking unit 525 issues a prefetch request according to the stride prefetch address.

[0095] In the embodiment of the present invention, please refer to Figure 7 , Figure 7 1 is a schematic diagram of the structure of the Markov prefetch unit 54 of the hardware prefetch system 100 provided in an embodiment of the present invention. The Markov prefetch unit 54 includes a difference calculation unit 541, a difference shift register 542, a second reference prediction table 543, a confidence check unit 544, a prediction difference table 545, a second prefetch request generation unit 546, a third page boundary check unit 547, and a difference index scanning unit 548.

[0096] The second reference prediction table 543 is used to store a difference index, multiple predicted differences, and the difference confidence levels corresponding to the predicted differences. Each entry in the second reference prediction table 543 is assigned a valid bit to indicate whether the content of each entry is filled. The replacement method for the second reference prediction table 543 and the second prediction difference table 545 is the PLRU algorithm.

[0097] The difference calculation unit 541 is configured to calculate a current difference according to the current access address, and send the current difference to the difference shift register 542 .

[0098] The difference shift register 542 is used to store the received current difference and generate a current difference index, and send the current difference index and the current difference to the difference index scanning unit 548 and the predicted difference table 545 respectively.

[0099] The difference index scanning unit 548 is configured to check and update the difference index in the second reference prediction table 543 based on the received current difference index. When the difference index scanning unit 548 scans the second reference prediction table 543, it compares the difference index in the second reference prediction table 543 with the current difference index shifted out of the difference shift register 542. If there is no matching difference index, the new difference index is updated to the second reference prediction table 543.

[0100] The confidence checking unit 544 is configured to update the difference confidences in the second reference prediction table 543. If one of the difference confidences is greater than or equal to a third preset threshold, a prediction difference verification signal is issued to the prediction difference table 545. Specifically, if both the current difference index and the predicted difference match, the corresponding difference confidence value is incremented by 1. If the difference confidence value is greater than or equal to the third preset threshold, a prediction difference verification signal is asserted to indicate that the selected prediction difference can be used to generate a prefetch request address.

[0101] The prediction difference table 545 is used to update the prediction difference in the second reference prediction table 543 according to the received current difference, and to read the corresponding prediction difference from the second reference prediction table 543 according to the received prediction difference verification signal, and output it to the second prefetch request generation unit 546.

[0102] Specifically, the prediction difference table 545 performs two different operations for checking: prediction difference checking and prediction difference updating. For the prediction difference checking, if a prediction difference verification signal is received, the corresponding prediction difference is read from the second reference prediction table 543 and output to the second prefetch request generation unit 546 to calculate the prefetch address. For the prediction difference updating, if the current difference (new prediction difference) does not match the prediction difference (old prediction difference) in the second reference prediction table 543, the current difference is written to the second reference prediction table 543, and an entry in the second reference prediction table 543 is replaced using the PLRU algorithm.

[0103] The second prefetch request generating unit 546 is configured to generate a Markov prefetch address according to the current access address and the received prediction difference value, and send the Markov prefetch address to the third page boundary checking unit 547 .

[0104] The third page boundary check unit 547 is used to perform a page boundary check on the received Markov prefetch address to determine whether the Markov prefetch address exceeds the address access boundary. If not, the third page boundary check unit 547 issues a prefetch request according to the Markov prefetch address.

[0105] Specifically, the Markov prefetch unit 54 observes the difference between the current access address and the last missed address and updates the second reference prediction table 543. If the difference index corresponding to the current access address is not recorded in the second reference prediction table 543, it is recorded in the second reference prediction table 543, and the next missed address difference corresponding to the difference index corresponding to the current access address is recorded in the prediction difference table 545. Each address difference index is assigned multiple entries in the prediction difference table 545. Once the address difference recorded in the prediction difference table 545 appears again, followed by the index address difference, the corresponding difference confidence value is increased by 1. If the difference confidence value is greater than or equal to a third preset threshold, a prediction difference verification signal is asserted and sent to the second prefetch request generation unit 546 to issue a prefetch request.

[0106] In the embodiment of the present invention, the prefetch management unit 51 can dynamically manage the operations of the stride prefetch unit 52, the Markov prefetch unit 54 and the optimal offset value prefetch unit 53 in different time slots according to the access pattern or historical prefetch accuracy results.

[0107] Please refer to Figure 8 , Figure 81 is a schematic diagram of the structure of the prefetch management unit 51 of the hardware prefetch system 100 provided in an embodiment of the present invention. The prefetch management unit 51 includes a stride prefetch counter 511, a Markov prefetch counter 512, an optimal offset value prefetch counter 513, and a prefetcher state unit 514;

[0108] The stride prefetch counter 511 is used to record the confidence of the stride prefetch unit 52 in the training mode;

[0109] The Markov prefetch counter 512 is used to record the confidence of the Markov prefetch unit 54 in the training mode;

[0110] The optimal offset value pre-fetch counter 513 is used to record the confidence of the optimal offset value pre-fetch unit 53 in the training mode;

[0111] The prefetcher state unit 514 is configured to adjust the operating states of the stride prefetch unit 52, the Markov prefetch unit 54, and the optimal offset prefetch unit 53, respectively, based on the count values ​​of the stride prefetch counter 511, the Markov prefetch counter 512, and the optimal offset prefetch counter 513. The operating states include an initial state, a standby state, and a selection state. The initial state and the standby state each represent the states of the three prefetch units (the stride prefetch unit 52, the Markov prefetch unit 54, and the optimal offset prefetch unit 53) during the training process. The selection state represents the state in which a prefetch unit begins issuing a prefetch request.

[0112] In the embodiment of the present invention, the prefetch management unit 51 can select a corresponding prefetch unit according to different address modes. After selection, only the prefetch unit will be activated, thereby reducing power consumption and improving prefetch accuracy.

[0113] Please refer to Figure 9 , Figure 9 This is a logical diagram of the prefetcher state unit 514 of the hardware prefetch system 100 provided by an embodiment of the present invention. The prefetcher state unit 514 defines, in ascending order, an initial line, a confidence line, a near-full line, and a full line. The prefetcher state unit 514 adjusts the operating states of the stride prefetch unit 52, the Markov prefetch unit 54, and the optimal offset value prefetch unit 53 based on the count values ​​of the three prefetch units within the intervals of the initial line, the confidence line, the near-full line, and the full line. Each time a line is crossed, a certain damping value is applied to prevent the counter from frequently switching between different states. The interval is in the initial state when it is within the initial line, the interval is in the preparatory state when it is between the initial line and the near-full line, and the interval is in the selection state when it is between the near-full line and the full line.

[0114] For example, when the three prefetch units are reset, the counters corresponding to all prefetch units are at the initial line, and the next line prefetching starts by default, and the other three prefetch modules 5 enter the pattern recognition state.

[0115] When the count value corresponding to any one of the stride prefetch counter 511, the Markov prefetch counter 512 and the optimal offset value prefetch counter 513 is greater than or equal to the preset count threshold (about to be full), the corresponding stride prefetch unit 52 or the Markov prefetch unit 54 or the optimal offset value prefetch unit 53 is enabled to issue a prefetch request, and the training of the other units stops.

[0116] When the count values ​​corresponding to at least two of the stride prefetch counter 511, the Markov prefetch counter 512, and the optimal offset value prefetch counter 513 are simultaneously greater than or equal to the preset count threshold (about to be full), the selection priority of the prefetch management unit 51 is, from first to last, the stride prefetch unit 52, the Markov prefetch unit 54, and the optimal offset value prefetch unit 53.

[0117] In the embodiment of the present invention, the prefetch module 5 further includes an output buffer unit 55 ; the output buffer unit 55 is used to temporarily store the prefetch request issued by the prefetch module 5 .

[0118] For details, please refer to Figure 10 , Figure 10 The figure is a schematic diagram of the structure of the output buffer unit 55 of the hardware prefetch system 100 provided in an embodiment of the present invention. Each prefetch unit is connected to the output buffer unit 55. Generated prefetch requests are first temporarily stored in the output buffer unit 55. The output buffer unit 55 is provided with a counter. The counter is initially set to 0 and increments by 1 with each clock cycle if the output buffer unit 55 is not empty. If data is read from the output buffer unit 55, the counter is reset to 0. If the counter value exceeds a threshold (no data is sent to the downstream sub-module for a period of time), all outstanding prefetch requests in the output buffer unit 55 are flushed and discarded.

[0119] However, in some cases, the downstream module may not be able to receive prefetch requests for a period of time, and the output buffer unit 55 may eventually become full. If the upstream module continuously generates cache miss requests, the output buffer unit 55 may overflow. To handle this problem, once the output buffer unit 55 is full and a new prefetch request is generated, the oldest prefetch request in the output buffer unit 55 is popped and deleted, and spare space is reversed for the incoming new prefetch request.

[0120] Compared to existing technologies, the prefetch module proposed in this invention is used for prefetching at all levels of cache. For the second and third level caches, if there is only a single address sequence, a stride prefetch unit and a Markov prefetch unit can be selected based on a fixed address pattern. If multiple address sequences exist, an optimal offset prefetch unit can be used to find the optimal offset value for prefetching. The prefetcher management module can select different prefetch units based on the processor's current access address, thereby improving prefetch accuracy. Furthermore, the output buffer unit 55 can cancel prefetch requests when they time out or when the prefetch output queue overflows, thereby reducing bandwidth resources.

[0121] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0122] Through the above description of the embodiments, those skilled in the art will clearly understand that the methods of the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is the more preferred implementation method. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal (such as a mobile phone, computer, server, air conditioner, or network device) to execute the methods described in the various embodiments of the present invention.

[0123] The embodiments of the present invention are described above in conjunction with the accompanying drawings. What is disclosed is only a preferred embodiment of the present invention. However, the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms and equivalent changes without departing from the scope of protection of the purpose of the present invention and the claims, which are all within the protection of the present invention.

Claims

1. A hardware prefetch system, characterized in that: The hardware prefetch system includes a processor, a first-level cache, a second-level cache, a third-level cache, a prefetch module and a memory; The processor is used to read and execute instructions in the first-level cache, the second-level cache, the third-level cache and the memory; The first-level cache is used to cache instructions; The secondary cache is used to cache instructions and serves as an object for the processor to read instructions when the processor reads the primary cache and misses; The third-level cache is used to cache instructions, and serves as the object of the processor to read instructions when the processor reads instructions from both the first-level cache and the second-level cache and misses; The memory is used to store instructions and serves as an object for the processor to read instructions when the processor reads instructions from the first-level cache, the second-level cache, and the third-level cache and none of them hits; The prefetch module is used to issue prefetch requests to the secondary cache, the third-level cache and the memory respectively; The prefetch module includes a prefetch management unit, a stride prefetch unit, an optimal offset value prefetch unit, and a Markov prefetch unit; The prefetch management unit is used to dynamically select one of the stride prefetch unit, the optimal offset value prefetch unit and the Markov prefetch unit, and issue a prefetch request to the secondary cache, the third level cache and the memory through the selected unit; The stride prefetch unit is used to issue a prefetch request to the L2 cache, the L3 cache and the memory according to the stride between the current access address of the processor and the historical access address of the processor; The optimal offset value prefetch unit is used to issue a prefetch request to the secondary cache, the third level cache and the memory according to the optimal offset value between the current access address and the historical access address; The Markov prefetch unit is used for issuing a prefetch request to the L2 cache, the L3 cache and the memory according to the difference between the current access address and the historical access address.

2. The hardware prefetching system according to claim 1, wherein: The optimal offset value pre-fetching unit includes an optimal offset value table, a recent request table, and a first page boundary checking unit; The optimal offset value table is used to store a plurality of preset offset values, and select the optimal offset value from the preset offset values ​​according to the current access address and a preset rule and send the optimal offset value to the first page boundary checking unit; The recent request table is used to record the historical access addresses; The first page boundary check unit is used to calculate an offset value prefetch address based on the received optimal offset value and the current access address, and perform a page boundary check on the offset value prefetch address to determine whether the offset value prefetch address exceeds the address access boundary: if not, the first page boundary check unit issues a prefetch request based on the offset value prefetch address.

3. The hardware prefetching system according to claim 2, wherein: The preset rules are: When the processor reads a miss: The optimal offset value table performs parallel training on each of the preset offset values ​​according to the current access address based on an offset value training algorithm to obtain a training result corresponding to the preset offset value; The recent request table matches each of the training results with the historical access address to obtain a training score corresponding to the training result; When a training score corresponding to one of the training results is equal to a first preset threshold, the preset offset value corresponding to the training result is used as the optimal offset value.

4. The hardware prefetching system according to claim 1, wherein: The stride prefetch unit includes a first reference prediction table, a tag scanning unit, a stride comparison unit, a first prefetch request generation unit, and a second page boundary checking unit; The first reference prediction table is used to store the historical access address, multiple stride values, and stride confidences corresponding to the stride values; The label scanning unit is used to check and update the labels of the first reference prediction table; The stride comparison unit is configured to calculate a current stride value according to the current access address, and compare the current stride value with all the stride values ​​in the first reference prediction table, and send a match signal to the first prefetch request generation unit if the current stride value matches one of the stride values ​​in the first reference prediction table and the corresponding stride confidence is greater than or equal to a second preset threshold; The first prefetch request generating unit is configured to determine, based on the received matching signal, whether there is a stride value in the first reference prediction table whose corresponding stride confidence is greater than or equal to the second preset threshold; if so, generate a stride prefetch address, and send the stride prefetch address to the second page boundary checking unit; The second page boundary check unit is used to perform a page boundary check on the received stride prefetch address to determine whether the stride prefetch address exceeds the address access boundary: if not, the second page boundary check unit issues a prefetch request according to the stride prefetch address.

5. The hardware prefetching system according to claim 1, wherein: The Markov prefetch unit includes a difference calculation unit, a difference shift register, a difference index scanning unit, a second reference prediction table, a confidence check unit, a prediction difference table, a second prefetch request generation unit, and a third page boundary check unit; The second reference prediction table is used to store a difference index, a plurality of predicted differences, and difference confidences corresponding to the predicted differences; The difference calculation unit is used to calculate a current difference according to the current access address, and send the current difference to the difference shift register; The difference shift register is used to store the received current difference value and generate a current difference value index, and send the current difference value index and the current difference value to the difference index scanning unit and the predicted difference value table respectively; The difference index scanning unit is configured to check and update the difference index of the second reference prediction table according to the received current difference index; The confidence checking unit is configured to update the difference confidences in the second reference prediction table, and send a prediction difference verification signal to the prediction difference table if one of the difference confidences is greater than or equal to a third preset threshold; The prediction difference table is used to update the prediction difference value in the second reference prediction table according to the received current difference value, and read the corresponding prediction difference value from the second reference prediction table according to the received prediction difference value verification signal, and output it to the second prefetch request generating unit; The second prefetch request generating unit is configured to generate a Markov prefetch address according to the current access address and the received prediction difference value, and send the Markov prefetch address to the third page boundary checking unit; The third page boundary check unit is used to perform a page boundary check on the received Markov prefetch address to determine whether the Markov prefetch address exceeds the address access boundary: if not, the third page boundary check unit issues a prefetch request according to the Markov prefetch address.

6. The hardware prefetching system according to claim 1, wherein: The prefetch management unit includes a stride prefetch counter, a Markov prefetch counter, an optimal offset value prefetch counter and a prefetcher status unit; The stride prefetch counter is used to record the confidence of the stride prefetch unit in the training mode; The Markov prefetch counter is used to record the confidence of the Markov prefetch unit in the training mode; The optimal offset value pre-fetch counter is used to record the confidence of the optimal offset value pre-fetch unit in the training mode; The prefetcher status unit is used to adjust the working states of the stride prefetch unit, the Markov prefetch unit and the optimal offset value prefetch unit according to the count values ​​of the stride prefetch counter, the Markov prefetch counter and the optimal offset value prefetch counter respectively.

7. The hardware prefetching system according to claim 6, wherein: When the count value corresponding to any one of the stride prefetch counter, the Markov prefetch counter and the optimal offset value prefetch counter is greater than or equal to the preset count threshold, the corresponding stride prefetch unit or the Markov prefetch unit or the optimal offset value prefetch unit is enabled to issue a prefetch request, and the other units stop training.

8. The hardware prefetching system according to claim 6, wherein: When the count values ​​corresponding to at least two of the stride prefetch counter, the Markov prefetch counter and the optimal offset value prefetch counter are simultaneously greater than or equal to the preset count threshold, the selection priority of the prefetch management unit is from first to last: the stride prefetch unit, the Markov prefetch unit, and the optimal offset value prefetch unit.

9. The hardware prefetching system according to claim 1, wherein: The prefetch module further includes an output buffer unit; the output buffer unit is used to temporarily store the prefetch request issued by the prefetch module.

Images