A cache replacement system and method based on instruction stream and memory access mode learning

Abstract

The present invention provides a cache replacement system based on instruction stream and memory access mode learning, including a branch prediction module, which predicts the instruction stream and writes it into an instruction fetch address queue; a memory access instruction recording module, which sequentially records the information of the submitted memory access instructions and writes it into its memory access instruction buffer; and simultaneously obtains the memory access instruction sequence through query; a memory access mode learning module, which records the memory access instruction sequence in its memory access history buffer, learns the memory access mode accordingly, and then predicts the memory access physical address of the memory access instruction and writes it into the memory access address queue; a cache replacement decision module, which receives the physical address of the cache replacement candidate and uses it to retrieve the instruction fetch address queue or the memory access address queue, and selects a cache replacement candidate as a kick-out item according to the obtained reuse distance and feeds it back to the first-level cache. The present invention also provides a corresponding method. The cache replacement system of the present invention can avoid the interference of out-of-order execution and cache prefetch, and improve the accuracy of memory access sequence prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of computer architecture, and in particular to a cache replacement system and method based on instruction stream and memory access mode learning. Background Art

[0002] Cache is an important mechanism in modern processors. It copies frequently used data from memory to cache. Subsequent data access can be directly read from cache, thereby reducing the number of accesses to slow DRAM memory and improving processor performance. The capacity of cache is limited. In actual use, replacement of cache content is inevitable. The performance of cache is greatly affected by the cache replacement strategy.

[0003] Cache access behavior is complex. The memory access sequence received by the cache system in the processor will be interfered by the processor's out-of-order execution and cache prefetch mechanism, which further increases the difficulty of predicting memory access behavior in the cache system.

[0004] Specifically, the traditional cache replacement method learns the memory access behavior pattern based on the memory access sequence received by the cache system. However, the memory access sequence received in the cache system is the result of a combination of several memory access instructions intertwined together. The execution order of different instructions and the behavior pattern of each instruction will affect the memory access sequence finally received by the cache system. It is difficult to learn the memory access behavior pattern based on such a complex and changeable memory access sequence, and it faces great uncertainty.

[0005] Existing cache replacement methods cannot effectively reduce these uncertainties. Summary of the invention

[0006] The present invention provides a cache replacement system and method based on instruction stream and memory access pattern learning, so as to reduce the learning difficulty of memory access behavior pattern and improve the performance of cache replacement strategy.

[0007] In order to achieve the above-mentioned object, the present invention provides a cache replacement system based on instruction stream and memory access mode learning, including a branch prediction module, an instruction fetch address queue, a memory access instruction recording module, a memory access mode learning module, a memory access address queue and a cache replacement decision module;

[0008] The branch prediction module is configured to use an advance prediction technique to predict the instruction stream of the target program and write the prediction result into the instruction fetch address queue;

[0009] The memory access instruction recording module is configured to sequentially record information of submitted memory access instructions and write the information of the memory access instructions into its memory access instruction buffer; at the same time, whenever a new entry is written into the instruction fetch address queue, the memory access instruction buffer is queried with the starting address of the instruction block corresponding to the entry, so as to obtain a memory access instruction sequence through query attempts and output it to the memory access instruction learning module and the memory access address queue;

[0010] The memory access mode learning module is configured to record the memory access instruction sequence in its memory access history buffer, learn the memory access mode of the memory access instruction according to the historical information stored in the memory access history buffer, and predict the memory access physical address of each memory access instruction in the memory access instruction sequence according to the learned memory access mode and write it into the memory access address queue;

[0011] The cache replacement decision module is configured to receive the physical addresses of all cache replacement candidates issued by the first-level cache, use these physical addresses to retrieve the instruction fetch address queue or the memory access address queue, obtain the reuse distance of each cache replacement candidate, and select a cache replacement candidate as a kick-out item based on the reuse distance to feed back to the first-level cache.

[0012] The branch prediction module is configured to write the prediction result into the instruction fetch address queue with instruction block as the granularity; the branch prediction module is configured to execute:

[0013] A1: In each cycle, the instruction block where the current prediction address is located is taken as the current instruction block, and the current prediction address is taken as the prediction start address of the current instruction block; a fixed prediction window starting from the current instruction block is determined according to the prediction start address of the current instruction block;

[0014] A2: After obtaining the predicted starting address of the current instruction block, search and predict the branch instruction in the current instruction block to determine whether the current instruction block hits the branch instruction to jump;

[0015] A3: According to the judgment result, if the current instruction block does not hit the branch instruction to jump, it means that there is no branch instruction in the current instruction block or none of the identified branch instructions jump, then the information of the current instruction block is written into the instruction fetch address queue as the prediction result of the branch prediction module; then, the predicted starting address of the next instruction block is determined, and the next instruction block is used as the new current instruction block, and the process returns to step A2 until the current instruction block is the last instruction block in the fixed prediction window, at which time the current prediction address is incremented according to the number of instruction blocks in the fixed prediction window to enter the next cycle;

[0016] Otherwise, the first jump branch instruction with the smallest linear address is selected from the hit jump branch instructions as the end address of the current instruction block, the information of the current instruction block is written into the instruction fetch address queue as the prediction result, and the current predicted address is updated to the jump address of the first jump branch instruction to enter the next cycle.

[0017] Alternatively, consistent with the above, the branch prediction module is configured to write the prediction result into the instruction fetch address queue with instruction blocks as the granularity, except that the branch prediction module is configured to perform the following steps:

[0018] A1': In each cycle, the instruction block where the current prediction address is located is taken as the current instruction block, and the current prediction address is taken as the prediction start address of the current instruction block; then, the prediction start addresses of all instruction blocks in the fixed prediction window starting from the current instruction block are determined according to the prediction start address of the current instruction block;

[0019] A2': According to the predicted starting address of each instruction block, the branch instruction is searched and predicted in each instruction block to determine whether each instruction block hits the branch instruction of the jump;

[0020] A3': According to the judgment result, if all instruction blocks do not hit the branch instruction of the jump, the information of all instruction blocks is written into the instruction fetch address queue in sequence as the prediction result of the branch prediction module; the current prediction address is incremented according to the number of instruction blocks in the fixed prediction window to enter the next cycle; otherwise, if there is at least one instruction block that hits at least one branch instruction of the jump, the first branch instruction of the jump with the smallest linear address is selected from the hit branch instructions as the end address of the current instruction block, and the information of the current instruction block and all the instruction blocks before it are written into the instruction fetch address queue as the prediction result, and the current prediction address is updated to the jump address of the first branch instruction of the jump to enter the next cycle.

[0021] The information of the instruction block includes the row linear address, row physical address, starting offset, ending offset and jump bit of the instruction block; when writing the information of the instruction block into the instruction fetch address queue, the predicted starting address of the instruction block is split into a high-order row linear address and a low-order starting offset, and the row physical address is obtained by querying the TLB through the row linear address, and the obtained row linear address, row physical address and starting offset are written into the corresponding fields in the entry of the instruction fetch address queue; in addition, if the instruction block hits a jump branch instruction, the end byte offset of the first jump branch instruction is written into the instruction fetch address queue as the ending offset; otherwise, the ending offset in the instruction fetch address queue is set to 63.

[0022] The structure of each entry in the instruction fetch address queue is:

[0023] <valid,line_addr,phys_addr,begin_offset,end_offset,taken>,

[0024] Among them, valid represents the valid bit; line_addr represents the line linear address; phys_addr represents the line physical address; begin_offset represents the starting offset; end_offset represents the ending offset; taken represents the jump bit;

[0025] The instruction fetch address queue is also provided with a commit pointer, a read pointer and a write pointer; the commit pointer of the instruction fetch address queue points to the entry item of the instruction block where the next instruction to be committed is located in the instruction fetch address queue; the read pointer of the instruction fetch address queue points to the entry item of the instruction block where the next instruction to be read is located in the instruction fetch address queue; the write pointer of the instruction fetch address queue points to the position where the branch prediction module will write next; after the mainstream pipeline of the processor is refreshed, the read pointer and the write pointer of the instruction fetch address queue are rolled back to the position of the branch refresh or the position of the commit pointer according to the refresh type;

[0026] The structure of each entry in the memory access instruction buffer is:

[0027] <LineAddr,PhyAddr,InstType,Memlen> ,

[0028] Wherein, LineAddr indicates the instruction linear address of the memory access instruction; PhyAddr indicates the memory access physical address of the memory access instruction at the last execution; InstType indicates the type of the memory access instruction, InstType∈{DirectInst,InDirectInst}, where DirectInst indicates a direct memory access instruction and InDirectInst indicates an indirect memory access instruction; Memlen indicates the memory access length of the memory access instruction;

[0029] The memory access history buffer is an array indexed by the PC of the memory access instruction, and each entry in the memory access history buffer records the physical addresses of the past 12 memory accesses of the same memory access instruction;

[0030] And the structure of each entry in the memory access address queue is:

[0031] <valid,inst_line_addr,mem_phys_addr,memlen,inst_queue_index> ,

[0032] Among them, valid represents the valid bit; inst_line_addr represents the instruction linear address; mem_phys_addr represents the memory access physical address; memlen represents the memory access length; inst_queue_index represents the instruction fetch address queue index;

[0033] The memory access address queue has a commit pointer, a read pointer and a write pointer; when a branch refresh occurs in the main pipeline, the read pointer and the write pointer of the memory access address queue are rolled back to the position of the branch refresh.

[0034] The branch prediction module is configured to: suspend the prediction process if the instruction fetch address queue is full; whether the instruction fetch address queue is full is determined based on a combination of a write pointer and a commit pointer of the instruction fetch address queue.

[0035] The memory access pattern learning module includes a step length predictor and a time association predictor, so that the memory access pattern learning module can learn and predict the step length pattern and the time association pattern through the step length predictor and the time association predictor based on the memory access history buffer.

[0036] Each entry of the step length predictor is used to record a step length mode, and the step length mode refers to a step length memory access sequence; each entry of the step length predictor includes a tag of a step length memory access sequence, a previous address, a previous step length, a trusted counter, a current mode, a first address, a maximum counter and a direction;

[0037] Each entry of the time correlation predictor is used to record a time correlation pattern, which refers to a time correlation memory access sequence. Each entry of the time correlation predictor includes the time correlation pattern, the time correlation sequence length and the time correlation sequence.

[0038] The cache replacement decision module is configured to: when selecting a cache replacement candidate as a kick-out item based on the reuse distance, if the cache replacement candidate with the smallest reuse distance among multiple cache replacement candidates is unique, then the cache replacement candidate with the largest reuse distance is fed back to the first-level cache as the kick-out item; otherwise, the item with the smallest path number sent by the first-level cache among the cache replacement candidates with the smallest reuse distance is selected and kicked out from the first-level cache.

[0039] The cache replacement decision module is also configured to: before retrieving the instruction fetch address queue or the memory access address queue, determine whether the valid prediction results in the instruction fetch address queue or the memory access address queue are less than the number threshold of valid prediction results; if so, the replacement decision module feeds back corresponding information to the first-level cache, so that the first-level cache uses the default replacement strategy for candidate selection, and the process ends at this time.

[0040] On the other hand, the present invention provides a cache replacement method based on instruction stream and memory access mode learning, comprising:

[0041] S1: using a branch prediction module and an advance prediction technique to predict the instruction stream of the target program and write the prediction result into an instruction fetch address queue;

[0042] S2: using a memory access instruction recording module to sequentially record information of submitted memory access instructions, and write the information of these memory access instructions into its memory access instruction buffer; whenever a new entry is written into the instruction fetch address queue, query the memory access instruction buffer with the starting address of the instruction block corresponding to the entry, so as to obtain the memory access instruction sequence through querying and output it to the memory access instruction learning module and the memory access address queue;

[0043] S3: using a memory access instruction learning module to record a memory access instruction sequence in a memory access history buffer, learning a memory access mode of the memory access instruction according to historical information stored in the memory access history buffer, and predicting a memory access physical address of each memory access instruction in the memory access instruction sequence according to the learned memory access mode, and writing the memory access physical address of each memory access instruction into a memory access address queue;

[0044] S4: Utilize a cache replacement decision module to receive the physical addresses of all cache replacement candidates issued by the first-level cache, use these physical addresses to retrieve the instruction fetch address queue or the memory access address queue, obtain the reuse distance of each cache replacement candidate, and select a cache replacement candidate as a kick-out item based on the reuse distance to feed back to the first-level cache.

[0045] The cache replacement system based on instruction stream and memory access pattern learning proposed by the present invention predicts the future memory access sequence based on branch prediction technology. The object of memory access pattern learning is specific to each memory access instruction rather than a combination of several memory access instructions, thereby reducing the learning difficulty of memory access behavior patterns and reducing the complexity of prediction; the prediction process is based on the program instruction stream obtained by branch prediction technology, and the predicted memory access sequence only contains real program requirements and is not affected by the cache prefetch mechanism. In addition, the system of the present invention records the information of the submitted memory access instructions in sequence, so that the memory access pattern learning is performed through the submitted sequence of memory access instructions rather than the memory access sequence received in the cache system, avoiding the interference of out-of-order execution.

[0046] In summary, the method of the present invention learns the memory access mode at the instruction level, is not interfered by out-of-order execution and cache prefetching, and reduces the uncertainty faced in the memory access sequence prediction process. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1It is a structural schematic diagram of one of the processor micro-architectures to which the cache replacement system and method based on instruction stream and memory access pattern learning of the present invention are applicable.

[0048] Figure 2 It is a structural schematic diagram of another processor micro-architecture to which the cache replacement system and method based on instruction stream and memory access mode learning of the present invention are applicable.

[0049] Figure 3 It is an overall framework diagram of a cache replacement system based on instruction stream and memory access pattern learning according to an embodiment of the present invention.

[0050] Figure 4 Yes Figure 3 The prediction flow chart of the branch prediction module of the cache replacement system based on instruction stream and memory access pattern learning is shown.

[0051] Figure 5 The present invention is a schematic diagram of the positional relationship of an instruction fetch address queue and a commit pointer, a read pointer and a write pointer of a cache replacement system based on instruction stream and memory access mode learning.

[0052] Figure 6 It is a schematic diagram of the query process of the memory access instruction buffer of the cache replacement system based on instruction stream and memory access mode learning of the present invention.

[0053] FIG7( a ) is a schematic diagram of a conventional time-correlated prefetching algorithm.

[0054] FIG. 7( b ) is a schematic diagram of the memory access pattern learning process based on memory access history buffering in the cache replacement system based on instruction stream and memory access pattern learning of the present invention.

[0055] Figure 8 It is a structural schematic diagram of entry items of a step size predictor of a cache replacement system based on instruction stream and memory access pattern learning of the present invention.

[0056] Fig. 9 It is a principle diagram of the refresh process of the instruction fetch address queue and the memory access address queue of the cache replacement system based on instruction stream and memory access mode learning of the present invention when a branch refresh occurs in the main pipeline.

[0057] Fig.10 This is a comparison chart of the performance improvement of the cache replacement method based on instruction stream and memory access pattern learning proposed by the present invention compared with various existing algorithms. DETAILED DESCRIPTION

[0058] The present invention is further described in detail below through specific implementation examples and drawings.

[0059] The cache replacement system and method based on instruction stream and memory access mode learning of the present invention are applicable to Figure 1The processor microarchitecture shown in FIG. The processor microarchitecture includes at least five stages: instruction fetch, decoding, execution, memory access and write back. The five stages correspond to the instruction fetch unit 100, the decoding unit 200, the execution unit 300, the memory access unit 400 and the write back unit 500, respectively. The present invention is also applicable to a processor including Figure 1 In a more complex processor microarchitecture with the functions shown, a certain stage may be refined in a more complex processor microarchitecture. For example, the execution stage can be split into three sub-stages: renaming, scheduling, and execution.

[0060] like Figure 2 As shown, according to the advance prediction technology in the prior art, the processor microarchitecture may also include a branch prediction unit 600, and the branch prediction unit 600 may interact with the instruction fetch unit through a prediction queue. Specifically, the branch prediction unit 600 predicts the future program instruction stream and writes the prediction result into the prediction queue 700 with the aligned instruction block as the granularity. Each entry in the prediction queue 700 contains the starting address, the ending address and other branch instruction information of the instruction block. The relationship between the branch prediction unit 600 and the instruction fetch unit 100 is similar to that of a producer-consumer. The instruction fetch unit 100 reads an entry from the prediction queue 700 per cycle, and reads the corresponding instruction block data from the cache system according to the read entry, and then sends the instruction block data to the subsequent units of the pipeline (i.e., the decoding unit 200, the execution unit 300, the memory access unit 400 and the write-back unit 500) to execute the instructions contained in the instruction block. When the cache replacement method based on instruction stream and memory access pattern learning of the present invention is applied to a processor microarchitecture including a branch prediction unit 600, the branch prediction unit 600 and the prediction queue 700 can be used as the branch prediction module and instruction fetch address queue of the system of the present invention, respectively, thereby reducing the workload of specific implementation.

[0061] Figure 3 FIG. 1 is an overall framework diagram of a cache replacement system based on instruction stream and memory access pattern learning according to an embodiment of the present invention. Figure 3As shown, the cache replacement system based on instruction stream and memory access mode learning includes six modules, namely, a branch prediction module 10, an instruction fetch address queue 20, a memory access instruction recording module 30, a memory access mode learning module 40, a memory access address queue 50, and a cache replacement decision module 60, based on the existing processor micro-architecture (including an instruction fetch unit 100, a decoding unit 200, an execution unit 300, a memory access unit 400, and a write-back unit 500). These six modules together constitute a prediction sub-pipeline with an execution rate faster than the main pipeline, and run independently outside the main pipeline of the processor (i.e., the sequential execution of the instruction fetch unit 100, the decoding unit 200, the execution unit 300, the memory access unit 400, and the write-back unit 500). The execution order of each module in the prediction sub-pipeline is branch prediction module 10→instruction fetch address queue 20→memory access instruction recording module 30→memory access mode learning module 40→memory access address queue 50. The cache replacement decision module 60 is a relatively independent module, which is configured to receive and respond to the replacement request of 11i / l1d in the main pipeline. The relationship between the cache replacement decision module 60 and the other five modules of the prediction sub-pipeline is similar to that of a producer-consumer. The instruction fetch address queue 20 and the memory access address queue 50 are filled by the prediction sub-pipeline, and the cache replacement decision module 60 retrieves these two queues.

[0062] As described above, in some embodiments, the branch prediction module 10 and the instruction fetch address queue 20 may be the branch prediction unit 600 and the prediction queue 700 in the existing processor microarchitecture. In other embodiments, the branch prediction module 10 and the instruction fetch address queue 20 are newly added modules.

[0063] (I) Branch prediction module 10

[0064] The branch prediction module 10 is the starting point of the prediction sub-pipeline. The branch prediction module 10 is configured to: use the advance prediction technology to predict the instruction stream of the target program and write the prediction results into the instruction fetch address queue 20 with instruction blocks as the granularity (i.e., in the form of aligned instruction blocks); if the instruction fetch address queue 20 is full, the prediction process is suspended. The entries in the instruction fetch address queue 20 obtained are used as the future access sequence of the first-level instruction cache in the instruction fetch unit 100, that is, the predicted result of the access sequence of the instruction stream of the target program.

[0065] The branch prediction module 10 predicts the instruction flow of the target program by judging whether there is a branch instruction to jump in all instruction blocks of the fixed prediction window in each cycle, and predicts the instruction flow of the target program according to the judgment result. The prediction process in each cycle is performed based on the current prediction address, and the current prediction address is updated to enter the next cycle. A very important function of the branch prediction module 10 is to maintain the current prediction address, which is a 48-bit linear address.

[0066] Maintaining the current predicted address includes: if a branch instruction of a jump is predicted, the jump address of the branch instruction of the jump will be updated to the current predicted address; that is, the branch prediction module 10 is a self-looping branch prediction unit, and in addition to writing the prediction result of the branch prediction module 10 into the instruction fetch address queue 20, the jump address of the branch instruction of the jump in the prediction result will be returned to the branch prediction module 10 itself as the current predicted address, as the starting address of the next prediction cycle. In addition, during the initialization phase of the processor or when the main pipeline is refreshed, the instruction fetch unit will send the initialization address or the refresh address to the branch prediction module 10 to update the current predicted address.

[0067] Figure 4 The prediction process of the branch prediction module 10 is shown. The fixed prediction window of the branch prediction module is multiple instruction blocks, that is, multiple consecutive instruction blocks can be predicted per cycle, and the bit width of the instruction block is equal to the instruction fetch bit width of the instruction fetch unit 100, which is convenient for interaction with the instruction fetch unit 100.

[0068] The fixed prediction window of the branch prediction module is a plurality of 64B instruction blocks, that is, a plurality of consecutive 64B instruction blocks can be predicted per cycle, and the selection of 64B instruction blocks is used to match the bandwidth of the instruction fetch unit 100 .

[0069] like Figure 4 As shown, the branch prediction module 10 is configured to perform the following steps:

[0070] Step A1: In each cycle (i.e., when the current predicted address is initialized or updated), the instruction block where the current predicted address is located is taken as the current instruction block, and the current predicted address is taken as the predicted starting address of the current instruction block; a fixed prediction window starting from the current instruction block is determined according to the predicted starting address of the current instruction block;

[0071] Since the prediction of the instruction stream of the target program is based on the granularity of aligned instruction blocks, the fixed prediction window of each cycle is multiple instruction blocks, that is, multiple instruction blocks can be predicted per cycle, such as a 32B fixed prediction window can be split into two 16B instruction blocks for parallel prediction. In this embodiment, assuming that the current prediction address is 5, the fixed prediction window of each cycle of the branch prediction module 10 is 128B, that is, two aligned 64B instruction blocks are predicted per cycle. The predicted starting address of the first 64B instruction block is the current prediction address (that is, 5), and the instruction block range is [5,63]; the predicted starting address of the second 64B instruction block is 64, and the instruction block range is [64,127]. Since it is an aligned instruction block, the default starting address of each instruction block is an integer multiple of 64B. However, considering that the starting address of the first instruction block in the prediction process may not be an integer multiple of 64B, because this address may be jumped from a branch instruction, there is no way to ensure that this address is strictly aligned; therefore, the starting address of the second instruction block (and the third, fourth, etc.) in the prediction process must be an integer multiple of 64B.

[0072] In this embodiment, Figure 4 The multiple instruction blocks predicted by the branch prediction module in each cycle are predicted one by one in serial order, that is, each time the predicted starting address of the next instruction block is calculated based on the predicted starting address of the current instruction block.

[0073] Step A2: after obtaining the predicted starting address of the current instruction block, searching and predicting the branch instruction in the current instruction block to determine whether the current instruction block hits the branch instruction for jumping;

[0074] Among them, the branch prediction module 10 needs to use an existing branch prediction algorithm to realize the prediction of branch instructions. The present invention does not rely on a specific branch prediction algorithm, and the existing branch prediction algorithms are applicable to the branch prediction module. Taking the common TAGE branch prediction algorithm as an example, first use the predicted starting address of the current instruction block to retrieve the BTB (branchtarget buffer), and identify all branch instructions contained in the current instruction block according to the branch instruction location stored in the BTB. This identification process may hit multiple branch instructions. Then predict the jump direction and jump address of each identified branch instruction to determine whether there is a branch instruction for jumping among these branch instructions.

[0075] Step A3: According to the judgment result, if the current instruction block does not hit the branch instruction for jumping, it means that there is no branch instruction in the current instruction block or none of the identified branch instructions jump, then the information of the current instruction block is written into the instruction fetch address queue as the prediction result of the branch prediction module 10; then, the predicted starting address of the next instruction block is determined, and the next instruction block is used as the new current instruction block, and the process returns to step A2 until the current instruction block is the last instruction block in the fixed prediction window. At this time, the current prediction address is incremented according to the number of instruction blocks in the fixed prediction window, thereby updating the current prediction address and entering the next cycle;

[0076] Otherwise, that is, the current instruction block hits the branch instruction that jumps, which means that the branch instruction identified in the instruction block has one or more predicted jumps, then the first jump branch instruction with the smallest linear address is selected from the hit jump branch instructions as the end address of the current instruction block, and the information of the current instruction block is written into the instruction fetch address queue as the prediction result, and the current predicted address is updated to the jump address of the first jump branch instruction to enter the next cycle.

[0077] The information of the instruction block includes the row linear address, row physical address, start offset, end offset, jump bit, etc. of the instruction block.

[0078] The addresses used by the branch prediction module 10 in the prediction process of each cycle (such as the predicted starting address of the current instruction block, etc.) are all in the form of linear addresses. However, the addresses stored in the instruction fetch address queue 20 are in the form of physical addresses. This is because the addresses used in the cache system are all physical addresses, and the cache replacement decision module 60 will use the physical address of the candidate to retrieve the instruction fetch address queue and the memory access address queue. Therefore, the addresses stored in the instruction fetch address queue 20 need to be physical addresses corresponding to the linear addresses.

[0079] Therefore, the row linear address and starting offset of the current instruction block can be determined based on the predicted starting address of the current instruction block. When the row linear address in the information of the current instruction block is written into the instruction fetch address queue, it is first necessary to search through the TLB (translation lookaside buffer) to obtain the row physical address, and both the row linear address and the row physical address are saved in the instruction fetch address queue 20.

[0080] Specifically, when writing the information of the instruction block into the instruction fetch address queue 20, it is necessary to split the predicted starting address of the 48-bit instruction block into a high 42-bit row linear address and a low 6-bit starting offset, and query the TLB through the 42-bit row linear address to obtain a 34-bit row physical address, and write the obtained row linear address, row physical address and starting offset into the corresponding fields in the entry item of the instruction fetch address queue 20. In addition, if the instruction block hits a branch instruction of a jump, the end byte offset of the first branch instruction of the jump is written into the instruction fetch address queue 20 as the end offset; otherwise, the end offset in the instruction fetch address queue 20 is set to 63. In addition, if the current instruction block hits a branch instruction of a jump, the jump position in the entry item of the instruction fetch address queue 20 is set to 1.

[0081] Because the prediction of each instruction block is relatively independent, in other embodiments, the predicted starting addresses of multiple instruction blocks may be predicted simultaneously in a parallel manner.

[0082] Accordingly, the branch prediction module 10 is configured to perform the following steps:

[0083] Step A1': In each cycle (i.e., when the current prediction address is initialized or updated), the instruction block where the current prediction address is located is taken as the current instruction block, and the current prediction address is taken as the prediction start address of the current instruction block; then, the prediction start addresses of all instruction blocks in the fixed prediction window starting from the current instruction block are determined according to the prediction start address of the current instruction block;

[0084] As mentioned above, if the range of the first instruction block is [5,63], the range of the second instruction block is [64,127], and the range of the third instruction block is [128,191]. Except for the first instruction block, the starting addresses of other instruction blocks are all 64B aligned.

[0085] Step A2': searching and predicting branch instructions in each instruction block according to the predicted starting address of each instruction block, so as to determine whether each instruction block hits the branch instruction for jumping;

[0086] Step A3': according to the judgment result, if all instruction blocks do not hit the branch instruction of the jump, the information of all instruction blocks is sequentially written into the instruction fetch address queue as the prediction result of the branch prediction module 10; the current prediction address is incremented according to the number of instruction blocks in the fixed prediction window;

[0087] Otherwise, if there is at least one instruction block that hits at least one jump branch instruction, the first jump branch instruction with the smallest linear address is selected from the hit jump branch instructions as the end address of the current instruction block, and the information of the current instruction block and all the previous instruction blocks are written into the instruction fetch address queue as the prediction results, and the current predicted address is updated to the jump address of the first jump branch instruction.

[0088] (II) Instruction fetch address queue 20

[0089] See also Figure 3 The instruction fetch address queue 20 is set to store the prediction results written by the branch prediction module 10 in units of instruction blocks. In the design of the present invention, the instruction fetch address queue has 4096 entries. In this embodiment, each entry in the instruction fetch address queue represents the prediction result of a 64B instruction block. The 64B width is to match the 64B bandwidth of the instruction fetch unit 100 when accessing the first-level instruction cache.

[0090] The structure of each entry in the instruction fetch address queue can be expressed as:

[0091] <valid,line_addr,phys_addr,begin_offset,end_offset,taken> ,

[0092] Among them, valid represents the valid bit; line_addr represents the row linear address; phys_addr represents the row physical address; begin_offset represents the starting offset; end_offset represents the ending offset; taken represents the jump bit.

[0093] Among them, the row linear address, the starting offset and the ending offset determine the range of valid bytes in the 64B instruction block. This range of valid bytes is used to retrieve the memory access instruction recording module 30 to query the memory access instructions in the instruction block. Each entry item in the instruction fetch address queue represents an instruction block. Each instruction block contains a starting offset and an ending offset. The order of each instruction block is connected to form the instruction stream of the target program. The row physical address is used for address matching calculation when the cache replacement decision module 60 queries the reuse distance of the replacement candidate. The jump bit is used to indicate whether the data block corresponding to the current entry item has a branch instruction for jumping.

[0094] like Figure 5 As shown, the instruction fetch address queue 20 is further provided with a commit pointer, a read pointer and a write pointer. These three pointers each point to one of the entries in the instruction fetch address queue. Each pointer can be regarded as a number in the range of [0, 4095].

[0095] The commit pointer of the instruction fetch address queue 20 points to the entry item in the instruction fetch address queue 20 of the instruction block where the next instruction to be submitted is located. When all instructions in a certain instruction block are submitted in the mainstream pipeline, the entry item corresponding to the instruction block can be removed from the instruction fetch address queue, and the commit pointer is increased by 1. Specifically, the write-back unit 500 in the mainstream pipeline is configured to feedback the number of the instruction being submitted in the instruction fetch address queue 20 to the instruction fetch address queue 20 in each cycle. When this number is greater than the commit pointer, it can be confirmed that all instructions in the instruction block pointed to by the commit pointer have been submitted in the mainstream pipeline, and the commit pointer is increased by 1. The entry item before the commit pointer does not need to be actively cleared. The prediction result of the branch prediction module 10 can only be written into the entry item before the commit pointer, thereby automatically overwriting the entry item before the commit pointer.

[0096] The read pointer of the instruction fetch address queue 20 points to the entry of the instruction block where the next instruction to be read is located in the instruction fetch address queue. When all instructions in the instruction block currently read by the instruction fetch unit 100 are read, the read pointer is incremented by 1 to ensure that the read pointer of the instruction fetch address queue 20 always points to the entry of the instruction block where the next instruction to be read is located in the instruction fetch address queue.

[0097] That is, the instruction block between the instruction fetch address queue commit pointer and the read pointer is already in the main pipeline of the processor (ie, the execution flow from the instruction fetch unit 100 to the write-back unit 500).

[0098] The write pointer of the instruction fetch address queue 20 points to the location where the branch prediction module 10 will write next, and is used to determine the write location when the branch prediction module 10 writes a new entry. The write pointer is set to increase by 1 each time the branch prediction module 10 writes a new entry. Therefore, the instruction block between the read pointer and the write pointer is a valid predicted instruction block, and the write pointer and the commit pointer can be combined to determine the empty or full status of the instruction fetch address queue 20 (i.e., whether the instruction fetch address queue 20 is full).

[0099] After the main pipeline of the processor is refreshed, the read and write pointers will roll back to the position of the branch refresh or the position of the commit pointer according to the refresh type. Specifically, when a branch refresh occurs (i.e., the refresh type is branch refresh), the execution unit 300 in the main pipeline will feedback the number of the refreshed branch instruction in the instruction fetch address queue 20 to the instruction fetch address queue 20 as the branch instruction refresh pointer, and the read and write pointers will roll back to the position of the branch refresh. When a write-back refresh occurs (i.e., the refresh type is write-back refresh), the read and write pointers roll back to the position of the commit pointer. It should be noted that the refresh process does not actively clear the data in the queue, but only changes the assignment of the read and write pointers. Branch prediction is not 100% accurate, so the prediction result in the instruction fetch address queue may be wrong. At this time, it is necessary to use the refresh process to roll back the read and write pointers, so as to clear the wrong data and rewrite the correct data.

[0100] (III) Memory access instruction recording module 30

[0101] See also Figure 3 The memory access instruction recording module 30 is configured to receive the instruction execution information fed back by the write-back unit 500, sequentially record the information of the submitted memory access instructions, and write the information of the memory access instructions into a memory access instruction buffer 31. Thus, it is possible to determine whether the instruction block contains a memory access instruction.

[0102] The memory access instruction recording module 30 has a memory access instruction buffer 31, whose storage structure is a table structure, which has 4096 table entries.

[0103] The information of the memory access instruction (ie, the information of the memory access instruction that has been submitted) includes information such as the linear address of the memory access instruction, the memory access address, type and memory access length of the memory access instruction.

[0104] The structure of each entry of the memory access instruction buffer 31 is represented as follows:

[0105] <LineAddr,PhyAddr,InstType,Memlen> ,

[0106] Among them, LineAddr represents the instruction linear address of the memory access instruction; PhyAddr represents the memory access physical address of the memory access instruction when it was last executed; InstType indicates the type of the memory access instruction, InstType∈{DirectInst,InDirectInst}, where DirectInst represents a direct memory access instruction and InDirectInst represents an indirect memory access instruction; Memlen indicates the memory access length of the memory access instruction.

[0107] The memory access instruction recording module 30 is also configured to: whenever a new entry is written to the instruction fetch address queue 20, the starting address (i.e., row linear address + starting offset) of the instruction block corresponding to the entry is used to query the memory access instruction buffer 31, so as to try to obtain the instruction linear address (i.e., position) LineAddr and type InstType of the memory access instruction through query, and output them in turn as the instruction fetch address queue index of the memory access instruction in the memory access instruction sequence to the memory access instruction learning module 40 and the memory access address queue 50. That is, if the instruction block contains multiple memory access instructions, they are arranged in order according to the instruction linear address (i.e., position) of the memory access instruction to be added to the instruction fetch address queue 20. The instruction linear address LineAddr (i.e., position) of the memory access instruction corresponding to the newly added entry of the instruction fetch address queue 20 will be sent to the instruction learning module 40 for query as the instruction fetch address queue index, and this instruction fetch address queue index will be sent to the memory access address queue 50 in the subsequent process to record the position of the memory access instruction. The type of the memory access instruction is used for the memory access instruction learning module 40 to select between multiple modes.

[0108] Therefore, through the memory access instruction recording module 30 , the memory access instruction sequence of the program can be obtained according to the instruction block sequence of the program predicted by the branch prediction module 10 .

[0109] Among them, the memory access instruction recording module 30 is configured to use the instruction linear addresses of the memory access instructions in all table entries of the memory access instruction buffer 31 for hit judgment during the query process, so as to try to obtain the instruction linear address (i.e., position) and type of the memory access instruction by querying all the memory access instructions in the memory access instruction buffer 31.

[0110] The query process of the memory access instruction buffer 31 is as follows Figure 6 As shown. Figure 6 As shown, the hit judgment of the memory access instruction buffer 31 is an inclusion relationship judgment. Each entry item in the instruction fetch address queue 20 corresponds to an instruction block, which is an interval range; therefore, when performing a hit judgment, it is necessary to determine whether the instruction linear address of each memory access instruction is within the interval range where the new entry item is written in the instruction fetch address queue 20. If there is a memory access instruction within the interval range, the memory access instruction query hits, otherwise, the query hit fails, that is, no memory access instruction is hit, indicating that there is no memory access instruction in the corresponding instruction block, and no operation is performed at this time.

[0111] In this embodiment, the newly added entry of the instruction fetch address queue 20 includes the 48-bit start address and 48-bit end address of the instruction block, which can be converted into the high-order label of the high-order 42-bit linear address (i.e., the row linear address) and the low-order 6-bit start offset and end offset. Therefore, in the query process, the label and offset of the instruction block are used to query all the memory access instructions in the memory access instruction buffer 31. When the high-order label of the instruction linear address of a memory access instruction is the same as the row linear address of the instruction block, and its low-order offset is greater than or equal to the start offset of the instruction block and less than or equal to the end offset of the instruction block, it means that the memory access instruction is located in the instruction block, that is, the memory access instruction query hits.

[0112] In addition, the present invention further adds a step size predictor and a time correlation predictor based on the memory access instruction buffer 31. These predictors are included in the memory access pattern learning module 40 which will be described in detail below, so as to predict the memory access instructions that conform to a specific pattern.

[0113] (IV) Access Mode Learning Module 40

[0114] See also Figure 3 The memory access mode learning module 40 is configured to record the memory access instruction sequence sent by the memory access instruction recording module 30 in the memory access history buffer 41, learn the memory access mode of the memory access instruction according to the historical information stored in the memory access history buffer 41, and predict the memory access physical address of each memory access instruction in the memory access instruction sequence according to the learned memory access mode, and write the memory access physical address of each memory access instruction as the prediction result into the memory access address queue 50.

[0115] The memory access mode learning module 40 includes a memory access history buffer 41 (as shown in FIG. 7 ), which is configured to receive the PC and memory access physical address of the memory access instruction fed back by the processor in the write-back phase, and write the memory access physical address of the memory access instruction into the table entry corresponding to the memory access instruction in the memory access history buffer 41 according to the PC of the memory access instruction, and each table entry is a piece of historical information.

[0116] It should be noted that the memory access instruction buffer 31 of the memory access instruction recording module 30 and the memory access history buffer 41 of the memory access mode learning module 40 have certain similarities, and the difference between the two is that the memory access instruction buffer 31 in the memory access instruction recording module 30 stores instruction-related information, such as the instruction linear address of the memory access instruction, the memory access instruction type, etc.; the memory access instruction buffer also stores the memory access physical address, but only records the result of the last memory access physical address. The memory access history buffer 41 in the memory access mode learning module 40 is mainly used to record the past memory access history, and records the past 12 memory access physical addresses, on this basis, the memory access mode can be learned. Since the two storage arrays of the memory access instruction buffer 31 and the memory access history buffer 41 do have certain similarities, the two arrays are designed to be equal in size. Although the memory access instruction buffer 31 and the memory access history buffer 41 are logically two arrays, in the specific implementation, the memory access instruction buffer 31 and the memory access history buffer 41 can be uniformly stored in the same hardware location to be merged together.

[0117] The memory access history buffer 41 is an array indexed by the PC of the memory access instruction, with a total of 4096 entries. Each entry in the memory access history buffer 41 is a 12-entry circular queue, recording the physical addresses of the past 12 memory accesses of the same memory access instruction. Memory access instructions with the same PC are identified as the same memory access instruction.

[0118] The memory access mode learning process based on the memory access history buffer in the present invention is significantly different from the existing algorithm. As shown in FIG. 7(a), the traditional time-related prefetch algorithm acquires the memory access mode based on the global invalidation buffer rather than the memory access history buffer 41. In the time-related prefetch algorithm, cache invalidation is the trigger condition for generating the prefetch request. The address A of the cache invalidation is used to search the global invalidation buffer to obtain the address sequence {A, B, C, D} with address A as the first address. The three addresses {B, C, D} are the prefetch candidates of address A, and A, B, C, and D can all be used to send prefetch requests. In order to ensure that the prefetch request can cover more cache invalidation scenarios, the longest match principle is generally adopted when retrieving the address sequence with address A as the first address, so that more prefetch requests can be issued. And in order to prevent excessive prefetch requests from wasting the cache system bandwidth, the number of prefetch requests issued is usually restricted by the prefetch depth (degree). For traditional algorithms, the length of the address sequence (i.e., the prefetch depth) does not affect the accuracy of the prediction, and the length only plays a role in controlling the prefetch advance amount.

[0119] As shown in FIG. 7( b ), the instruction block where the memory access instruction M in the present invention is located may appear multiple times in the instruction fetch address queue 20, and the present invention needs to independently perform address prediction for each occurrence of the memory access instruction M. If the behavior of the memory access physical address of the memory access instruction M conforms to the pattern {A, B, C, D}, and A, B, C, and D are all the memory access physical addresses of the memory access instruction M, then the present invention must learn that the length of the step pattern is 4 before it can accurately predict that the memory access physical address of the memory access instruction M when it appears for the fifth time is A. Therefore, the step pattern of the memory access pattern obtained through learning is a factor that must be considered in the learning stage of the present invention.

[0120] Therefore, the memory access pattern learning module 40 includes a step length predictor 42 and a time correlation predictor 43, so that the memory access pattern learning module 40 can realize the learning and prediction of multiple memory access patterns such as step length pattern, time correlation pattern, etc. through the step length predictor 42 and the time correlation predictor 43 based on the memory access history buffer 41.

[0121] The step length predictor 42 of the present invention is based on the traditional IBSP (intelligent blind signal processing method) algorithm. In this embodiment, each entry of the step length predictor 42 is used to record a step length mode, and the step length mode refers to the step length memory access sequence. The structure of the entry is as follows: Figure 8As shown, each entry of the step length predictor 42 includes a tag of a step length memory access sequence, a previous address, a previous step length, a trusted counter, a current mode, a first address, a maximum counter and a direction. That is to say, based on the traditional IBSP algorithm, the entry of the step length predictor of the present invention extends the trusted counter, the current mode, the first address and the maximum counter. The original IBSP algorithm only needs to record the step length information, because the pre-fetch operation of the data is only calculated based on the current memory access physical address. For a sequence of memory access physical addresses with a fixed step length mode such as the step length memory access sequence {A, A+K, A+2K,..., A+NK}, the step length predictor of the present invention not only records its step length K, but also records the starting address A of the step length memory access sequence and the maximum count value N. Therefore, in order to increase the accuracy of the step length predictor 42, the step length predictor 42 adds a trusted counter and a current mode. Specifically, when the previous step length is the same as the current step length, the trusted counter is incremented by one, otherwise the trusted counter is reset to zero. When the value of the trusted counter is greater than a certain threshold, the current mode is set to 1, indicating that the behavior of the current memory access instruction is indeed in line with the step length mode. If the current mode is 0, it indicates that the current is still in the learning stage, and the step length predictor in the learning stage does not output the prediction result. The step length predictor also includes a step length and a direction bit. The step length is always a positive value, and the direction bit determines whether the step length memory access sequence is increasing or decreasing. When the step length predictor is allocating entries of the memory access instruction, the first address of the entry is set to the current memory access physical address. In the step length mode counting process, the first address is not updated, and only the last address is updated to the current latest memory access physical address. In this way, the first address of a step length memory access sequence is saved. When the step length predictor 42 is updated for a memory access instruction that conforms to the step length memory access sequence, it is found that the memory access physical address of the current memory access instruction is exactly equal to the first address of the current step length memory access sequence in the step length predictor 42, indicating that the step length memory access sequence corresponding to the first address has been executed again. At this time, the value of the trusted counter is updated to the maximum counter so that the head and tail of the step length memory access sequence are recorded.

[0122] The following is an exemplary step size pattern recognition algorithm.

[0123] Algorithm 1. Step pattern recognition algorithm

[0124] Input: submitted memory access instruction PC and physical address PhysAddr;

[0125] Output: Is it step mode?

[0126] if (current mode == step mode);

[0127] if (the current direction and step size do not match the historical record)

[0128] if(PhysAddr == first address &&(max counter == 0 || trusted counter != max counter))

[0129] Maximum counter = trusted counter;

[0130] else

[0131] Clear the step length related information;

[0132] end if

[0133] else

[0134] TrustedCounter++;

[0135] if (max counter != 0 && trusted counter > max counter)

[0136] Maximum counter = trusted counter;

[0137] end if

[0138] end if

[0139] else

[0140] if (the current direction and step size do not match the historical record)

[0141] Clear the step length related information;

[0142] else

[0143] Trusted counter +1;

[0144] if (trusted counter > learning threshold)

[0145] Current mode = step mode;

[0146] end if

[0147] end if

[0148] end if

[0149] Each entry of the time correlation predictor 43 is used to record a time correlation pattern, which refers to a memory access sequence that appears repeatedly in a fixed order, that is, a time correlation memory access sequence. Each entry of the time correlation predictor 43 includes a time correlation pattern, a time correlation sequence length, and a time correlation sequence. For example, we observe that after a memory access sequence such as {A, B, C, D, A, B, C, D}, a sequence such as {B, C, D} has a high probability of appearing immediately after A. There are also time correlation patterns in instruction-level memory access behavior.

[0150] Algorithm 2. Temporal Correlation Pattern Recognition

[0151] Input: The physical addresses of the last 12 memory accesses;

[0152] Output: time-related information;

[0153] Record the physical addresses of the past 12 memory accesses in the memory access history buffer [11:0];

[0154] if(Access history buffer [3n-1:0] matches

[0155] {A1,A2,…,A n ,A1,A2,…,A n ,A1,A2,…,A n})

[0156] Time correlation mode = 1;

[0157] Time correlation sequence length = n;

[0158] Time-related sequence = access history buffer [n-1:0];

[0159] end if

[0160] The time correlation predictor relies on the physical addresses of the past 12 memory accesses of each memory access instruction recorded in the memory access history buffer, and identifies the time correlation pattern based on the 12 historical information. In the algorithm of the present invention, the same memory access sequence must be repeated 3 times to be confirmed as a time correlation pattern, and supports time correlation patterns with a sequence length of no more than 4.

[0161] In addition to this basic time association pattern, the algorithm of the present invention also supports a "step-time association pattern" such as {A, B, C, A+n, B+n, C+n, A+2n, B+2n, C+2n, ...}.

[0162] (V) Memory access address queue 50

[0163] See also Figure 3 The memory access address queue 50 is used to store the prediction results of the memory access instruction recording module 30 and the memory access mode learning module 40, that is, the memory access instruction sequence output by the memory access instruction recording module 30 and the memory access physical address of each memory access instruction output by the memory access mode learning module 40. The content in the memory access address queue 50 can be regarded as the future access sequence of the first-level data cache.

[0164] The memory access address queue 50 has a plurality of entries, each of which corresponds to information of a memory access instruction and corresponds to an entry in the instruction fetch address queue 20. The memory access address queue 50 has a commit pointer, a read pointer, and a write pointer corresponding to the entries pointed to by the commit pointer, the read pointer, and the write pointer in the instruction fetch address queue 20. Specifically, the commit pointer of the memory access address queue 50 points to the entry of the next instruction to be committed in the memory access address queue 50, the read pointer of the memory access address queue 50 points to the entry of the next instruction to be read in the memory access address queue 50, and the write pointer of the memory access address queue 50 points to the location where the memory access instruction recording module 30 and the memory access mode learning module 40 will write next.

[0165] In this embodiment, the memory access address queue 50 has 65536 entries. 65536 is obtained on the basis that the instruction fetch address queue 20 has 4096 entries. Since each entry of the instruction fetch address queue 20 represents a 64B aligned instruction block, we assume that there are at most 16 memory access instructions in each instruction block, so 4096 instruction blocks contain at most 65536 memory access instructions. Each entry in the memory access address queue 50 corresponds to the information of a memory access instruction, and an entry in each instruction fetch address queue 20 may correspond to 0 to 16 entries of the memory access address queue 50. At this time, since not all 64B instruction blocks have 16 memory access instructions, the capacity of the memory access address queue 50 has design redundancy.

[0166] The structure of each entry in the memory access address queue 50 is represented as follows:

[0167] <valid,inst_line_addr,mem_phys_addr,memlen,inst_queue_index> ,

[0168] Among them, valid represents the valid bit; inst_line_addr represents the instruction linear address; mem_phys_addr represents the memory access physical address; memlen represents the memory access length, and cross-line memory access instructions can be identified based on the memory access physical address and memory access length; inst_queue_index represents the instruction fetch address queue index, which is convenient for synchronous refresh of the instruction fetch address queue and the memory access address queue when the pipeline is refreshed.

[0169] Among them, the instruction linear address inst_line_addr, the memory access length memlen and the instruction fetch address queue index inst_queue_index come from the memory access instruction buffer 31 in the memory access instruction recording module 30, and the memory access physical address mem_phys_addr comes from the memory access mode learning module 40 or the memory access instruction recording module 30. If the memory access mode learning module 40 identifies the step mode and / or time association mode to which the memory access instruction belongs, the memory access physical address of the memory access instruction is given by the corresponding predictor in the memory access mode learning module 40; otherwise, it means that the memory access instruction does not belong to a specific mode, and the memory access physical address comes from the physical address of the memory access instruction in the memory access instruction buffer 31 when it was last executed.

[0170] The principle of the refresh process of the memory access address queue 50 when a branch refresh occurs in the main pipeline is as follows: Fig. 9 As shown. When a branch refresh occurs in the mainstream pipeline, the execution unit of the mainstream pipeline feeds back the number of the branch instruction in the instruction fetch address queue 20 to the instruction fetch address queue 20, and the read and write pointers of the instruction fetch address queue 20 are rolled back to the branch refresh position according to the number of the branch instruction as the branch instruction refresh pointer. At the same time, the number of the branch instruction in the instruction fetch address queue 20 is also sent to the memory access address queue 50. The memory access address queue 50 first traverses the queue from the position of the commit pointer, finds the position of the first entry whose instruction fetch address queue index inst_queue_index is equal to the number of the branch instruction in the instruction fetch address queue 20, which is the position of the branch refresh, and then rolls back the read and write pointers to the branch refresh position.

[0171] (VI) Cache replacement decision module 60

[0172] See also Figure 3 The cache replacement decision module 60 is used to implement the cache replacement decision process.

[0173] According to the above description, the contents in the instruction fetch address queue 20 and the memory access address queue 50 are the future access sequence of the first-level instruction cache and the future access sequence of the first-level data cache, respectively.

[0174] The cache replacement decision module 60 is configured to receive the physical addresses of all cache replacement candidates issued by the first-level cache (i.e., the first-level instruction cache L1I), use these physical addresses to retrieve the instruction fetch address queue 20, obtain the reuse distance of each cache replacement candidate, and select a cache replacement candidate as a kick-out item based on the reuse distance to feed back to the first-level cache (i.e., the first-level instruction cache L1I) to kick the cache replacement candidate out of the first-level cache.

[0175] The replacement request processing of the first-level data cache L1D is similar, except that the first-level instruction cache L1I is replaced by the first-level data cache L1D, and the instruction fetch address queue 20 is replaced by the memory access address queue 50 .

[0176] The calculation process of the reuse distance of the cache replacement candidate is as follows: use 64B aligned physical addresses for matching during the retrieval process; starting from the read pointer of the instruction fetch address queue 20 / memory access address queue 50 and ending with the write pointer, the distance between the position of the first entry matching the cache replacement candidate and the read pointer is the reuse distance of the cache replacement candidate. If a cache replacement candidate does not hit, the reuse distance of the cache replacement candidate is set to the maximum value (the maximum value is the total number of entries in the instruction fetch address queue 20 / memory access address queue 50 + 1).

[0177] The cache replacement decision module 60 is also configured to: before retrieving the instruction fetch address queue 20 or the memory access address queue 50, determine whether the valid prediction results in the instruction fetch address queue 20 or the memory access address queue 50 are less than the threshold value of the number of valid prediction results. If so, the replacement decision module 60 will feedback corresponding information to the first-level cache, so that the first-level cache uses the default replacement strategy to select candidates, and the process ends at this time. If there are not enough valid prediction results in the instruction fetch address queue 20 or the memory access address queue 50, the replacement decision module 60 will feedback corresponding information to the first-level cache, and the first-level cache will use the default replacement strategy to select candidates. The default replacement strategy is a prior art, generally an LRU (least used) strategy, but it can also be other strategies.

[0178] Preferably, if the number of valid prediction results in the instruction fetch address queue 20 or the memory access address queue 50 is less than 4096, the default replacement strategy is used. That is, the threshold value of the number of valid prediction results is 4096. The selection of 4096 is based on 8 times the size of the first-level cache. The first-level instruction or data cache has a capacity of 32KB and contains 512 64B cache lines. 512*8=4096. This is why the instruction fetch address queue selects 4096 items. However, in other embodiments, the threshold value of the number of valid prediction results can also be replaced by other values.

[0179] When selecting a cache replacement candidate as a kick-out item based on the reuse distance, if the cache replacement candidate with the smallest reuse distance among multiple cache replacement candidates is unique, then the cache replacement candidate with the largest reuse distance is fed back to the first-level cache as the kick-out item; otherwise, that is, the cache replacement candidate with the smallest reuse distance among multiple cache replacement candidates is not unique, and there are multiple cache replacement candidates with the same reuse distance, then the cache replacement candidate with the smallest reuse distance and the one with the smallest path number sent by the first-level cache is selected and kicked out from the first-level cache.

[0180] The multiple cache replacement candidates sent by the first-level cache are in order. For example, if 4 replacement candidates are sent, but the reuse distances of these 4 cache replacement candidates are the same (all 4 cache replacement candidates are not hit, so the reuse distances are all maximum values), then the first cache replacement candidate is selected.

[0181] Therefore, the cache replacement system based on instruction stream and memory access pattern learning proposed in the present invention can avoid the interference of out-of-order execution and cache prefetching. The prediction object is an independent memory access instruction with simple behavior, which improves the accuracy of memory access sequence prediction and significantly reduces the number of cache failures.

[0182] Based on the cache replacement system based on instruction stream and memory access pattern learning described above, the cache replacement method based on instruction stream and memory access pattern learning implemented includes:

[0183] Step S1: using a branch prediction module and an advance prediction technique to predict the instruction stream of the target program and write the prediction result into an instruction fetch address queue; if the instruction fetch address queue 20 is full, the prediction process is suspended;

[0184] The step S1 comprises:

[0185] Step A1: In each cycle, the instruction block where the current prediction address is located is used as the current instruction block, and the current prediction address is used as the prediction start address of the current instruction block; a fixed prediction window starting from the current instruction block is determined according to the prediction start address of the current instruction block;

[0186] Step A2: after obtaining the predicted starting address of the current instruction block, searching and predicting the branch instruction in the current instruction block to determine whether the current instruction block hits the branch instruction for jumping;

[0187] Step A3: According to the judgment result, if the current instruction block does not hit the branch instruction for jumping, it means that there is no branch instruction in the current instruction block or none of the identified branch instructions jump, then the information of the current instruction block is written into the instruction fetch address queue as the prediction result of the branch prediction module 10; then, the predicted starting address of the next instruction block is determined, and the next instruction block is used as the new current instruction block, and the process returns to step A2 until the current instruction block is the last instruction block in the fixed prediction window. At this time, the current prediction address is incremented according to the number of instruction blocks in the fixed prediction window, thereby updating the current prediction address and entering the next cycle;

[0188] Otherwise, the first jump branch instruction with the smallest linear address is selected from the hit jump branch instructions as the end address of the current instruction block, the information of the current instruction block is written into the instruction fetch address queue as the prediction result, and the current predicted address is updated to the jump address of the first jump branch instruction to enter the next cycle.

[0189] Alternatively, the step S1 includes:

[0190] Step A1': In each cycle (i.e., when the current prediction address is initialized or updated), the instruction block where the current prediction address is located is taken as the current instruction block, and the current prediction address is taken as the prediction start address of the current instruction block; then, the prediction start addresses of all instruction blocks in the fixed prediction window starting from the current instruction block are determined according to the prediction start address of the current instruction block;

[0191] Step A2': searching and predicting branch instructions in each instruction block according to the predicted starting address of each instruction block, so as to determine whether each instruction block hits the branch instruction for jumping;

[0192] Step A3': according to the judgment result, if all instruction blocks do not hit the branch instruction of the jump, the information of all instruction blocks is sequentially written into the instruction fetch address queue as the prediction result of the branch prediction module 10; the current prediction address is incremented according to the number of instruction blocks in the fixed prediction window to enter the next cycle;

[0193] Otherwise, if there is at least one instruction block that hits at least one jump branch instruction, the first jump branch instruction with the smallest linear address is selected from the hit jump branch instructions as the end address of the current instruction block, and the information of the current instruction block and all the previous instruction blocks are written into the instruction fetch address queue as the prediction results, and the current predicted address is updated to the jump address of the first jump branch instruction to enter the next cycle.

[0194] Step S2: using a memory access instruction recording module to record information of submitted memory access instructions, and writing the information of these memory access instructions into its memory access instruction buffer; whenever a new entry is written into the instruction fetch address queue 20, the memory access instruction buffer is queried with the starting address of the instruction block corresponding to the entry, so as to obtain the memory access instruction sequence through querying and output it to the memory access instruction learning module and the memory access address queue;

[0195] Step S3: using a memory access instruction learning module to record a memory access instruction sequence in a memory access history buffer, learning a memory access mode of the memory access instruction according to the historical information stored in the memory access history buffer, and predicting a memory access physical address of each memory access instruction in the memory access instruction sequence according to the learned memory access mode, and writing the memory access physical address of each memory access instruction into a memory access address queue;

[0196] Step S4: Utilize a cache replacement decision module to receive the physical addresses of all cache replacement candidate items issued by the first-level cache, use these physical addresses to retrieve the instruction fetch address queue or the memory access address queue, obtain the reuse distance of each cache replacement candidate item, and select a cache replacement candidate item as a kick-out item based on the reuse distance to feed back to the first-level cache.

[0197] Experimental verification of the present invention:

[0198] The GEM5 simulator is used as the basic experimental environment, and the alpha instruction set is used. The DerivO3CPU fine-grained CPU model is used, the branch prediction algorithm is TAGE_L, the first-level cache is 32KB, the second-level cache is 256KB, the instruction fetch width is 64B, and the memory is 4GB. Five SPEC2006 programs, GemsFDTD, astar, hmmer, lbm, and bwaves, are selected to compare the performance of the method of the present invention with other methods.

[0199] like Fig.10 As shown, the cache replacement method based on instruction stream and memory access mode learning (i.e., RTRDP algorithm) proposed by the present invention improves the performance by an average of 3.2% compared with the LRU algorithm. Compared with the BRRIP algorithm, the BIP algorithm, the SHIP algorithm, and the PERCEPTRON algorithm, the RTRDP algorithm improves the performance by an average of 12.3%, 14.4%, 9.17%, and 18.07%, respectively. This shows that the performance of high-performance processors cannot be fully utilized due to the bottleneck of the current cache system, and the present invention has a significant effect on improving processor performance.

[0200] The method of the present invention is applicable to all processor microarchitectures and is not limited to specific branch prediction algorithms and instruction sets. Of course, due to differences in instruction sets and specific processor microarchitectures, the specific implementation process may vary slightly, but it also falls within the scope of protection of the present invention.

[0201] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A cache replacement system based on instruction stream and memory access pattern learning, characterized in that: It includes a branch prediction module, an instruction fetch address queue, a memory access instruction recording module, a memory access mode learning module, a memory access address queue and a cache replacement decision module; The branch prediction module is configured to use an advance prediction technique to predict the instruction stream of the target program and write the prediction result into the instruction fetch address queue; The memory access instruction recording module is configured to sequentially record information of submitted memory access instructions and write the information of the memory access instructions into its memory access instruction buffer; at the same time, whenever a new entry is written into the instruction fetch address queue, the memory access instruction buffer is queried with the starting address of the instruction block corresponding to the entry, so as to obtain a memory access instruction sequence through query attempts and output it to the memory access instruction learning module and the memory access address queue; The memory access mode learning module is configured to record the memory access instruction sequence in its memory access history buffer, learn the memory access mode of the memory access instruction according to the historical information stored in the memory access history buffer, and predict the memory access physical address of each memory access instruction in the memory access instruction sequence according to the learned memory access mode and write it into the memory access address queue; The cache replacement decision module is configured to receive the physical addresses of all cache replacement candidates issued by the first-level cache, use these physical addresses to retrieve the instruction fetch address queue or the memory access address queue, obtain the reuse distance of each cache replacement candidate, and select a cache replacement candidate as a kick-out item based on the reuse distance to feed back to the first-level cache.

2. The cache replacement system based on instruction stream and memory access pattern learning according to claim 1, characterized in that: The branch prediction module is configured to write the prediction result into the instruction fetch address queue with instruction block as the granularity; and the branch prediction module is configured to execute: Step A1: In each cycle, the instruction block where the current prediction address is located is used as the current instruction block, and the current prediction address is used as the prediction start address of the current instruction block; a fixed prediction window starting from the current instruction block is determined according to the prediction start address of the current instruction block; Step A2: after obtaining the predicted starting address of the current instruction block, searching and predicting the branch instruction in the current instruction block to determine whether the current instruction block hits the branch instruction for jumping; Step A3: According to the judgment result, if the current instruction block does not hit the branch instruction to jump, it means that there is no branch instruction in the current instruction block or the identified branch instructions do not jump, then the information of the current instruction block is written into the instruction fetch address queue as the prediction result of the branch prediction module; then, the predicted starting address of the next instruction block is determined, and the next instruction block is used as the new current instruction block, and the process returns to step A2 until the current instruction block is the last instruction block in the fixed prediction window, at which time the current prediction address is incremented according to the number of instruction blocks in the fixed prediction window to enter the next cycle; Otherwise, select the first jump branch instruction with the smallest linear address from the hit jump branch instructions as the end address of the current instruction block, write the information of the current instruction block into the instruction fetch address queue as the prediction result, and update the current prediction address to the jump address of the first jump branch instruction to enter the next cycle. Alternatively, the branch prediction module is configured to write the prediction result into the instruction fetch address queue with instruction blocks as the granularity; and the branch prediction module is configured to perform the following steps: Step A1': in each cycle, the instruction block where the current prediction address is located is taken as the current instruction block, and the current prediction address is taken as the prediction start address of the current instruction block; then, the prediction start addresses of all instruction blocks in the fixed prediction window starting from the current instruction block are determined according to the prediction start address of the current instruction block; Step A2': searching and predicting branch instructions in each instruction block according to the predicted starting address of each instruction block, so as to determine whether each instruction block hits the branch instruction for jumping; Step A3': According to the judgment result, if all instruction blocks do not hit the branch instruction of the jump, the information of all instruction blocks is written into the instruction fetch address queue in sequence as the prediction result of the branch prediction module; the current prediction address is incremented according to the number of instruction blocks in the fixed prediction window to enter the next cycle; Otherwise, if there is at least one instruction block that hits at least one jump branch instruction, the first jump branch instruction with the smallest linear address is selected from the hit jump branch instructions as the end address of the current instruction block, and the information of the current instruction block and all the previous instruction blocks are written into the instruction fetch address queue as the prediction results, and the current predicted address is updated to the jump address of the first jump branch instruction to enter the next cycle.

3. The cache replacement system based on instruction stream and memory access pattern learning according to claim 2, characterized in that: The information of the instruction block includes the row linear address, row physical address, start offset, end offset and jump bit of the instruction block; When writing the information of the instruction block into the instruction fetch address queue, the predicted starting address of the instruction block is split into a high-order row linear address and a low-order starting offset, and the row physical address is obtained by querying the TLB through the row linear address, and the obtained row linear address, row physical address and starting offset are written into the corresponding fields in the entry of the instruction fetch address queue; In addition, if the instruction block hits a jump branch instruction, the end byte offset of the first jump branch instruction is written into the instruction fetch address queue as the end offset; otherwise, the end offset in the instruction fetch address queue is set to 63.

4. The cache replacement system based on instruction stream and memory access pattern learning according to claim 1, characterized in that: The structure of each entry in the instruction fetch address queue is: <valid,line_addr,phys_addr,begin_offset,end_offset,taken> , Among them, valid represents the valid bit; line_addr represents the line linear address; phys_addr represents the line physical address; begin_offset represents the starting offset; end_offset represents the ending offset; taken represents the jump bit; The instruction fetch address queue is also provided with a commit pointer, a read pointer and a write pointer; the commit pointer of the instruction fetch address queue points to the entry item of the instruction block where the next instruction to be committed is located in the instruction fetch address queue; the read pointer of the instruction fetch address queue points to the entry item of the instruction block where the next instruction to be read is located in the instruction fetch address queue; the write pointer of the instruction fetch address queue points to the position where the branch prediction module will write next; after the mainstream pipeline of the processor is refreshed, the read pointer and the write pointer of the instruction fetch address queue are rolled back to the position of the branch refresh or the position of the commit pointer according to the refresh type; The structure of each entry in the memory access instruction buffer is: <LineAddr,PhyAddr,InstType,Memlen> , Wherein, LineAddr indicates the instruction linear address of the memory access instruction; PhyAddr indicates the memory access physical address of the memory access instruction at the last execution; InstType indicates the type of the memory access instruction, InstType∈{DirectInst,InDirectInst}, where DirectInst indicates a direct memory access instruction and InDirectInst indicates an indirect memory access instruction; Memlen indicates the memory access length of the memory access instruction; The memory access history buffer is an array indexed by the PC of the memory access instruction, and each entry in the memory access history buffer records the physical addresses of the past 12 memory accesses of the same memory access instruction; And the structure of each entry in the memory access address queue is: <valid,inst_line_addr,mem_phys_addr,memlen,inst_queue_index> , Among them, valid represents the valid bit; inst_line_addr represents the instruction linear address; mem_phys_addr represents the memory access physical address; memlen represents the memory access length; inst_queue_index represents the instruction fetch address queue index; The memory access address queue has a commit pointer, a read pointer and a write pointer; when a branch refresh occurs in the main pipeline, the read pointer and the write pointer of the memory access address queue are rolled back to the position of the branch refresh.

5. The cache replacement system based on instruction stream and memory access pattern learning according to claim 4, characterized in that: The branch prediction module is configured to: suspend the prediction process if the instruction fetch address queue is full; whether the instruction fetch address queue is full is determined based on a combination of a write pointer and a commit pointer of the instruction fetch address queue.

6. The cache replacement system based on instruction stream and memory access pattern learning according to claim 1, characterized in that: The memory access pattern learning module includes a step length predictor and a time association predictor, so that the memory access pattern learning module can learn and predict the step length pattern and the time association pattern through the step length predictor and the time association predictor based on the memory access history buffer.

7. The cache replacement system based on instruction stream and memory access pattern learning according to claim 6, characterized in that: Each entry of the step length predictor is used to record a step length mode, and the step length mode refers to a step length memory access sequence; each entry of the step length predictor includes a tag of a step length memory access sequence, a previous address, a previous step length, a trusted counter, a current mode, a first address, a maximum counter and a direction; Each entry of the time correlation predictor is used to record a time correlation pattern, which refers to a time correlation memory access sequence. Each entry of the time correlation predictor includes the time correlation pattern, the time correlation sequence length and the time correlation sequence.

8. The cache replacement system based on instruction stream and memory access pattern learning according to claim 1, characterized in that: The cache replacement decision module is configured to: when selecting a cache replacement candidate as a kick-out item based on the reuse distance, if the cache replacement candidate with the smallest reuse distance among multiple cache replacement candidates is unique, then the cache replacement candidate with the largest reuse distance is fed back to the first-level cache as the kick-out item; otherwise, the item with the smallest path number sent by the first-level cache among the cache replacement candidates with the smallest reuse distance is selected and kicked out from the first-level cache.

9. The cache replacement system based on instruction stream and memory access pattern learning according to claim 1, characterized in that: The cache replacement decision module is also configured to: before retrieving the instruction fetch address queue or the memory access address queue, determine whether the valid prediction results in the instruction fetch address queue or the memory access address queue are less than the number threshold of valid prediction results; if so, the replacement decision module feeds back corresponding information to the first-level cache, so that the first-level cache uses the default replacement strategy for candidate selection, and the process ends at this time.

10. A cache replacement method based on instruction stream and memory access pattern learning, characterized in that: include: Step S1: using a branch prediction module and an advance prediction technique to predict the instruction stream of the target program and write the prediction result into an instruction fetch address queue; Step S2: using a memory access instruction recording module to sequentially record information of submitted memory access instructions, and write the information of these memory access instructions into its memory access instruction buffer; whenever a new entry is written into the instruction fetch address queue, the memory access instruction buffer is queried with the starting address of the instruction block corresponding to the entry, so as to obtain a memory access instruction sequence through query attempts and output it to the memory access instruction learning module and the memory access address queue; Step S3: using a memory access instruction learning module to record a memory access instruction sequence in a memory access history buffer, learning a memory access mode of the memory access instruction according to the historical information stored in the memory access history buffer, and predicting a memory access physical address of each memory access instruction in the memory access instruction sequence according to the learned memory access mode, and writing the memory access physical address of each memory access instruction into a memory access address queue; Step S4: Utilize a cache replacement decision module to receive the physical addresses of all cache replacement candidate items issued by the first-level cache, use these physical addresses to retrieve the instruction fetch address queue or the memory access address queue, obtain the reuse distance of each cache replacement candidate item, and select a cache replacement candidate item as a kick-out item based on the reuse distance to feed back to the first-level cache.

Images