Cache access method, controller and cache prediction system

Abstract

The application provides a cache access method, a controller and a cache prediction system, including: receiving a cache access request sent by a requester; the cache access request including target address information; generating a corresponding prediction way number through the way predictor based on the target address information; initiating a pre-access to a data array based on the prediction way number, and simultaneously accessing a tag array in parallel to obtain an actual hit way number; sending the prediction way number and the actual hit way number to the five-state machine control module; determining whether the prediction way number is consistent with the actual hit way number through the five-state machine control module, if consistent, determining that the way predictor is successful in prediction, and returning target data obtained through the pre-access to the requester. In this way, access delay can be reduced, cache access performance can be improved, and control logic can be simplified.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a cache access method, controller, and cache prediction system. Background Technology

[0002] In modern processors, large-scale multi-way set-associative L2 caches are key components for improving system performance. However, traditional cache access methods require serially comparing all paths in the tag array before data can be read from the data array, resulting in high load-to-use latency.

[0003] To address this, existing technologies have introduced path prediction methods to shorten access times. Some schemes employ instruction-based predictors, but their prediction accuracy is insufficient and they are primarily applicable to L1 caches. Other more complex schemes involve collaborative prediction using both L1 and L2 caches, but this not only significantly increases the design complexity of the pipeline and interface but also incurs additional hardware overhead. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a cache access method, controller and cache prediction system that can reduce access latency, improve cache access performance and simplify control logic.

[0005] In a first aspect, the present invention provides a cache access method applied to a controller of a cache prediction system, the cache prediction system further including a path predictor and a five-state machine control module; the method includes:

[0006] Receive cache access requests sent by the requester; the cache access requests include target address information.

[0007] Based on the target address information, the corresponding predicted route number is generated by the route predictor.

[0008] Pre-access to the data array is initiated based on the predicted road number, while the label array is accessed in parallel to obtain the actual hit road number.

[0009] The predicted route number and the actual hit route number are sent to the five-state machine control module.

[0010] The five-state machine control module determines whether the predicted route number matches the actual hit route number. If they match, the route predictor is confirmed to have made a successful prediction, and the target data obtained from the pre-access is returned to the requester.

[0011] In an optional implementation, the step of generating a corresponding predicted road number based on the target address information using a road predictor includes:

[0012] The tag field in the target address information is hashed to obtain the compressed tag value.

[0013] The compressed tag value is matched with the hash tag pre-stored in the road predictor.

[0014] If the compressed tag value and the hash tag are consistent, the route predictor has hit the target and outputs the route number information corresponding to the hash tag as the predicted route number.

[0015] If the compressed label value does not match the hash label, the path predictor has missed.

[0016] The hit or miss result of the path predictor is sent to the five-state machine control module.

[0017] In an optional implementation, the route predictor pre-stores hash tags and corresponding route number information generated from historical access requests; each hash tag has a valid bit flag; the valid bit flag is used to indicate whether the hash tag is valid.

[0018] In an optional implementation, the step of accessing the tag array in parallel to obtain the actual hit road number includes:

[0019] It identifies whether the tag field of the target address information is in the tag entry in the tag array; the tag entry stores the complete tag field and the corresponding road number information.

[0020] If the label field is in the label entry, it is determined that the label array has been hit, and the road number information corresponding to the label field is output as the actual hit road number.

[0021] If the label field is not in the label entry, it indicates that the label array has not been matched.

[0022] The results of whether the tag array has been hit or not are sent to the five-state machine control module.

[0023] In an optional implementation, after the step of the five-state machine control module determining whether the predicted route number matches the actual hit route number, the method further includes:

[0024] If the predicted road number does not match the actual hit road number, the road predictor fails to predict, the pre-access is canceled, and the cached access request and the actual hit road number are written to the replay buffer.

[0025] In subsequent cycles, a cache access request is re-initiated, and the data array is accessed at the same stage as the pre-access, based on the actual hit road number; wherein, the re-initiated cache access request carries the actual hit road number.

[0026] The target data obtained from the revisit is returned to the requester.

[0027] In an optional implementation, if the five-state machine control module receives a tag array miss result, the method further includes:

[0028] The five-state machine control module determines that the cache access request is a cache miss and cancels the pre-access.

[0029] Write the cache access request to the replay buffer, and at the same time send a data access request to the next level cache.

[0030] After obtaining the target data, the target data is populated into the data array and then returned to the requester.

[0031] In an optional implementation, the five-state machine control module receives the tag array hit result and the path predictor miss result, and the method further includes:

[0032] If the five-state machine control module determines that the road predictor has failed, it writes the cache access request and the actual hit road number into the replay buffer.

[0033] In subsequent cycles, a cache access request is re-initiated, and the data array is accessed at the same stage as the pre-access, based on the actual hit road number; wherein, the re-initiated cache access request carries the actual hit road number.

[0034] The target data obtained from the revisit is returned to the requester.

[0035] In an optional implementation, after determining that the path predictor has failed to predict, the method further includes:

[0036] Set the valid bit flag of the hash label corresponding to the predicted route number in the route predictor to invalid.

[0037] And / or, set the valid bit flag of the hash tag corresponding to the actual hit road number in the road predictor to valid.

[0038] In a second aspect, the present invention provides a controller applied to a cache prediction system, the cache prediction system further including a path predictor and a five-state machine control module; the controller includes:

[0039] The request receiving unit is used to receive cache access requests sent by the requester; the cache access request includes target address information.

[0040] The access control unit is used to generate the corresponding predicted route number based on the target address information via the route predictor.

[0041] The access control unit is also used to initiate a pre-access to the data array based on the predicted route number, while simultaneously accessing the tag array in parallel to obtain the actual hit route number.

[0042] The decision-making unit is used to send the predicted route number and the actual hit route number to the five-state machine control module;

[0043] The decision unit is also used to determine whether the predicted route number is consistent with the actual hit route number through the five-state machine control module. If they are consistent, the route predictor is confirmed to have made a successful prediction and the target data obtained from the pre-access is returned to the requester.

[0044] Thirdly, the present invention provides a cache prediction system, including a path predictor, a five-state machine control module, a tag array and a data array, and also includes a controller as described in the above embodiments. The controller is deployed in the cache prediction system to coordinate the operation of the path predictor, the five-state machine control module, the tag array and the data array.

[0045] This application provides a cache access method, controller, and cache prediction system. By generating a predicted path number and initiating a pre-access to the data array after receiving a cache access request, the latency from data retrieval to use can be shortened, thereby improving processor operating efficiency. By using a five-state machine control module to determine the consistency between the predicted path number and the actual hit path number, the use of erroneous data can be avoided, thus ensuring access correctness. In the case of prediction failure or miss, the access mechanism of the replay buffer and the next level cache can be combined to ensure the integrity of the data access process and data availability, thereby improving the fault tolerance and stability of the system. Furthermore, by updating the valid bit flag in the path predictor, the accuracy of subsequent predictions can be improved, thereby reducing hardware overhead while improving the overall performance of the cache prediction system.

[0046] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application are realized and obtained through the structures particularly pointed out in the description, claims and drawings.

[0047] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 A flowchart of a cache access method provided in an embodiment of this application;

[0050] Figure 2 A flowchart of the five-state machine control module provided in this application embodiment;

[0051] Figure 3 A schematic diagram of the controller provided in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of a cache prediction system provided in an embodiment of this application.

[0053] Icons: 1-Request receiving unit; 2-Access control unit; 3-Decision unit; 4-Path predictor; 5-Five-state machine control module; 6-Tag array; 7-Data array; 8-Controller. Detailed Implementation

[0054] To make the objectives, technical solutions, and beneficial effects of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with the embodiments and accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0055] To facilitate understanding of this embodiment, the application scenarios and design concepts of this application will be briefly introduced below.

[0056] In modern processors, the L2 cache (Level 2 cache) typically employs a large-capacity, multi-way set-associative architecture, such as a 16-way design, with a relatively large tag width. Traditional access methods require serial comparison of all paths in the tag array before accessing the data array, often taking more than one cycle to obtain a hit result. This leads to high overall cache access latency, which in turn increases the CPU's (Central Processing Unit's) load-to-use latency and impacts system performance.

[0057] In existing technologies, a common optimization approach is to introduce prediction mechanisms to shorten access time. One type of method uses a predictor based on instruction counters, but its prediction accuracy is insufficient and it is mainly applicable to L1 cache (Level 1 cache) systems, with limited effectiveness for large-capacity L2 caches. Another approach achieves cooperative prediction by adding L2 cache line pointers to the L1 cache, but this method significantly increases the complexity of the pipeline and cache interface, while requiring additional storage resources, resulting in high costs. In large-scale caching scenarios, it also introduces multi-level prediction latency, and the prediction relies on a serial process, creating a performance bottleneck.

[0058] Furthermore, the introduction of existing predictors may lead to inconsistencies in the pipeline stages for accessing the data array: successful predictions allow for early access, while failed predictions require access to the data array based on label comparison results, resulting in differences in pipeline stages and increasing the complexity of cache design. Additionally, incorrect predictions may trigger multiple replays, increasing latency and further degrading system performance.

[0059] To address this, this application proposes a cache access method, controller, and cache prediction system. By introducing hashed tags, the prediction storage overhead is reduced. A five-state machine combined with a replay buffer mechanism is used to ensure that only one replay is needed when the prediction fails. The system also ensures that data array access is always in a unified pipeline stage, thereby reducing replay overhead while reducing load-to-use latency and simplifying cache design complexity.

[0060] After introducing the application scenarios and design concepts of this application, the technical solutions provided by this application will be described in detail below.

[0061] This application provides a cache access method, which is applied to the controller of a cache prediction system. The cache prediction system also includes a path predictor and a five-state machine control module.

[0062] Here, the cache prediction system includes a tag array (TA), a data array (DA), a path predictor, a five-state machine control module, and a controller for coordinating their operation.

[0063] The path predictor is a hardware logic unit used to optimize cache access. It is used to quickly make a high-probability guess of the cache path where the data may be located based on the address information of the access request before performing a formal and complete query on the cached TA.

[0064] The five-state machine control module is the core control hub of the cache access method. It is configured to receive and synthesize the parallel processing results from the path predictor and the TA (Targeting Analyzer), and precisely direct subsequent operations such as confirming data, canceling access, initiating replay, or filling requests based on five preset state combinations (e.g., predictor hit / TA hit and path number matches, predictor hit / TA hit but path number doesn't match, predictor hit / TA miss, etc.). The five states refer to the five different states of the cache request after path prediction and TA comparison.

[0065] The controller can be an L2 cache controller for a multi-core processor.

[0066] Reference Figure 1 Cache access methods include:

[0067] Step S101: Receive a cache access request sent by the requester; the cache access request includes target address information.

[0068] Here, the requester can be one or more cores of the CPU (Central Processing Unit), specifically the load / store unit within the core, or a hardware prefetcher, graphics processing unit, or other bus master device capable of accessing the cache.

[0069] A cache access request can be a read request (Load) or a write request (Store), containing the memory address information of the target data. Typically, the memory address information can be a physical address or a virtual address, used to identify the target storage unit of the requester. Specifically, the target address information can be parsed into three fields:

[0070] The tag field is used to uniquely identify a cache line within a cache group.

[0071] The index field is used to select a specific cache group from multiple associative cache groups.

[0072] Offset field: Used to locate a specific byte or word within a cache line.

[0073] In other feasible embodiments, in addition to the target address information, the cache access request may also carry other auxiliary information, such as: access type (read request, write request, instruction fetch request, etc.), security attributes, service quality level, or status information related to the consistency protocol.

[0074] Step S102: Based on the target address information, generate the corresponding predicted route number through the route predictor.

[0075] Here, upon receiving a request, to avoid the latency caused by traditional serial access to TA, a road predictor is used to quickly predict the road where the data may be located, in order to generate a predicted road number.

[0076] In one feasible embodiment, path prediction can be implemented as follows:

[0077] Partial tag matching: Directly extract a portion (such as the high or low bits) of the complete tag field as an index or match.

[0078] Index-based prediction: Predictions can be made based on the index field of the address to count the most frequently accessed paths of a specific cache group.

[0079] More sophisticated prediction algorithms: Advanced algorithms similar to those used in CPU branch prediction can be introduced, such as predictors that combine global or local access history, to improve prediction accuracy.

[0080] Confidence Mechanism: The route predictor can output a confidence score along with the predicted route number. The controller can use this score to decide whether to initiate subsequent pre-accesses, for example, only performing pre-accesses when the confidence score is higher than a certain threshold, in order to achieve a balance between performance and power consumption.

[0081] In one embodiment, step S102 includes the following steps S201-S205.

[0082] Step S201: Hash the tag field in the target address information to obtain the compressed tag value.

[0083] Here, after the controller parses the target address information from the cache access request, it first extracts the tag field used for addressing in the cache. Since the full tag field has a long bit width (e.g., 32 bits or more), directly performing multi-way parallel comparisons on it would consume a lot of time and power. Therefore, this step performs a hash operation on the full tag field to generate a compressed tag value with a significantly shorter bit width.

[0084] In one embodiment, the hashing process uses a folded hashing algorithm. For example, a 32-bit tag field can be divided into a high 16 bits and a low 16 bits, and then these two parts can be XORed to obtain a 16-bit compressed tag value.

[0085] In other feasible embodiments, the hashing algorithm is not limited to this. Depending on the different trade-offs between collision rate, latency, and hardware overhead, the hashing algorithm can be customized or configured by the user according to the application scenario. For example, a multinomial hashing algorithm with slightly higher computational overhead but potentially lower collision rate, an algorithm based on cyclic redundancy check (CRC) code generation, or other algorithms that can quickly generate representative short signatures from long tag fields using bit extraction, bit flipping, bit rotation, or combinations of various logical operations can be used.

[0086] Step S202: Match the compressed tag value with the hash tag pre-stored in the road predictor.

[0087] Here, after generating the compressed tag value, the controller performs parallel matching or comparison operations with a set of hash tags pre-stored inside the route predictor. The internal storage structure of the route predictor can be a multi-way associative storage similar to the data area structure of the cache, with each row corresponding to a cache set, and each row storing the hash tags and corresponding road number information of multiple roads in that set.

[0088] The pre-stored hash labels are preferably dynamically learned and generated from historical access requests. For example, when a cached access finally confirms a hit on a certain path, the controller can hash the complete label of that access and write the resulting hash label and the corresponding actual hit path number into the corresponding position of the path predictor as the basis for subsequent predictions.

[0089] Step S203: If the compressed tag value and the hash tag are consistent, the route predictor is confirmed to have hit the target, and the route number information corresponding to the hash tag is output as the predicted route number.

[0090] Here, if the compressed tag value generated by the input request is exactly the same as a hash tag stored in the route predictor, then the route predictor is considered to have hit the target. At this time, the route predictor will output the route number information corresponding to the matching hash tag, which will be used as the predicted route number for this cache access.

[0091] Step S204: If the compressed tag value is inconsistent with the hash tag, it is determined that the road predictor has missed.

[0092] Here, if the compressed tag value is inconsistent with all the hash tags stored in the road predictor, it is determined that the road predictor has missed. In this case, the road predictor will not output a valid predicted road number, or it will output a specific invalid indication signal.

[0093] Step S205: Send the hit or miss result of the path predictor to the five-state machine control module.

[0094] Here, the path predictor sends the result of this prediction, i.e., the hit or miss status signal, along with the predicted path number generated in the case of a hit, to the subsequent five-state machine control module.

[0095] In one embodiment, the route predictor pre-stores hash tags and corresponding route number information generated from historical access requests; wherein each hash tag has a valid bit flag; the valid bit flag is used to indicate whether the hash tag is valid.

[0096] Here, the hash tags stored in the path predictor can be dynamically generated and updated by historical access requests, and a validity bit can be set for each hash tag to indicate whether the entry is valid. The validity bit is used to indicate whether its corresponding hash tag is currently valid. For example, when a prediction entry is first created or verified after a successful prediction, its validity bit can be set to a valid state. Conversely, when a prediction made using that entry is confirmed as incorrect by subsequent actual access results, its validity bit can be set to an invalid state.

[0097] In other embodiments, the internal structure of the road predictor can be optimized by combining other strategies, such as introducing a Least Recently Used (LRU) strategy to dynamically replace hash labels to improve prediction accuracy.

[0098] Step S103: Initiate a pre-access to the data array based on the predicted route number, and simultaneously access the tag array in parallel to obtain the actual hit route number.

[0099] Here, cache access is performed in two parallel paths.

[0100] Path 1 (Predicted Path): The controller initiates a speculative pre-access to the DA using the generated predicted path number. Specifically, the controller selects the cache group using the index field of the target address, then selects a specific path within that group using the predicted path number, and begins reading the cache line data at that location. This access is a pre-access, and its correctness has not yet been verified. If the prediction is subsequently proven correct, the requested data has already been retrieved from the DA in advance, eliminating the need to wait for the TA comparison to complete, thus saving at least one clock cycle of latency and significantly reducing load-to-use latency. In one embodiment, the pre-access reads all data in the corresponding cache line in the DA. In other embodiments, the pre-access may only read the critical word first fields urgently needed by the requester to reduce unnecessary bus bandwidth and power consumption.

[0101] Path Two (Verification Path): While pre-accessing the data array, the controller performs a regular TA access. The controller achieves this by comparing the complete, uncompressed tag field in the target address information with all tag entries stored in the TA. If a matching entry is found in the TA, the TA is considered a match, and the corresponding road number is output as the actual matched road number; otherwise, the TA is considered a miss.

[0102] In other feasible embodiments, it may also include:

[0103] Multiple pre-access: When the path predictor can output multiple candidate predicted path numbers (e.g., the two most likely), the controller can initiate pre-accesses on the data arrays of these two paths in parallel to increase the probability of a successful prediction.

[0104] In one embodiment, step S103, which involves parallel access to the tag array to obtain the actual hit road number, includes the following steps S301-S304.

[0105] Step S301: Identify whether the tag field of the target address information is in the tag entry in the tag array; the tag entry stores the complete tag field and the corresponding road number information.

[0106] Here, the tag array in the caching system is a high-speed memory used to store the address tag information of all valid data lines in the current cache. For an N-way set-associative cache, the tag array is divided into multiple groups based on the address index field, with each group containing N tag entries, corresponding to N different ways.

[0107] Each tag entry stores at least two pieces of information:

[0108] The complete tag field is the uncompressed or unhashed complete tag information extracted from the original memory address of the data row, used to uniquely and accurately identify a data row.

[0109] The corresponding road number information is used to indicate which road in the current cache group the tag entry belongs to.

[0110] In addition, in typical cache designs, tag entries usually contain other status bits, such as a valid bit (indicating whether the cache line is valid), a dirty bit (indicating whether the cache line has been modified), and cache coherence protocol status bits (such as MESI status) in multi-core systems.

[0111] The controller parses the tag and index fields from the requested target address information. The controller uses the index field to locate a specific group within the tag array.

[0112] Step S302: If the label field is in the label entry, it is determined that the label array has been hit, and the road number information corresponding to the label field is output as the actual hit road number.

[0113] Here, the controller performs a parallel, synchronous comparison between the tag field extracted from the target address and the complete tag field stored in all N tag entries within the selected group.

[0114] If the complete tag field in the request is exactly the same as the tag content stored in a tag entry in the tag array, then the tag array is considered a hit. This indicates that the data requested by the requester does indeed exist in the cache. At this time, the controller will output the road number information corresponding to the matching tag entry as the actual hit road number for this access.

[0115] Step S303: If the label field is not in the label entry, determine that the label array has not been hit.

[0116] Here, if, after parallel comparison, no tag entry in this group has a complete tag field that matches the requested tag field, or if a match exists but its valid bits are invalid, then it is determined that the tag array has not been hit. This indicates that the requested data does not currently exist in this level of cache.

[0117] In other embodiments, to optimize power consumption, accessing and comparing the tag array is not limited to comparing all paths in parallel at once. For example, in some low-power modes, a phased or grouped comparison method can be used. In addition, besides storing the complete tag field and path number information, the tag entry can also contain other status information, such as the status bits of cache coherence protocols (such as MESI) and the status bits of the LRU (Least Recently Used) algorithm. This information works in conjunction with the comparison process to jointly complete the verification of a cache access.

[0118] Step S304: Send the result of whether the tag array has been hit or not to the five-state machine control module.

[0119] Here, the access result of the tag array, i.e., the hit or miss status signal, along with the actual hit route number generated in the case of a hit, is sent to the subsequent five-state machine control module. This result is then compared with the prediction result from the route predictor. Based on the combination of the two, the five-state machine control module generates one of five precise states and executes the corresponding subsequent processing flow, thereby ensuring the accuracy and efficiency of the entire cache access process.

[0120] Step S104: Send the predicted route number and the actual hit route number to the five-state machine control module.

[0121] Here, after obtaining the predicted route number and the actual hit route number, these two results are transmitted to the five-state machine control module.

[0122] Step S105: The five-state machine control module determines whether the predicted route number is consistent with the actual hit route number. If they are consistent, the route predictor is confirmed to have made a successful prediction, and the target data obtained from the pre-access is returned to the requester.

[0123] Here, in the scenario of successful prediction (i.e., the five-state machine control module receives the hit result from the path predictor and the hit result from the tag array), the five-state machine control module receives the predicted path number and the actual hit path number, and compares the two. If the two are completely consistent, it indicates that the path predictor's prediction is correct.

[0124] At this point, the pre-access result of DA in the parallel path is confirmed as valid. The controller does not need to wait and can immediately return the pre-read target data to the requester via the data bus.

[0125] In other embodiments, the five-state machine control module is a decision core capable of accurately processing various prediction results. It is also responsible for handling complex situations such as prediction failures (including path predictor hits but TA misses, path predictor misses but TA hits, both hit but path numbers are inconsistent) and cache misses (both misses). Based on different states, it performs a series of precise control operations, such as canceling pre-access, writing requests to the replay buffer, updating the path predictor state, or initiating access to the next-level cache, thus forming a complete, efficient, and robust cache access control system.

[0126] In one embodiment, if the five-state machine control module receives the hit result of the path predictor and the hit result of the tag array, after the step of determining whether the predicted path number and the actual hit path number are consistent through the five-state machine control module in step S105, the method further includes the following steps S401-S403.

[0127] Step S401: If the predicted road number is inconsistent with the actual hit road number, the road predictor fails to predict, the pre-access is canceled, and the cached access request and the actual hit road number are written to the replay buffer.

[0128] Here, a road predictor failure means that the data does exist in the cache (i.e., the label array is hit), but the road predictor fails to give the correct road number.

[0129] If the path predictor provides a predicted path number (path predictor hit), but this predicted path number is inconsistent with the actual hit path number returned by the tag array, it is determined that the path predictor prediction failed.

[0130] This situation typically occurs due to a tag aliasing problem in the hash algorithm used by the road predictor. That is, two or more different complete tags generate the same compressed tag value after hashing. Therefore, although the compressed tag value accessed in this instance happens to match a hash tag stored in the road predictor, that hash tag actually corresponds to another different, historically stored complete tag.

[0131] Because the prediction failed, the data read from the previous pre-access to DA based on the incorrect predicted route number is invalid and erroneous. To prevent data contamination, the controller immediately performs a cancellation operation, discarding the speculative read from the pipeline to prevent the use of erroneous speculative data.

[0132] After determining that the prediction has failed and canceling the pre-access, in order to correct the error later, the controller will write the current cache access request, along with the correct actual hit road number provided by the TA, into the replay buffer.

[0133] The replay buffer is a dedicated hardware storage unit used to temporarily store requests that failed to complete successfully in the current pipeline cycle but need to be re-initiated in a later stage. Storing requests with the correct path number in this buffer serves two purposes: first, it avoids blocking subsequent new cache access requests, ensuring pipeline throughput; second, by carrying the known correct path number, it ensures that the next execution of the request will definitely succeed, thus avoiding the performance penalty caused by multiple replays.

[0134] In step S402, a cache access request is re-initiated in a subsequent cycle, and the data array is accessed at the same stage as the pre-access based on the actual hit road number; wherein the re-initiated cache access request carries the actual hit road number.

[0135] Here, in a later available pipeline cycle, the request stored in the replay buffer will be rescheduled and initiated.

[0136] Unlike the initial speculative access, this re-initiated request is a deterministic one. It no longer requires route prediction or TA matching; instead, it directly carries the previously known, correct, actual hit route number. The controller will directly use this route number to access the corresponding cache line in the Data Access Controller (DA), thereby accurately reading the target data required by the requester.

[0137] Specifically, the replay access to the data array occurs at the same pipeline stage as the first pre-access access. For example, if the first pre-access is executed in stage T3 of the pipeline, then this replay access will also be precisely scheduled to execute in stage T3. This ensures that the access timing to the DA remains constant regardless of whether the prediction is successful or failed, greatly simplifying the pipeline design complexity of the cache controller and avoiding the introduction of additional control logic and potential timing risks.

[0138] Step S403: Return the target data obtained from the re-access to the requester.

[0139] Here, after successfully reading the correct target data from the data array via replay access, the controller returns the data to the original requester. Thus, a prediction failure caused by label aliasing is successfully corrected and finally completed.

[0140] In one embodiment, if the five-state machine control module receives a tag array miss result, the method further includes the following steps S501-S503.

[0141] Step S501: The five-state machine control module determines that the cache access request is a cache miss and cancels the pre-access.

[0142] Here, once the TA reports a cache miss, the five-state machine control module will immediately determine the nature of this cache access request as a cache miss.

[0143] In this scenario, if the path predictor had previously mishit a path due to label aliasing and initiated a pre-access to the Data Access Provider (DA), then this pre-access is invalid. The controller will immediately cancel the pre-access operation and discard any invalid data that may have been read from the DA to prevent data contamination. If the path predictor had also previously missed a path, then there was no pre-access operation and no cancellation is necessary.

[0144] The process for handling TA misses is fully compatible with the miss handling process in traditional cache designs without a path predictor, and will not introduce additional architectural modifications or complexity to existing cache miss handling logic and pipeline design.

[0145] Step S502: Write the cache access request to the replay buffer and simultaneously send a data access request to the next level cache.

[0146] Here, after determining that a cache miss has occurred, the controller stores the current failed cache access request in the replay buffer. The state of this request is temporarily stored so that it can be re-pipelined as a cache fill request once the required data is retrieved from the next level of storage.

[0147] At the same time, the cache controller will initiate a data access request to the next-level storage system to retrieve the target data that was missed. Depending on the processor architecture, the next-level cache can be a Level 3 cache (L3 cache) or main memory (such as DDR SDRAM).

[0148] Step S503: After obtaining the target data, fill the data array with the target data and return the target data to the requester.

[0149] Here, after the next-level cache or main memory responds to the data access request and returns the target data, the original request stored in the replay buffer is activated and re-enters the pipeline as a cache fill request.

[0150] The controller first selects a cache line in the Data Address Layer (DA) for replacement based on the cache replacement policy (such as LRU, Least Recently Used). Then, the target data retrieved from the next level of storage is written to the selected cache line. Simultaneously, the controller updates the Data Acquisition Target (TA) and the route predictor, writing the complete label and hash label corresponding to the data block into the relevant entries and setting the validity bits so that subsequent accesses can hit the target.

[0151] Simultaneously or after data population is complete, the controller returns the target data to the original requester via the data bus. This concludes a complete cache miss handling and data population process.

[0152] In one embodiment, the five-state machine control module receives the tag array hit result and the path predictor miss result, and the method further includes the following steps S601-S603.

[0153] Step S601: The five-state machine control module determines that the road predictor has failed to predict, and writes the cache access request and the actual hit road number into the replay buffer.

[0154] Here, when the five-state machine control module receives these two contradictory results, it will take the result of TA as the standard. A TA hit means that this access request is a cache hit, and the data does indeed exist in the cache. However, the miss result of the path predictor indicates that this prediction was a failure.

[0155] In this scenario, since the route predictor failed to provide a predicted route number, the system did not initiate a pre-access operation to the DA, and therefore there was no pre-access to cancel. The controller directly writes the current cache access request, along with the accurate actual hit route number provided by the TA, into a dedicated replay buffer for subsequent processing.

[0156] By incorporating requests that were cache hits but failed to be completed through the fast prediction path into a unified replay mechanism, the overall design of the controller is simplified, allowing all hit scenarios that were not directly predicted to succeed to be completed through a unified process.

[0157] In step S602, a cache access request is re-initiated in a subsequent cycle, and the data array is accessed at the same stage as the pre-access based on the actual hit road number; wherein the re-initiated cache access request carries the actual hit road number.

[0158] Here, in a subsequent available pipeline cycle, the request stored in the replay buffer will be rescheduled and initiated. This re-initiated request is a deterministic request. It carries the previously known correct route number provided by the TA, which the controller will directly use to access the corresponding cache line in the DA, thereby accurately reading the target data.

[0159] Similarly, the replay access to the data array in this scenario occurs at the same pipeline stage as the pre-access access to the data array in a successful prediction scenario.

[0160] Step S603: Return the target data obtained from the re-access to the requester.

[0161] Here, after successfully reading the correct target data from the data array via replay access, the controller returns that data to the original requester.

[0162] In one embodiment, after determining that the path predictor has failed to predict, the method further includes:

[0163] Set the valid bit flag of the hash label corresponding to the predicted route number in the route predictor to invalid.

[0164] Here, the route predictor hit an incorrect route number due to a label alias. In this case, to prevent this incorrect prediction entry from causing the same error in other access requests in the future, the controller will perform a penalty or failure operation.

[0165] In one embodiment, the controller finds the hash tag entry in the road predictor that corresponds to the erroneously predicted road number and sets its valid bit flag to invalid (e.g., from 1 to 0). By invalidating this entry, the road predictor will ignore this invalid entry when matching, even if other requests subsequently generate the same compressed tag value, thereby avoiding the recurrence of the same prediction failure and correcting the erroneous prediction.

[0166] And / or, set the valid bit flag of the hash tag corresponding to the actual hit road number in the road predictor to valid.

[0167] Here, if the TA provides a correct actual hit road number, the controller generates a corresponding hash label based on the complete label of the access request, and then updates the entry in the road predictor corresponding to the actual hit road number: write the newly generated hash label and set its valid bit flag to valid (e.g., from 0 to 1).

[0168] When the predicted road number is inconsistent with the actual hit road number, not only can the previously incorrect entries be invalidated, but the correct mapping relationship is also established, thus completing the full correction of the label alias problem.

[0169] In cases where the tag array hits but the path predictor misses, new, valid cache block information is added to the path predictor, thus training the predictor. When the same access request arrives again in the future, the path predictor will be able to hit successfully, thereby improving the overall prediction hit rate.

[0170] Reference Figure 2 In one specific embodiment, the complete process of the cache access method includes:

[0171] The controller receives a cache access request from the requester, which contains the target address information, and the method begins execution.

[0172] First, based on the target address information, the route predictor performs a rapid determination. This determination is based on matching the compressed tag value obtained by hashing the tag field of the target address with the hash tags stored internally by the route predictor.

[0173] The first scenario: The path predictor hits.

[0174] If the route predictor hits, it generates a predicted route number. The controller immediately initiates a speculative pre-access to the data array based on this predicted route number. Meanwhile, the tag array (TA) continues to compare the complete tag fields in parallel.

[0175] The comparison result for TA was a hit, and an actual hit road number was obtained.

[0176] The controller compares the predicted route number with the actual hit route number; if they match, in the ideal case of a successful prediction, the controller confirms the pre-access is valid, returns the acquired target data directly to the requester, and the access ends.

[0177] The controller, after comparison, found that the predicted road number did not match the actual hit road number. This was a prediction failure caused by a label alias, but a cache hit occurred. The controller canceled the pre-access and wrote the request and the correct actual hit road number to the replay buffer, while correcting the internal state of the road predictor based on the result (e.g., invalidating incorrect predictions and validating correct ones). The request will be deterministically replayed subsequently.

[0178] The matching result for TA was a miss.

[0179] This is a case of a predicted hit but a cache miss caused by a tag alias. The controller cancels the invalid pre-access, writes the request to the replay buffer, and simultaneously sends a data request to the next level cache for subsequent cache filling.

[0180] The second scenario: The path predictor missed.

[0181] If the path predictor misses, the controller will not initiate a pre-access and will directly wait for the authoritative comparison result from the TA.

[0182] The comparison result for TA was a hit, and an actual hit road number was obtained.

[0183] This is a prediction failure but a cache hit. The controller writes the request and the actual hit road number to the replay buffer, and can update the road predictor based on this correct information. The request will be deterministically replayed subsequently.

[0184] The matching result for TA was a miss.

[0185] This is a genuine cache miss. The controller writes the request to the replay buffer and sends a data request to the next level cache, pending subsequent cache filling.

[0186] The cache access method provided in this application significantly shortens cache access latency by generating predicted road numbers in advance and initiating pre-access after receiving a cache access request using a road predictor. The five-state machine control module performs consistency checks between the predicted road numbers and the actual hit road numbers, and re-accesses the data in conjunction with the replay buffer when prediction fails, ensuring the correctness and integrity of data access. Furthermore, when the tag array misses, collaborative access with the next-level cache and updating the valid bit flag of the road predictor after prediction failure can improve data availability and prediction accuracy, thereby reducing hardware overhead while improving the stability and overall performance of the cache system.

[0187] Based on the above embodiments, this application provides a controller, referring to... Figure 3 The controllers used in the cache prediction system include:

[0188] The request receiving unit 1 is used to receive cache access requests sent by the requester; the cache access request includes target address information.

[0189] Access control unit 2 is used to generate the corresponding predicted road number based on the target address information through the road predictor.

[0190] Access control unit 2 is also used to initiate a pre-access to the data array based on the predicted route number, while simultaneously accessing the tag array in parallel to obtain the actual hit route number.

[0191] Decision unit 3 is used to send the predicted route number and the actual hit route number to the five-state machine control module.

[0192] Decision unit 3 is also used to determine whether the predicted route number is consistent with the actual hit route number through the five-state machine control module. If they are consistent, the route predictor is confirmed to have made a successful prediction and the target data obtained from the pre-access is returned to the requester.

[0193] In an optional implementation, the access control unit 2 is further configured to:

[0194] The tag field in the target address information is hashed to obtain the compressed tag value.

[0195] The compressed tag value is matched with the hash tag pre-stored in the road predictor.

[0196] If the compressed tag value and the hash tag are consistent, the route predictor has hit the target and outputs the route number information corresponding to the hash tag as the predicted route number.

[0197] If the compressed label value does not match the hash label, the path predictor has missed.

[0198] The hit or miss result of the path predictor is sent to the five-state machine control module.

[0199] In an optional implementation, the route predictor pre-stores hash tags and corresponding route number information generated from historical access requests; each hash tag has a valid bit flag; the valid bit flag is used to indicate whether the hash tag is valid.

[0200] In an optional implementation, the access control unit 2 is further configured to:

[0201] It identifies whether the tag field of the target address information is in the tag entry in the tag array; the tag entry stores the complete tag field and the corresponding road number information.

[0202] If the label field is in the label entry, it is determined that the label array has been hit, and the road number information corresponding to the label field is output as the actual hit road number.

[0203] If the label field is not in the label entry, it indicates that the label array has not been matched.

[0204] The results of whether the tag array has been hit or not are sent to the five-state machine control module.

[0205] In an optional implementation, if the five-state machine control module receives the hit result of the path predictor and the hit result of the tag array, the decision unit 3 is further used to...

[0206] If the predicted road number does not match the actual hit road number, the road predictor fails to predict, the pre-access is canceled, and the cached access request and the actual hit road number are written to the replay buffer.

[0207] In subsequent cycles, a cache access request is re-initiated, and the data array is accessed at the same stage as the pre-access, based on the actual hit road number; wherein, the re-initiated cache access request carries the actual hit road number.

[0208] The target data obtained from the revisit is returned to the requester.

[0209] In an optional implementation, if the five-state machine control module receives a tag array miss result, decision unit 3 is further configured to:

[0210] The five-state machine control module determines that the cache access request is a cache miss and cancels the pre-access.

[0211] Write the cache access request to the replay buffer, and at the same time send a data access request to the next level cache.

[0212] After obtaining the target data, the target data is populated into the data array and then returned to the requester.

[0213] In an optional implementation, the five-state machine control module receives the tag array hit result and the path predictor miss result, and the decision unit 3 is further configured to:

[0214] If the five-state machine control module determines that the road predictor has failed, it writes the cache access request and the actual hit road number into the replay buffer.

[0215] In subsequent cycles, a cache access request is re-initiated, and the data array is accessed at the same stage as the pre-access, based on the actual hit road number; wherein, the re-initiated cache access request carries the actual hit road number.

[0216] The target data obtained from the revisit is returned to the requester.

[0217] In an optional implementation, after determining that the path predictor has failed, decision unit 3 is further configured to:

[0218] Set the valid bit flag of the hash label corresponding to the predicted route number in the route predictor to invalid.

[0219] And / or, set the valid bit flag of the hash tag corresponding to the actual hit road number in the road predictor to valid.

[0220] The controller provided in this application embodiment can realize the complete process of prediction, pre-access, hit judgment and result feedback after receiving a cache access request, thereby reducing the hardware implementation complexity while improving the speed and accuracy of cache access, and thus improving the overall performance and stability of the cache prediction system.

[0221] Based on the above embodiments, this application provides a cache prediction system, referring to... Figure 4 The cache prediction system includes a path predictor 4, a five-state machine control module 5, a tag array 6, and a data array 7, as well as the aforementioned controller 8. The controller 8 is deployed in the cache prediction system to coordinate the operation of the path predictor 4, the five-state machine control module 5, the tag array 6, and the data array 7.

[0222] Here, the route predictor 4 is a high-speed, small-capacity storage and comparison logic unit. Its storage structure can be largely consistent with the TA (Task Analyzer), but it does not store complete tags. Specifically, it stores hash tags compressed using a hash algorithm, with a bit width preferably half that of a complete TA tag. This makes the time and power consumption required for matching them much less than with a TA. Furthermore, each hash tag entry also carries a valid bit to indicate whether the entry is valid. The route predictor is used to provide a fast, predictive route number upon receiving a cache access request.

[0223] Tag array 6 is a standard component of the cache, and its storage structure is basically the same as that of the route predictor. However, it stores uncompressed, complete tags that correspond one-to-one with the data blocks in the DA. The TA is used to indicate the actual hit route number.

[0224] Data array 7 is the physical unit in the cache that actually stores data blocks. Its storage structure typically stores one complete cache line of data per row, and the controller reads the data stored in it according to the path number provided by the TA or path predictor.

[0225] The five-state control module 5 is the decision-making module of this system. It receives the prediction results from the route predictor and the verification results from the TA. By comparing these two results, it can generate five different state branches. Depending on the state, it executes different control instructions, such as deciding whether to access the DA based on the predicted route number, or whether the control request needs to be stored in the replay buffer.

[0226] Controller 8 is a logic control unit that coordinates the work of all the above modules. It receives external requests and distributes address information to the path predictor and TA; based on the decision results of the five-state machine control module, it controls the data reading, cancellation, or return of DA; it also manages the writing and scheduling of the replay buffer.

[0227] The replay buffer is a dedicated hardware buffer queue used to temporarily store requests that failed to complete in the current pipeline. It can store requests caused by TA misses or route predictor tag aliases, and can also store accurate hit route information provided by the TA.

[0228] Specifically, under the coordination of the controller, the aforementioned modules work together to implement the cache access method described in this application embodiment. When a cache access request arrives, the controller simultaneously sends its address to the route predictor and the TA. The route predictor quickly generates a predicted route number using its hash label, and the controller immediately performs speculative data pre-access according to the predicted route number instruction DA. In parallel, the TA also compares the complete label to generate an actual hit route number.

[0229] Subsequently, the results from the route predictor and the TA (Transmission Target) are sent to the five-state machine control module. Based on the combination of results (e.g., prediction hit and matches TA, prediction hit but doesn't match TA, TA miss), the five-state machine control module outputs corresponding control commands to the controller. The controller executes the final operation according to the commands: if the prediction is successful, the data pre-accessed by the DA (Digital Target) is used directly; if the prediction fails but TA hits, the request and the correct route number are written to the replay buffer, awaiting subsequent replay; if TA misses, the request is also written to the replay buffer, and the data is requested from the next lower-level cache for subsequent cache filling. In this way, the entire system constitutes a highly efficient, accurate, and self-correcting closed-loop control system.

[0230] The cache prediction system provided in this application realizes the coordinated operation of prediction, pre-access, hit judgment and result processing in the cache access process, thereby effectively reducing access latency while ensuring the correctness of data access, and thus improving the overall processing performance and operational stability of the system.

[0231] The computer program product provided in this application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0232] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0233] Furthermore, in the description of the embodiments of this application, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0234] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0235] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0236] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in this application, or make equivalent substitutions for some of the technical features. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application.

Claims

1. A cache access method, characterized in that, A controller applied to a cache prediction system, the cache prediction system further including a path predictor and a five-state machine control module; the method includes: Receive a cache access request sent by the requester; the cache access request includes target address information; Based on the target address information, the corresponding predicted road number is generated by the road predictor; Based on the predicted road number, a pre-access is initiated to the data array, while the tag array is accessed in parallel to obtain the actual hit road number. The predicted route number and the actual hit route number are sent to the five-state machine control module; The five-state machine control module determines whether the predicted route number is consistent with the actual hit route number. If they are consistent, it is determined that the route predictor has made a successful prediction, and the target data obtained from the pre-access is returned to the requester. Based on the target address information, the step of generating the corresponding predicted road number through the road predictor includes: The tag field in the target address information is hashed to obtain the compressed tag value; The compressed tag value is matched with the hash tag pre-stored in the road predictor; If the compressed tag value and the hash tag are the same, it is determined that the road predictor has hit, and the road number information corresponding to the hash tag is output as the predicted road number. If the compressed tag value is inconsistent with the hash tag, it is determined that the path predictor has missed. The hit or miss result of the path predictor is sent to the five-state machine control module; The steps for parallel access to the tag array to obtain the actual hit road number include: The system identifies whether the tag field of the target address information is present in a tag entry in the tag array; the tag entry stores complete tag fields and corresponding road number information. If the label field is in the label entry, and the label array is determined to be hit, the road number information corresponding to the label field is output as the actual hit road number; If the tag field is not in the tag entry, it is determined that the tag array has not been matched; The result of whether the tag array is hit or not is sent to the five-state machine control module.

2. The cache access method according to claim 1, characterized in that, The road predictor pre-stores hash tags and corresponding road number information generated from historical access requests; each hash tag has a valid bit flag; the valid bit flag is used to indicate whether the hash tag is valid.

3. The cache access method according to claim 1, characterized in that, If the five-state machine control module receives the hit result of the path predictor and the hit result of the tag array, after the step of determining whether the predicted path number is consistent with the actual hit path number through the five-state machine control module, the method further includes: If the predicted route number is inconsistent with the actual hit route number, it is determined that the route predictor has failed to predict, the pre-access is canceled, and the cache access request and the actual hit route number are written into the replay buffer. In subsequent cycles, the cache access request is re-initiated, and the data array is accessed at the same stage as the pre-access based on the actual hit road number; wherein the re-initiated cache access request carries the actual hit road number; The target data obtained from the re-access is returned to the requesting party.

4. The cache access method according to claim 1, characterized in that, If the five-state machine control module receives a no-hit result for the tag array, the method further includes: The five-state machine control module determines that the cache access request is a cache miss and cancels the pre-access. Write the cache access request into the replay buffer, and simultaneously send a data access request to the next level cache. After obtaining the target data, the target data is filled into the data array and then returned to the requester.

5. The cache access method according to claim 1, characterized in that, The five-state machine control module receives the tag array hit result and the path predictor miss result. The method further includes: If the five-state machine control module determines that the road predictor has failed to predict, it writes the cache access request and the actual hit road number into the replay buffer. In subsequent cycles, the cache access request is re-initiated, and the data array is accessed at the same stage as the pre-access based on the actual hit road number; wherein the re-initiated cache access request carries the actual hit road number; The target data obtained from the re-access is returned to the requesting party.

6. The cache access method according to any one of claims 3 or 5, characterized in that, After determining that the path predictor has failed to predict, the method further includes: Set the valid bit flag of the hash label corresponding to the predicted route number in the route predictor to invalid; And / or, Set the valid bit flag of the hash label corresponding to the actual hit road number in the road predictor to valid.

7. A controller, characterized in that, The system is applied to a cache prediction system, which also includes a path predictor and a five-state machine control module. The controller is configured to execute the cache access method as described in any one of claims 1-6, the controller comprising: A request receiving unit is used to receive a cache access request sent by a requester; the cache access request includes target address information. Access control unit, used to generate a corresponding predicted road number through the road predictor based on the target address information; The access control unit is also used to initiate a pre-access to the data array based on the predicted road number, and simultaneously access the tag array in parallel to obtain the actual hit road number. The decision unit is used to send the predicted route number and the actual hit route number to the five-state machine control module; The decision unit is also used to determine whether the predicted route number is consistent with the actual hit route number through the five-state machine control module. If they are consistent, the decision unit determines that the route predictor has made a successful prediction and returns the target data obtained from the pre-access to the requester.

8. A cache prediction system, characterized in that, It includes a path predictor, a five-state machine control module, a tag array, and a data array, and also includes the controller as described in claim 7, wherein the controller is deployed in the cache prediction system for coordinating the operation of the path predictor, the five-state machine control module, the tag array, and the data array.

Images