A non-blocking cache elastic expansion method for concurrent memory access and related device

Abstract

The application discloses a non-blocking cache elastic expansion method for concurrent memory access and a related device, and belongs to the technical field of data memory access, and comprises the following steps: receiving a memory access request; when a physical address tag in the memory access request does not match a main MSHR in a target Cache group, a new main MSHR is allocated to the memory access request; when the physical address tag in the memory access request matches the main MSHR in the target Cache group, it is judged whether the main MSHR is full; when the main MSHR is full, a target sacrifice slot is determined in parallel in a physical path associated with the target Cache group; the target sacrifice slot is reset in state and instantiated as an expanded MSHR, and then the context in the memory access request is written again. The method and the related device can solve the problems of low system throughput and bus resource waste caused by high concurrent access.

Description

Technical Field

[0001] This invention belongs to the field of data access technology, and relates to a non-blocking cache elastic expansion method and related apparatus for concurrent memory access. Background Technology

[0002] In the fields of high-performance computing and artificial intelligence, sparse matrix-vector multiplication is a core operator, characterized by irregular memory access, high concurrency, and weak locality. To leverage memory-level parallelism (MLP), modern processors commonly employ non-blocking cache technology, utilizing the MSHR (Miss Status Handling Register) to track pending missing transactions.

[0003] However, existing technologies still have the following defects and shortcomings: Structural capacity bottleneck: The capacity of a traditional MSHR subentry is limited by the physical bit width of a cache line (cache block). When the concurrent access to a hot address exceeds the hardware slot limit, it will trigger a concurrency overflow, causing the pipeline to be forced to stop, which severely limits the system throughput.

[0004] Bus resource waste: If a new independent MSHR is simply allocated for an overflow request, the system will send repeated read instructions to the external memory for the same address due to the lack of on-chip state association, resulting in bandwidth waste. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a non-blocking cache elastic expansion method and related apparatus for concurrent memory access. This method and related apparatus can solve the problems of low system throughput and wasted bus resources caused by high concurrency access.

[0006] To achieve the above objectives, this invention discloses a non-blocking cache elastic expansion method for concurrent memory access, comprising: Upon receiving a memory access request, match the physical address tag in the memory access request with the primary MSHR within the target cache group; If the physical address label in the memory access request does not match the primary MSHR in the target cache (cache memory) group, a new primary MSHR is allocated for the memory access request; if the physical address label in the memory access request matches the primary MSHR in the target cache group, it is determined whether the primary MSHR is overflowing. When the primary MSHR is not overflowing, the context in the memory access request is written into the primary MSHR; when the primary MSHR is overflowing, the target sacrificial slot is evaluated and determined in parallel in the physical path associated with the target cache group. The target sacrifice slot is reset and instantiated as an extended MSHR; Write the context in the memory access request into the extended MSHR.

[0007] Furthermore, the process of receiving a memory access request is as follows: The cache front-end request pipeline receives memory access requests.

[0008] Furthermore, the process of allocating a new primary MSHR to the memory access request is as follows: A new primary MSHR is allocated to the memory access request using the Cache Miss handling procedure.

[0009] Furthermore, the process of determining whether the main MSHR is overflowing is as follows: The number of sub-entries in the sub-entry counter inside the main MSHR is read, and it is determined whether the number of sub-entries in the sub-entry counter has reached the hardware saturation threshold. If the number of sub-entries in the sub-entry counter is less than the hardware saturation threshold, the main MSHR is considered not to be overflowing. If the number of sub-entries in the sub-entry counter is greater than or equal to the hardware saturation threshold, the main MSHR is considered to be overflowing.

[0010] Furthermore, the process of determining the target sacrifice slot in parallel evaluation within the physical path associated with the target cache group is as follows: In the physical paths associated with the target cache group, the physical path with the minimum eviction cost is evaluated in parallel and the best sacrifice slot is selected as the optimal sacrifice slot. It is then determined whether the optimal sacrifice slot satisfies the capacity isolation constraint. If the optimal sacrifice slot satisfies the capacity isolation constraint, the optimal sacrifice slot is selected as the target sacrifice slot.

[0011] Furthermore, it also includes: pausing the Cache front-end request pipeline when the optimal sacrifice slot does not meet the capacity isolation constraint.

[0012] Furthermore, the process of resetting the state of the target sacrifice slot and instantiating it as an extended MSHR is as follows: Force the valid bit of the target sacrifice slot to 1; flip the MSHR flag bit of the target sacrifice slot to 1; update the address tag register to the same physical address tag as the main MSHR; initialize the sub-entry counter inside the main MSHR and start counting to obtain the extended MSHR.

[0013] This invention discloses a non-blocking cache elastic expansion system for concurrent memory access, comprising: The matching module is used to receive memory access requests and match the physical address tag in the memory access request with the primary MSHR in the target cache group; The judgment module is used to allocate a new primary MSHR to the memory access request when the physical address label in the memory access request does not match the primary MSHR in the target cache group; and to determine whether the primary MSHR is overflowing when the physical address label in the memory access request matches the primary MSHR in the target cache group. The determination module is used to write the context in the memory access request into the main MSHR when the main MSHR is not overflowing; and to evaluate and determine the target sacrifice slot in parallel in the physical path associated with the target cache group when the main MSHR is overflowing. An extension module is used to reset the state of the target sacrifice slot and instantiate it as an extended MSHR; The write module is used to write the context in the memory access request into the extended MSHR.

[0014] The present invention discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the non-blocking cache elastic expansion method for concurrent memory access.

[0015] The present invention discloses a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the non-blocking cache elastic expansion method for concurrent memory access.

[0016] The present invention has the following beneficial effects: In practical operation, the non-blocking cache elastic expansion method and related apparatus for concurrent memory access described in this invention, when the main MSHR is overflowing, simultaneously evaluates and determines a target sacrificial slot in the physical path associated with the target cache group, resets the state of the target sacrificial slot, and instantiates it as an extended MSHR; the context in the memory access request is written into the extended MSHR, avoiding concurrent overflow that could force the pipeline to stop and improving system throughput. It should be noted that this invention, by simultaneously evaluating and determining the target sacrificial slot in the physical path associated with the target cache group, avoids sending repeated read instructions to external storage for the same address, thus avoiding bandwidth waste and bus resource waste, making it highly practical. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a graph showing the trend of average total execution cycles of the baseline architecture and EME (Encrypted Media Extensions) mechanism under different cache capacity configurations in the simulation experiment. Figure 3 This is a comparison chart of the baseline architecture and the EME mechanism in six representative complex workload execution cycles during the simulation experiment; Figure 4 A heatmap showing the reduction in absolute pipeline blockage rate of the EME mechanism relative to the baseline architecture in the simulation experiment; Figure 5 This is a system structure diagram of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0021] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0022] Example 1 refer to Figure 1 The non-blocking cache elastic expansion method for concurrent memory access described in this invention includes the following steps: 1) Receive a memory access request and determine whether the physical address tag (Request Tag) in the memory access request matches the primary MSHR in the target cache group. If they match, proceed to step 2); otherwise, use the Cache Miss handling process to allocate a new primary MSHR for the memory access request. Specifically, the cache front-end request pipeline receives memory access requests and compares the physical address tag of the memory access request with the existing MSHR in the target cache group to determine whether the physical address tag of the memory access request is in the target cache group. If the physical address tag of the memory access request is not in the target cache group, a new primary MSHR is allocated for the memory access request using the cache miss handling process; if the physical address tag of the memory access request is in the target cache group, the primary MSHR of the memory access request with the physical address tag in the target cache group is determined, and the process proceeds to step 2). 2) Determine whether the main MSHR is overflowing. If the main MSHR is overflowing, proceed to step 3). If the main MSHR is not overflowing, append the context of the memory access request as a sub-entry directly to the main MSHR. The specific operation of step 2) is as follows: Read the number of sub-entry counters inside the main MSHR; Determine whether the number of sub-entries in the sub-entry counter has reached the hardware saturation threshold; When the number of sub-entries in the sub-entry counter has not reached the hardware saturation threshold, it is considered that the main MSHR is not overflowing. Then, the context of the memory access request is directly appended to the main MSHR as a sub-entry, and the memory access request is processed. When the number of sub-entries in the sub-entry counter reaches the hardware saturation threshold, the main MSHR is considered to be overflowing, a concurrent overflow event is triggered, and the process proceeds to step 3.

[0023] It should be noted that this invention, by accurately identifying congestion bottlenecks, can trigger subsequent elastic scaling for high-concurrency hotspots without interfering with regular memory access, making it highly practical.

[0024] 3) In the physical path associated with the target cache group, determine the target sacrifice slot; The hardware control plane intercepts the backpressure signals of the cache front-end request pipeline, preventing the cache front-end request pipeline from stalling; The overflow scoring unit microarchitecture is triggered, and in the physical paths associated with the target cache group, the physical path with the minimum eviction cost is evaluated in parallel and the best sacrifice slot is found. It should be noted that the eviction cost is the cascading cost caused by the eviction of the scheduling deployment unit Pod. Determine whether the optimal sacrificial slot meets the capacity isolation constraint. If the optimal sacrificial slot meets the capacity isolation constraint, then the optimal sacrificial slot is used as the target sacrificial slot. If the optimal sacrificial slot does not meet the capacity isolation constraint, then an expansion slot cannot be allocated to release the backpressure signal, which will cause the cache front-end request pipeline to stop.

[0025] It should be noted that, in the physical paths associated with the target cache group, this invention evaluates in parallel and finds the physical path with the minimum eviction cost as the best sacrifice slot. At the same time, it prevents the cache front-end request pipeline from stalling in the early stage, thereby ensuring that the cache front-end request pipeline is uninterrupted and has low hardware overhead. In addition, this invention abandons the high-latency external buffer pool and uses pure gate circuits for single-cycle pathfinding to meet high frequency timing requirements, so that the cache front-end request pipeline can still maintain high throughput under extreme congestion.

[0026] 4) Reset the state of the target sacrifice slot and instantiate it as an extended MSHR; The specific operation of step 4) is as follows: Force the valid bit of the target sacrifice slot to 1; toggle the MSHR flag bit is_mshr of the target sacrifice slot to 1; update the address tag register to the same physical address tag as the main MSHR; initialize the sub-entry counter and start counting; thus, the physical path is successfully reconfigured and instantiated in place as an extended MSHR.

[0027] 5) Write the context in the memory access request into the extended MSHR.

[0028] Simulation Experiment Based on an architecture simulation platform, this experiment quantitatively analyzes the performance improvement effect of the EME mechanism (this invention) on the system from three aspects: average execution cycle, typical load acceleration performance, and pipeline blocking rate.

[0029] 1. Average execution cycle; Within the 8KB to 1024KB cache capacity range, the average total execution cycle of the baseline architecture (MiCache) and the EME mechanism for processing the entire test set was statistically analyzed.

[0030] Throughout the entire capacity range from 8KB to 1024KB, the average execution cycle of the EME mechanism was consistently lower than that of the baseline architecture, indicating that the EME mechanism can effectively reduce the overall processing time of the system.

[0031] Regarding resource sensitivity, the performance curves of the EME mechanism and the baseline architecture exhibit a performance inversion characteristic. Data observation shows that the baseline architecture with a 1024KB capacity still has a higher average execution cycle than the EME mechanism with only an 8KB capacity. This result objectively demonstrates that for irregular loads causing high-frequency concurrency issues, simply increasing the capacity of the data storage array cannot fundamentally solve the pipeline stall problem caused by limited MSHR resources. The EME mechanism, by breaking the physical isolation between the conventional CacheLine and MSHR, enables the system to have elastic space scheduling capabilities when dealing with concurrency overflow.

[0032] 2. Typical load acceleration; refer to Figure 3 This experiment selects six representative complex workloads to compare the execution cycles of the two architectures under multi-capacity nodes.

[0033] Optimization for Extreme Congestion Loads: Taking rail4284 and europe_osm loads as examples. On very small capacity nodes (e.g., 8KB and 16KB), the execution cycle of the baseline architecture is high. After introducing the EME mechanism, the height of the blue bars decreases significantly. For example, on the 8KB node of rail4284, the execution cycle decreases from approximately... dropped to about This demonstrates that under congested conditions, the EME mechanism successfully absorbed a large number of concurrent requests and maintained pipeline operation by allocating extended MSHRs within the same group.

[0034] Smooth convergence under low to medium concurrency loads: Taking wiki-Talk and ljournal-2008 loads as examples. In the 8KB to 32KB range, the EME mechanism exhibits a performance advantage; however, as the cache capacity expands to 64KB and above, the execution cycles of the two architectures gradually become equal. This phenomenon aligns with the intended logic of the EME mechanism design. When the physical capacity is sufficient to cover the load, cache miss events decrease, reducing the probability of triggering MSHR overflow. Under these circumstances, the EME mechanism does not introduce additional performance overhead, ensuring compatibility with large-capacity configurations and regular loads.

[0035] 3. Production line blockage rate; To more intuitively quantify the reduction in invalid waiting time for the system by the EME mechanism, this experiment extracted pipeline stall data during the simulation process and calculated the absolute reduction in stall rate of the EME mechanism relative to the baseline architecture.

[0036] Confirmation of high-yield range: Reference Figure 4 In the rail4284 load row at the top of the heatmap, at the 16KB and 32KB capacity points, the absolute reduction in blocking rate brought by the EME mechanism reached 60.5% and 60.6%, respectively. This means that in the baseline architecture, more than 60% of the clock cycles are in a pipeline stall state. The EME mechanism effectively reduces this part of the pipeline idle cycle and increases the proportion of effective execution cycles.

[0037] Impact of physical capacity on blocking rate: For most test loads, the horizontal color distribution of the heatmap shows that the 32KB to 128KB range is the primary area where the EME mechanism reduces blocking rate. Within this capacity range, the conflict between limited cache resources and high-frequency concurrent requests is most pronounced, and the dynamic space scheduling of EME achieves maximum effectiveness.

[0038] Robustness of the architecture: At the bottom of the heatmap, special or extreme boundary loads such as vector_only, all_mshr, and lru_killer are included. Under all capacity nodes of these loads, the blocking rate reduction of the EME mechanism remains at 0.0 or a slightly positive value, with no negative values. This objectively demonstrates from a data perspective that the EME mechanism has high architectural stability and does not introduce additional architectural-level performance overhead when facing application scenarios that do not require MSHR expansion or regular memory access.

[0039] Example 2 refer to Figure 5 The non-blocking cache elastic expansion system for concurrent memory access described in this invention includes: The matching module is used to receive memory access requests and match the physical address tag in the memory access request with the primary MSHR in the target cache group; The judgment module is used to allocate a new primary MSHR to the memory access request when the physical address label in the memory access request does not match the primary MSHR in the target cache group; and to determine whether the primary MSHR is overflowing when the physical address label in the memory access request matches the primary MSHR in the target cache group. The determination module is used to write the context in the memory access request into the main MSHR when the main MSHR is not overflowing; and to evaluate and determine the target sacrifice slot in parallel in the physical path associated with the target cache group when the main MSHR is overflowing. An extension module is used to reset the state of the target sacrifice slot and instantiate it as an extended MSHR; The write module is used to write the context in the memory access request into the extended MSHR.

[0040] The module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in each embodiment of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0041] Example 3 A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the non-blocking cache elastic expansion method for concurrent memory access, for example, including: receiving a memory access request; matching the physical address label in the memory access request with the primary MSHR in a target cache group; when the physical address label in the memory access request does not match the primary MSHR in the target cache group, allocating a new primary MSHR for the memory access request; when the physical address label in the memory access request matches the primary MSHR in the target cache group, determining whether the primary MSHR is overflowing; when the primary MSHR is not overflowing, writing the context in the memory access request into the primary MSHR; when the primary MSHR is overflowing, evaluating and determining a target sacrificial slot in parallel in the physical path associated with the target cache group; resetting the state of the target sacrificial slot and instantiating it as an extended MSHR; and writing the context in the memory access request into the extended MSHR. The memory may include main memory, such as high-speed random access memory (RAM), or non-volatile memory, such as at least one disk storage device. The processor, network interface, and memory are interconnected via an internal bus, which may be an industry-standard architecture bus, a peripheral component interconnection standard bus, or an extended industry-standard architecture bus. The bus can be categorized as an address bus, data bus, or control bus. The memory stores programs; specifically, the program may include program code, which includes computer operation instructions. The memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0042] Example 4 A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the non-blocking cache elastic expansion method for concurrent memory access. For example, the method includes: receiving a memory access request; matching the physical address label in the memory access request with the primary MSHR in the target cache group; when the physical address label in the memory access request does not match the primary MSHR in the target cache group, allocating a new primary MSHR for the memory access request; when the physical address label in the memory access request matches the primary MSHR in the target cache group, determining whether the primary MSHR is overflowing; when the primary MSHR is not overflowing, writing the context in the memory access request into the primary MSHR; when the primary MSHR is overflowing, evaluating and determining a target sacrificial slot in parallel in the physical path associated with the target cache group; resetting the state of the target sacrificial slot and instantiating it as an extended MSHR; and writing the context in the memory access request into the extended MSHR. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0043] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and disclosure of the invention. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0044] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

[0045] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A non-blocking cache elastic expansion method for concurrent memory access, characterized in that, include: Upon receiving a memory access request, match the physical address tag in the memory access request with the primary MSHR within the target cache group; If the physical address label in the memory access request does not match the primary MSHR in the target cache group, a new primary MSHR is allocated to the memory access request; if the physical address label in the memory access request matches the primary MSHR in the target cache group, it is determined whether the primary MSHR is overflowing. If the primary MSHR is not overflowing, the context in the memory access request is written into the primary MSHR; When the primary MSHR is full, the target sacrifice slot is determined in parallel in the physical path associated with the target cache group; The target sacrifice slot is reset and instantiated as an extended MSHR; Write the context in the memory access request into the extended MSHR.

2. The non-blocking cache elastic expansion method for concurrent memory access according to claim 1, characterized in that, The process of receiving a memory access request is as follows: The cache front-end request pipeline receives memory access requests.

3. The non-blocking cache elastic expansion method for concurrent memory access according to claim 1, characterized in that, The process of allocating a new primary MSHR for the memory access request is as follows: A new primary MSHR is allocated to the memory access request using the Cache Miss handling procedure.

4. The non-blocking cache elastic expansion method for concurrent memory access according to claim 1, characterized in that, The process of determining whether the main MSHR is overflowing is as follows: The number of sub-entries in the sub-entry counter inside the main MSHR is read, and it is determined whether the number of sub-entries in the sub-entry counter has reached the hardware saturation threshold. If the number of sub-entries in the sub-entry counter is less than the hardware saturation threshold, the main MSHR is considered not to be overflowing. If the number of sub-entries in the sub-entry counter is greater than or equal to the hardware saturation threshold, the main MSHR is considered to be overflowing.

5. The non-blocking cache elastic expansion method for concurrent memory access according to claim 2, characterized in that, The process of determining the target sacrifice slot in parallel evaluation within the physical path associated with the target cache group is as follows: In the physical paths associated with the target cache group, the physical path with the minimum eviction cost is evaluated in parallel and the best sacrifice slot is selected as the optimal sacrifice slot. It is then determined whether the optimal sacrifice slot satisfies the capacity isolation constraint. If the optimal sacrifice slot satisfies the capacity isolation constraint, the optimal sacrifice slot is selected as the target sacrifice slot.

6. The non-blocking cache elastic expansion method for concurrent memory access according to claim 5, characterized in that, Also includes: When the optimal sacrifice slot does not meet the capacity isolation constraint, the cache front-end request pipeline will be paused.

7. The non-blocking cache elastic expansion method for concurrent memory access according to claim 1, characterized in that, The process of resetting the state of the target sacrifice slot and instantiating it as an extended MSHR is as follows: Force the valid bit of the target sacrifice slot to 1; flip the MSHR flag bit of the target sacrifice slot to 1; update the address tag register to the same physical address tag as the main MSHR; initialize the sub-entry counter inside the main MSHR, start counting, and obtain the extended MSHR.

8. A non-blocking cache elastic expansion system for concurrent memory access, characterized in that, include: The matching module is used to receive memory access requests and match the physical address tag in the memory access request with the primary MSHR in the target cache group; The judgment module is used to allocate a new primary MSHR to the memory access request when the physical address label in the memory access request does not match the primary MSHR in the target cache group; and to determine whether the primary MSHR is overflowing when the physical address label in the memory access request matches the primary MSHR in the target cache group. The determination module is used to write the context in the memory access request into the main MSHR when the main MSHR is not overflowing; When the primary MSHR is full, the target sacrifice slot is determined in parallel in the physical path associated with the target cache group; An extension module is used to reset the state of the target sacrifice slot and instantiate it as an extended MSHR; The write module is used to write the context in the memory access request into the extended MSHR.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the non-blocking cache elastic expansion method for concurrent memory access as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the non-blocking cache elastic expansion method for concurrent memory access as described in any one of claims 1-7.

Images