Cache memory, request processing method and computer system

Abstract

The invention relates to a cache memory, a request processing method and a computer system. The cache memory comprises an access module used for receiving an access request sent by a central processing unit; determining a target cache group number corresponding to the access request according to an identifier of a target main memory corresponding to the access request; the judgment module is used for determining whether the access request hits the cache line in the target cache group indicated by the target cache group number or not; wherein the target cache group is used for caching data corresponding to the target main memory; and the response module is used for writing the missed access request into a target request scheduling queue corresponding to the identifier of the target main memory, and scheduling the access request to dequeue so as to send the access request to the target main memory. By adopting the cache memory, the storage resource allocation flexibility of the cache memory can be ensured, and the stability of accessing each main memory by the central processing unit is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cache, in particular to a cache memory, a request processing method and a computer system. BACKGROUND

[0002] In a computer system, a cache memory (Cache, referred to as cache) as an intermediate layer between a central processing unit CPU and a main memory, the core function is to break the bottleneck of the mismatch between the processor operation speed and the access speed of the main memory. The cache will temporarily store the data frequently accessed by the central processing unit in the cache line CacheLine inside the cache, so as to reduce the frequency of directly accessing the off-chip main memory.

[0003] With the continuous evolution of computer systems to multi-processor and distributed architecture, the cache memory design scheme under the multi-host scenario (that is, multiple hosts, multiple computing nodes, multiple virtual machines or multiple user processes share the same cache resource) has gradually become mainstream. Therefore, there are scenarios where the central processing unit accesses the data of a single or multiple main memories.

[0004] In the scenario where the central processing unit accesses the data of a single or multiple main memories, how to ensure the flexibility of cache memory storage resource allocation and improve the stability of the central processing unit accessing each main memory needs to be solved. SUMMARY

[0005] Therefore, a cache memory, a request processing method and a computer system capable of ensuring the flexibility of cache memory storage resource allocation and improving the stability of the central processing unit accessing each main memory are provided.

[0006] In a first aspect, the present application provides a cache memory. The cache memory comprises:

[0007] An access module configured to receive an access request sent by a central processing unit; and determine a target cache group number corresponding to the access request according to an identifier of a target main memory corresponding to the access request.

[0008] A determination module configured to determine whether the access request hits a cache line in a target cache group indicated by the target cache group number; wherein the target cache group is configured to cache data corresponding to the target main memory.

[0009] A response module configured to write the access request that does not hit into a target request scheduling queue corresponding to the identifier of the target main memory, and schedule the access request to be dequeued to send the access request to the target main memory.

[0010] In one of the embodiments, the access module is specifically configured to: perform hash calculation on the target index included in the access request by using a hash algorithm to obtain a target hash value; and in the case that the number of the main memories accessible by the central processor is multiple, determine the target cache group number according to the target hash value and the identification of the target main memory included in the access request.

[0011] In one of the embodiments, the access module is specifically configured to: obtain a reference hash value from the target hash value according to the number of the main memories accessible by the central processor; and splice the identification of the target main memory and the reference hash value to obtain the target cache group number with a preset bit width; wherein the preset bit width is determined according to the depth of the cache memory.

[0012] In one of the embodiments, the access module is specifically configured to: in the case that the depth of the cache memory is not a 2n power, perform uniform reduction processing on the target hash value to obtain a candidate hash value, and obtain a reference hash value from the candidate hash value according to the number of the main memories accessible by the central processor.

[0013] In one of the embodiments, the access module is specifically configured to receive the access request sent by the central processor, and write the access request into the request receiving queue corresponding to the identification of the target main memory; after the access request is dequeued, execute the step of determining the target cache group number corresponding to the access request according to the identification of the target main memory corresponding to the access request.

[0014] In one of the embodiments, the judgment module is specifically configured to: in the case that the access request is a write request, if the access request does not hit the cache line in the target cache group corresponding to the target cache group number, transmit the access request to the slave response module, or determine the available cache line from the target cache group, and write the data corresponding to the write request into the available cache line; in the case that the access request is a read request, if the access request does not hit the cache line in the target cache group corresponding to the target cache group number, transmit the access request to the response module.

[0015] In one of the embodiments, the judgment module is specifically configured to: judge whether there is an idle cache line in the target cache group; if yes, take one of the idle first cache lines as the available cache line; and if not, determine the available cache line from the target cache group based on the LRU algorithm.

[0016] In one of the embodiments, the judgment module is specifically configured to: determine a candidate cache line storing the least recently used data in the target cache group; remove the data in the candidate cache line, and take the candidate cache line as the available cache line.

[0017] In one of the embodiments, the judging module is specifically configured to: maintain a group register corresponding to the target cache group; the group register stores identification numbers of cache lines in the target cache group; in the case that there is a free cache line, write the identification number of the free first cache line to the highest bit of the group register, and shift all the cache line identification numbers currently stored in the group register to the lower bit by a preset number of bits; in the case that there is no free cache line, remove data in a second cache line corresponding to the cache line identification number in the lowest bit of the group register, and take the second cache line as a candidate cache line.

[0018] In one of the embodiments, the responding module is specifically configured to: for the access request dispatched from the target request scheduling queue, in the case that the access request is a read request, store the target index included in the access request into the index queue corresponding to the identification of the target main memory.

[0019] In one of the embodiments, the responding module is specifically configured to: receive the target read data returned by the target main memory in response to the access request, and in the case that a read response end signal fed back by the target main memory is received, determine that the target read data is received completely; after the target read data is received completely, store the identification of the target main memory attached to the target read data into the identification queue corresponding to the identification of the target main memory, and store the target read data and the read response end signal into the data queue corresponding to the identification of the target main memory.

[0020] In one of the embodiments, the responding module is specifically configured to: dispatch the main memory identification in the identification queue corresponding to the identification of each target main memory out of the queue; for the identification of the target main memory, according to the identification of the target main memory, dispatch the target index corresponding to the identification of the target main memory out of the queue from the index queue corresponding to the identification of the target main memory, and dispatch the target read data and the read response end signal corresponding to the identification of the target main memory out of the queue from the data queue corresponding to the identification of the target main memory; write the target index and the target read data back to the central processing unit, and / or write the target index and the target read data into the free cache line in the target cache group.

[0021] In a second aspect, the application further provides a request processing method. The method is used in the cache memory as described in any one of the first aspect, and the method comprises:

[0022] The system receives an access request sent by the central processing unit; determines the target cache group number corresponding to the access request based on the identifier of the target main memory corresponding to the access request; determines whether the access request hits a cache line in the target cache group indicated by the target cache group number; wherein the target cache group is used to cache data corresponding to the target main memory; writes the access request that does not hit into the target request scheduling queue corresponding to the identifier of the target main memory, and schedules the access request to be dequeued to send the access request to the target main memory.

[0023] Thirdly, this application also provides a computer system including a central processing unit, at least one main memory, and a cache memory as described in any of the first aspects.

[0024] In the aforementioned cache memory, request processing method, and computer system, the cache memory includes: an access module for receiving access requests sent by the central processing unit; determining the target cache group number corresponding to the access request based on the identifier of the target main memory corresponding to the access request; a judgment module for determining whether the access request hits a cache line in the target cache group indicated by the target cache group number; wherein, the target cache group is used to cache data corresponding to the target main memory; and a response module for writing the missed access request into the target request scheduling queue corresponding to the identifier of the target main memory, and scheduling the access request to be dequeued to send the access request to the target main memory. Since the target cache group number can be determined based on the identifier of the target main memory, that is, the cache group is allocated according to the specific number of existing target main memories, and the hit is determined based on the identifier of the main memory, cache resources can be evenly allocated regardless of whether there is a single main memory or multiple main memories. This avoids problems such as resource shortages in some main memories due to uneven resource allocation, and improves the flexibility of cache memory storage resource allocation. Furthermore, when handling missed access requests, access requests can be isolated based on the identifier of the target main memory and written into the corresponding target request scheduling queue. This can prevent abnormal requests from one main memory from spreading to other main memories and causing mutual interference between main memories, thus achieving main memory isolation. This improves the stability of the central processing unit's access to each main memory and ensures the stable operation of the computer system.

[0025] Other beneficial effects of this application will be further explained and illustrated in detail when combined with the accompanying drawings and specific embodiments. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a diagram illustrating the application environment of a cache memory in one embodiment;

[0028] Figure 2 This is a block diagram of the cache memory in one embodiment;

[0029] Figure 3 This is a schematic diagram of the structure of a cache memory in one embodiment;

[0030] Figure 4 This is a flowchart illustrating the process of determining the target cache group number in one embodiment;

[0031] Figure 5 This is a schematic diagram illustrating the processing procedure of a hash unit in one embodiment;

[0032] Figure 6 This is a schematic diagram of cache group partitioning in one embodiment;

[0033] Figure 7 This is a schematic diagram of another cache group partitioning in one embodiment;

[0034] Figure 8 This is a schematic diagram of another cache group partitioning in one embodiment;

[0035] Figure 9 This is a schematic diagram of a group register in one embodiment;

[0036] Figure 10 This is a schematic diagram of the response control module processing in one embodiment;

[0037] Figure 11 This is a schematic diagram illustrating the process of setting up a multi-host queue in one embodiment;

[0038] Figure 12 This is a schematic diagram of the access request processing flow in one embodiment;

[0039] Figure 13 This is a flowchart illustrating a request processing method in one embodiment. Detailed Implementation

[0040] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that many specific details are set forth in the following description in order to provide a full understanding of this application, but this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0042] It is understandable that "at least one" refers to one or more, while "multiple" refers to two or more.

[0043] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising / including” or “having,” etc., specify the presence of the stated features, wholes, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0044] In computer systems, the cache acts as an intermediary layer between the CPU and main memory, its core function being to overcome the bottleneck of the mismatch between CPU processing speed and main memory access speed. Based on the principle of locality of reference (including spatial and temporal locality), the cache temporarily stores frequently accessed data by the CPU in the internal cache line, thereby reducing the frequency of direct access to off-chip main memory and significantly improving overall system performance. Since the concept of caching was proposed, its architecture has gradually evolved from early direct mapping and fully associative structures to the current mainstream set-associative structure. The core goal has always been to maximize cache hit rate while reducing cache resource overhead by optimizing storage logic and replacement strategies.

[0045] As systems continue to evolve towards multi-processor and distributed architectures, caching designs for multi-host scenarios (i.e., multiple hosts, multiple main memories, multiple compute nodes, multiple virtual machines, or multiple user processes sharing the same cache resource) are gradually becoming mainstream. At this point, the role of caching changes from a "single-entity dedicated acceleration layer" to a "multi-entity shared resource," significantly increasing design complexity.

[0046] Traditional caching technologies have matured in single-host (main memory) scenarios, but their adaptability becomes increasingly apparent when facing multi-host collaborative access requirements: First, resource allocation mechanisms are rigid, often employing static fixed-ratio allocation strategies that cannot be adjusted based on the number of hosts—resource idleness is likely to occur when running on a single host, while resource strain may occur on some hosts during multi-host concurrency, creating performance bottlenecks; Second, cache conflicts and access interference intensify, as different hosts have significantly different access patterns, and traditional hash mapping is not based on the host dimension, easily leading to CacheLine conflicts, and global replacement strategies (such as global LRU) can cause frequent access to one host to "wash out" hot data on other hosts, resulting in an imbalance in cache hit rates across different hosts; Third, there is a lack of effective host isolation mechanisms, as abnormal states of a single host (such as continuous high load, data backpressure, or erroneous requests) may consume a large amount of cache resources and even interfere with the normal access of other hosts, affecting the stability of the entire system.

[0047] In summary, existing multi-host collaborative access mechanisms for caching typically allocate resources by pre-setting a fixed ratio, which has the following limitations: 1. Resource allocation lacks flexibility and does not support dynamic configuration of the number of hosts: In single-host mode, there is resource waste due to idle resources, and in multi-host mode, uneven allocation can easily lead to resource shortages on some hosts, forming performance bottlenecks and affecting overall cache efficiency; 2. Hash algorithms are not adapted to multi-host scenarios and do not incorporate the host dimension into the hit / miss judgment logic, resulting in significant differences in cache hit rates among different hosts and insufficient balance; 3. The LRU replacement strategy is not designed for multi-host scenarios, and the global LRU mechanism does not distinguish between host dimensions for cache line replacement, causing the strategy to fail when accessing multiple hosts collaboratively, and failing to guarantee the effectiveness of caches on each host; 4. There is a lack of effective host isolation mechanisms, and anomalies on a single host (such as backpressure, erroneous requests) can easily spread to other hosts, causing mutual interference and affecting system stability.

[0048] Therefore, cache management mechanisms for multi-main-memory scenarios need to be specifically designed in three dimensions: resource flexibility, access fairness, and isolation, in order to adapt to the needs of multiple entities sharing cache. This has also become an important direction for the development of current cache technology.

[0049] In view of this, this application provides a high-speed cache memory. The core concept is to create a cache collaboration architecture for multi-main-memory scenarios, characterized by "centralized management of multiple main memories and isolation of access requests between main memories." This aims to solve problems such as rigid resource allocation, interference between main memories, and uneven hit rates among main memories in traditional multi-main-memory shared caches. It ensures the flexibility of cache memory resource allocation and improves the stability of the central processing unit's access to each main memory.

[0050] The high-speed cache memory provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is illustrated. The computer system includes a cache memory, a central processing unit (CPU), and at least one main memory. The cache memory receives access requests from the CPU, determines the target cache group number corresponding to the access request based on the identifier of the target main memory, determines whether the access request hits a cache line in the target cache group indicated by the target cache group number, writes the missed access request into a target request scheduling queue corresponding to the identifier of the target main memory, and schedules the access request to be dequeued and sent to the target main memory. The target main memory is one of at least one main memory. Each module in the cache memory can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within the processor in the cache memory or stored in software within the cache memory, allowing the processor to invoke and execute the operations corresponding to each module.

[0051] In one embodiment, such as Figure 2 A cache memory is shown. The cache memory includes an access module, a decision module, and a response module.

[0052] The access module is used to receive target access requests sent by the central processing unit; and to determine the target cache group number corresponding to the target access request based on the identifier of the target main memory corresponding to the target access request.

[0053] The target access request sent by the central processing unit (CPU) can be, for example, a write request or a read request to access the target main memory. The number of main memories that the CPU can access through the cache can be configured to be one or more. The target main memory is one of the one or more main memories.

[0054] The cache memory receives the target access request and identifies the identifier of the target main memory included in the target access request.

[0055] A cache set is a logical grouping unit in the cache used to organize and manage cache lines. The cache set number is a unique identifier used to locate the cache set, enabling fast indexing and avoiding full cache searches. The way indicates the number of cache lines that each cache set can hold. The cache line is the smallest unit of data transfer between the cache and main memory.

[0056] The cache memory in this embodiment adopts a set-associative structure, which can receive multiple access requests to multiple main memories in parallel and asynchronously store each access request. It is understood that this embodiment describes the process of processing a single target access request, and the processing process for any access request is the same.

[0057] The target cache group number corresponding to the access request can be determined based on the identifier of the target main memory. Therefore, the cache groups are divided according to the identifiers of each main memory, creating separate cache groups for different main memories. Each main memory can correspond to multiple cache groups, and the cache lines within each cache group are used to store data for the corresponding main memory. Correspondingly, the target cache group is used to cache the data corresponding to the target main memory.

[0058] The judgment module is used to determine whether the target access request hits the cache line in the target cache group indicated by the target cache group number.

[0059] Once the target cache group number corresponding to the target access request is determined, the cache lines in the target cache group corresponding to the target cache group number can be determined accordingly. For example, the target access request includes a target index. The target index is used to select all paths within the target cache group. The tag field and valid bits stored in the cache line of each path are read out in parallel and compared with the high-order tag of the address in the target access request. If they match, a hit signal is generated, indicating that the access request has hit the cache line in the cache group indicated by the cache group number.

[0060] If a cache hit occurs, the access request is processed in the cache memory. If it is a write request, the data is written to the cache line that was hit. If it is a read request, the data stored in the cache line that was hit is returned to the central processing unit.

[0061] Conversely, if the access request is not hit, it needs to be forwarded to the target main memory to write or read data from the target main memory.

[0062] The response module is used to write the missed target access request into the target request scheduling queue corresponding to the identifier of the target main memory, and schedule the target access request to be dequeued to send the target access request to the target main memory.

[0063] The cache memory deploys request scheduling queues corresponding to the identifiers of each main memory. Missed access requests for each main memory are written to the request scheduling queue indicated by the identifier of the corresponding main memory. It can be understood that, depending on the number of main memories, there may be one or more request scheduling queues. Furthermore, the cache memory uses round-robin (RR) scheduling to dequeue access requests from one or more request scheduling queues and send them to the corresponding main memory. Therefore, for a target access request, if the target access request misses, the missed target access request is written to the target request scheduling queue corresponding to the identifier of the target main memory, and after the target access request is dequeued, it is sent to the target main memory.

[0064] The aforementioned cache memory can receive access requests sent by the central processing unit; determine the target cache group number corresponding to the access request based on the identifier of the target main memory corresponding to the access request; determine whether the access request hits a cache line in the target cache group indicated by the target cache group number; wherein, the target cache group is used to cache data corresponding to the target main memory; write the missed access request into the target request scheduling queue corresponding to the identifier of the target main memory, and schedule the access request to be dequeued to send the access request to the target main memory. Since the target cache group number can be determined based on the identifier of the target main memory, that is, the cache group is allocated according to the specific number of existing target main memories, and the hit is determined according to the identifier of the main memory, cache resources can be evenly allocated regardless of whether there is a single main memory or multiple main memories, avoiding problems such as resource shortages in some main memories due to uneven resource allocation, and improving the flexibility of cache memory storage resource allocation. Furthermore, when handling missed access requests, access requests can be isolated based on the identifier of the target main memory and written into the corresponding target request scheduling queue. This can prevent abnormal requests from one main memory from spreading to other main memories and causing mutual interference between main memories, thus achieving main memory isolation. This improves the stability of the central processing unit's access to each main memory and ensures the stable operation of the computer system.

[0065] In optional embodiments of this application, exemplary examples include... Figure 3A schematic diagram of the cache memory structure is shown. The access module includes an access control unit (Cache_access_ctrl) and a hash unit (Hash). The access control unit receives a target access request sent by the central processing unit, writes the target access request into a request receive queue corresponding to the identifier of the target main memory, and schedules the target access request to be dequeued. The hash unit determines the target cache group number corresponding to the target access request based on the identifier of the target main memory. The judgment module includes a storage unit (Tag_Mem) and a data usage judgment unit (LRU). The storage unit determines whether the target access request hits a cache line in the target cache group indicated by the target cache group number. The data usage judgment unit updates the way in the cache group, and then determines an available cache line from the target cache group and writes the data corresponding to the write request into the available cache line. The response module includes a response control module (Cache_dma_ctrl) and a data return module (Cache_resp_ctrl). The response control module executes the steps of writing the missed target access request into the target request scheduling queue corresponding to the identifier of the target main memory, and scheduling the target access request to be dequeued to send the target access request to the target main memory. The Req queue is the request scheduling queue corresponding to the identifiers of different main memories. The response control module can also execute the steps of receiving the target read data returned by the target main memory in response to the target access request, and determining that the target read data reception is complete upon receiving the read response end signal from the target main memory; after the target read data reception is complete, storing the identifier of the target main memory attached to the target read data into the identifier queue corresponding to the identifier of the target main memory, and storing the target read data and the read response end signal into the data queue corresponding to the identifier of the target main memory. The Pending Req Que is the queue for storing the received data. The data return module is used to return the data returned to the CPU. Among them, the Hit_RespQ queue stores the data returned by the hit request, and the Miss_RespQ queue is used to store the data returned by the missed request.

[0066] In one embodiment, the access module is specifically configured to: receive a target access request sent by the central processing unit and write the target access request into a request receiving queue corresponding to the identifier of the target main memory; after scheduling the target access request to be dequeued, execute the step of determining the target cache group number corresponding to the target access request based on the identifier of the target main memory corresponding to the target access request.

[0067] The cache memory deploys a request receiving queue corresponding to the identifier of each main memory. After the access request corresponding to each main memory is received, it is written to the corresponding request receiving queue, so as to achieve the purpose of request isolation according to the main memory dimension.

[0068] The process of determining the target cache group number corresponding to the target access request is explained below.

[0069] In one embodiment, such as Figure 4 This diagram illustrates the process of determining the target cache group number using the access module. The access module is specifically used for:

[0070] Step 401: Using a hash algorithm, perform a hash calculation on the target index included in the target access request to obtain the target hash value.

[0071] Optionally, the hash unit supports multiple software-configurable hash algorithms. One of these hash algorithms can be used to perform a hash calculation on the target index.

[0072] Various hash algorithms, including truncated hashing, XOR toeplitz hashing, CRC hashing, etc., are not specifically limited here.

[0073] Step 402: When there are multiple main memories accessible to the central processing unit, determine the target cache group number based on the target hash value and the identifier of the target main memory included in the target access request.

[0074] The amount of main memory can be pre-configured in the cache. Understandably, when the data in the cache is cleared, the amount of main memory can be reconfigured.

[0075] Furthermore, based on the statically configured number of main memories and the identifier of the main memories, the target index can be grouped according to the main memory dimension to form a cache group division corresponding to different main memories.

[0076] The target hash value and the identifier of the target main memory can be concatenated to obtain the target cache group number with a preset bit width.

[0077] In an optional embodiment of this application, the access module is specifically configured to: extract a reference hash value from the target hash value based on the number of main memories accessible to the central processing unit; and concatenate the identifier of the target main memory and the reference hash value to obtain a target cache group number with a preset bit width. The preset bit width is determined based on the depth of the cache memory.

[0078] Specifically, a reference hash value with a first bit width can be extracted from the target hash value, and a reference identifier with a second bit width can be extracted from the identifier of the target main memory. The reference hash value and the reference identifier are then concatenated to obtain the target cache group number with a preset bit width. The sum of the first bit width and the second bit width equals the preset bit width. This method satisfies the depth requirements of the cache memory and allows for cache group partitioning based on the identifier of the main memory.

[0079] In an optional embodiment of this application, the number of main memories accessible to the central processing unit is 2 to the power of Y, and the second bit width is Y. It is understood that if the number of accessible main memories is a single memory location and Y is 0, then the target hash value with the preset bit width is directly used as the target cache group number.

[0080] In one embodiment, the access module is specifically configured to: uniformly reduce the target hash value to obtain candidate hash values ​​when the depth of the cache memory is not a power of 2, and extract a reference hash value from the candidate hash values ​​based on the amount of main memory accessible to the central processing unit.

[0081] By uniformly reducing the target hash value, the probability of each cache group being hit can be made as equal as possible, thereby avoiding the occurrence of hot spots and reducing the collision failure rate.

[0082] In summary, for ease of understanding, a complete embodiment is provided as an example to illustrate the process of determining the target cache group number using the hash unit in the access module. Figure 5The diagram illustrates the processing steps of a hash unit. 1. Set the bit width of the target cache group number output by the hash unit according to the cache depth. Taking a 16-way set-connected hash unit as an example, if the cache depth is 16K, there are 1K sets, and each set has 16 ways. If the cache depth is not a power of 2, the hash result needs to be uniformly reduced. For example, if the cache depth is 5K, which does not satisfy a power of 2, the hash result needs to be processed as follows: hash[31:0] * 5120 / 16 >> 32; where multiplying by 320 can be achieved by shifting, {1'h0,hash,8'h0} + {3'h0,hash,6'h0}. 2. After the target hash value is calculated, it needs to be truncated according to the number of hosts. Combined with the host_id identifier of the target main memory corresponding to the current target access request and the truncated reference hash value, the set number of the group-associative cache is determined to realize the function of host partitioning of the cacheline. For example, taking a group-associative cache depth of 16K as an example, there are 1K sets (10 bits wide), and each set has 16 ways. If the number of hosts is 1, then the target cache group number set[9:0] = hash[9:0]; if the number of hosts is 2, then the target cache group number set[9:0] = {host_id[0], hash[8:0]}; if the number of hosts is 4, then set[9:0] = {host_id[1:0], hash[7:0]}; and so on.

[0083] Therefore, after the access request is processed by the hash unit, it is divided into cache groups according to the host dimension. Consequently, cache lines are also stored according to the host dimension. For example, if the cache depth is 16K and there are 1K sets, such as... Figure 6 This diagram illustrates cache group partitioning when there is a main memory. Figure 7 This diagram illustrates the cache set partitioning when there are two main memories, as shown below. Figure 8 A schematic diagram of cache group partitioning is shown when there are 4 main memories.

[0084] Therefore, this application embodiment supports software-configurable host count, which can automatically adjust the memory resource allocation method according to the actual application scenario, enabling single-host mode to exclusively enjoy memory resources and multi-host mode to equally distribute memory resources. This eliminates the need to redesign the hardware architecture and improves the system's adaptability to diverse scenarios.

[0085] The following describes the process of determining whether a target access request hits a cache line in the target cache group indicated by the target cache group number, and the subsequent processing steps.

[0086] In one embodiment, the determination module is specifically used to: if the target access request is a read request and the target access request does not hit the cache line in the target cache group corresponding to the target cache group number, then transmit the target access request to the response module.

[0087] It is understandable that if the target access request is a read request and a miss occurs, it means that the cache does not store the data that the read request needs to read.

[0088] In one embodiment, the determination module is specifically used to: if the target access request is a write request and the target access request does not hit the cache line in the target cache group corresponding to the target cache group number, then transmit the target access request to the response module.

[0089] Alternatively, in one embodiment, the determination module is specifically used to: if the target access request is a write request and the target access request does not hit the cache line in the target cache group corresponding to the target cache group number, determine the available cache line from the target cache group and write the data corresponding to the write request into the available cache line.

[0090] Furthermore, if an available cache line cannot be determined, the target access request is considered to have missed, and the target access request is then passed to the response module.

[0091] The process of determining available cache lines is illustrated below.

[0092] In one embodiment, the determination module is specifically used to: determine whether there are any free cache lines in the target cache group; if so, then take one of the free first cache lines as an available cache line; if not, then determine an available cache line from the target cache group based on the LRU algorithm.

[0093] In set-associative caches, when a cache hits or needs replacement after a miss, the Least Recently Used (LRU) algorithm is used for replacement. Based on this algorithm, the cache always retains data from the most frequently accessed cache lines, while evicting the oldest idle data. This ensures that the least recently referenced cache line within the same group is always evicted, thereby maximizing the hit rate and mitigating collision failures while keeping implementation complexity under control.

[0094] In one embodiment, the determination module is specifically used to: determine candidate cache lines in the target cache group that store the least recently used data; remove the data in the candidate cache lines and use the candidate cache lines as available cache lines.

[0095] Based on the time-dependent nature of computer architecture, this embodiment uses the LRU algorithm to replace the way in the set. Each way corresponds to a cache line.

[0096] Since the cache groups have been divided by host, LRU replacement can be performed directly based on the granularity of the cache group.

[0097] In one embodiment, the judgment module is specifically used to: maintain the group register corresponding to the target cache group; store the identifier of each cache line in the target cache group in the group register; if there is a free cache line, write the identifier of the first free cache line into the highest bit of the group register, and shift all the cache line identifiers currently stored in the group register to the lower bits by a preset number of bits.

[0098] Optionally, if each cache set contains 2^K ways, then the identifier for each cache line is K bits. The identifier for a cache line can be the ID number of the way. For example, if each cache set has 16 ways, and each way defines a 4-bit ID number, then each set needs to define a 64-bit register A.

[0099] The preset bit depth can be K.

[0100] By writing the identifier of the first cache line containing the data to be written in the latest target access request to the most significant bit of the group register, and shifting all cache line identifiers currently stored in the group register down by a preset number of bits, the register can be updated, ensuring that the identifiers of subsequently updated cache lines can continue to be written to the most significant bit. Alternatively, the write data can simply be written to the first cache line.

[0101] In one embodiment, the determination module is specifically used to: remove the data in the second cache line corresponding to the least significant cache line identifier in the group register when there is no free cache line, and use the second cache line as a candidate cache line.

[0102] It is understandable that the second cache line corresponding to the least recently used cache line ID stores the least recently used data, which means that it is accessed relatively infrequently and therefore can be removed.

[0103] The second cache line is selected as a candidate cache line, and the data to be written is written to the second cache line.

[0104] Optionally, if the data originally stored in the second cache line is marked as dirty, the dirty data is removed from the second cache line and sent to the corresponding main memory for storage to avoid data loss. In the cache, dirty data refers to a cache line that has been modified by the processor but is inconsistent with the content of the next level of memory (such as main memory). To identify this inconsistency, the cache sets a "dirty" bit for each line. When the CPU performs a write operation on that line, it sets the bit to 1, indicating that the data in that line is "dirty." Therefore, when the dirty data is replaced, the entire line content needs to be written back to the next level of memory to ensure the consistency of global system data. Furthermore, the "clean" data that has not been modified and is replaced can be directly discarded without needing to be written back to main memory.

[0105] In summary, for ease of understanding, a complete embodiment is provided to exemplify the process of using the LRU algorithm with the judgment module to replace the way in the set. Figure 9 A schematic diagram of the group register is shown. The LRU algorithm implementation process is as follows: 1. Taking an example where each set has 16 ways, each way is defined with a 4-bit ID number, and each set needs to define a 64-bit register. For example, the ID of way0 is 00002 (4 bits). 2. When writing / reading a set, a free way is selected, and the unique ID number of the corresponding way is written to [63:60] of the 64-bit register A and then shifted right by 4 bits. 3. When writing / reading a set is needed, if there is no free way, the way corresponding to [3:0] of the 64-bit register A is kicked off-chip.

[0106] In this way, dividing cache groups based on the host dimension and using the LRU management and replacement strategy independently can reduce the mutual interference of access requests for different hosts, making it easier to store hot data of each host in the cache, thereby improving the overall cache hit rate of the system and reducing the access pressure on off-chip memory.

[0107] When a target access request fails, it needs to be processed by Cache_dma_ctrl and then forwarded to an off-chip host for access. Access from different hosts needs to be isolated from each other. The process of handling failed access requests is explained below.

[0108] In one embodiment, the response module is specifically configured to: for a target access request scheduled out of the target request scheduling queue, if the target access request is a read request, store the target index included in the target access request into the index queue corresponding to the identifier of the target main memory.

[0109] The cache memory deploys index queues corresponding to the identifiers of different target main memories. These index queues store the indexes in access requests that include the identifier of the target main memory. Thus, when subsequent read data from main memory is returned to the cache memory, the index associated with the read data can be retrieved from the indicated index queue based on the identifier of the target main memory, allowing the read data and the corresponding index to be synchronously returned to the central processing unit.

[0110] Optionally, the index queue can be a FIFO (First In, First Out) queue.

[0111] In one embodiment, the response module is specifically configured to: receive target read data returned by the target main memory in response to the target access request, and determine that the target read data has been received after receiving the read response end signal fed back by the target main memory; after the target read data has been received, store the identifier of the target main memory attached to the target read data into the identifier queue corresponding to the identifier of the target main memory, and store the target read data and the read response end signal into the data queue corresponding to the identifier of the target main memory.

[0112] In other words, the target main memory returns the target read data and the target main memory's identifier to the cache memory, thus marking the data source.

[0113] Considering that the target index corresponding to the target read data needs to be determined before the target read data is sent to the central processing unit, and that if the main memory returns read data corresponding to multiple access requests, the data reception rate is relatively fast, in order to avoid transmission errors, the target read data is stored in the data queue, and the target main memory identifier attached to the target read data is also stored in the identifier queue.

[0114] The end-of-read response signal is the last entry of the read response signal rlast. When the rlast signal is high, it is determined that the end-of-read response signal has been received, which means that the target data has been completely received.

[0115] Optionally, the identifier queue and the data queue can be FIFO queues.

[0116] In this way, the target main memory identifiers can be dequeued sequentially in the identifier queue to determine the target index, and then the target data can be dequeued to send the data and index to the central processing unit, thus ensuring the stability of the data response process.

[0117] In one embodiment, the response module is specifically configured to: schedule the main memory identifiers to be dequeued from the identifier queue corresponding to the identifiers of each target main memory; for the identifiers of the target main memory, schedule the target index corresponding to the identifier of the target main memory to be dequeued from the index queue corresponding to the identifier of the target main memory, and schedule the target read data and read response end signal corresponding to the identifier of the target main memory to be dequeued from the data queue corresponding to the identifier of the target main memory; write the target index and target read data back to the central processing unit, and / or write the target index and target read data into the free cache line in the target cache group.

[0118] The target read data is the latest data from the target main memory. This data has a high probability of being accessed subsequently, so it can be cached in a free cache line for easy access later. The target index is stored synchronously with the target read data to facilitate the subsequent determination of the cache line for storing this target read data.

[0119] In this embodiment, there are index queues, identifier queues, and data queues corresponding to the identifiers of each main memory. For a certain type of queue (e.g., all index queues), when scheduling the dequeueing of information from each queue, the dequeueing of information from each queue is scheduled in a round-robin fashion.

[0120] In summary, for ease of understanding, a complete embodiment is provided to exemplify the process of using the LRU algorithm with the judgment module to replace the way in the set. Figure 10The diagram illustrates the processing flow of the response control module. The response control module, Cache_dma_ctrl, isolates access requests from different main memories; that is, each queue is configured as a MultiHost FIFO (see description below). 1. Missed access requests for different hosts are written to the request scheduling FIFO queue. 2. The corresponding host's FIFO queue is read through the management scheduling selection group. 3. Missed access requests are sent to the corresponding off-chip main memory for processing. 4. The index of the missed access request is stored in the index queue rdindex FIFO indicated by the corresponding main memory identifier. The corresponding index is matched after the off-chip host returns read response data. 5. When the read response signal rlast is high (the last read response), the memory identifier corresponding to the read data is stored in the identifier rlast FIFO queue. 6. The read data and read response signal for each read response are stored in the data queue rdata FIFO according to the host_id. 7a. When the rlast FIFO is not empty and the data read from the rdata FIFO is the last record of the read response, read the data to obtain the memory identifier, and then read the corresponding index in the rdindex FIFO based on this memory identifier. 7b. When the rlast FIFO is not empty and the data read from the rdata FIFO is the last record of the read response, read the data to obtain the memory identifier, and then read the rdata / rlast information corresponding to the rdata FIFO based on this memory identifier. If the read response needs to be returned over multiple clock cycles, it needs to continuously read from the rdata FIFO until rlast in the read data is high (i.e., the last clock cycle of the read response). 8. Return the index in the rdindex FIFO and the rdata in the rdata FIFO together to the central processing unit and write them into a cache line.

[0121] This achieves strict isolation of requests and data between hosts, not only avoiding backpressure transmission, but also preventing anomalies on a single host (such as unauthorized access or data errors) from spreading to other hosts, reducing the risk of system-level failures, enhancing operational reliability, and improving overall system stability.

[0122] Regarding the MultiHost FIFO mechanism, this mechanism allows selection of the corresponding FIFO control logic based on the number of hosts, enabling dedicated memory resources in single-host mode and equal sharing of memory resources in multi-host mode. For example... Figure 11This diagram illustrates the processing steps for setting up a multi-host queue. 1. Select the corresponding FIFO control logic based on the number of hosts. 2. When there is one host, select the 1_host_fifo_ctrl control mode. That is, if there is only a single main memory host0, then one FIFO queue is set up for each type of queue, and data is written to the corresponding FIFO queue. 3. When there is data in host0_fifo, the data is automatically read out. 4. When there are two hosts, select the 2_host_fifo_ctrl control mode. That is, if there are two main memory hosts, host0 and host1, then two FIFO queues are set up for each type of queue, and data is written to the corresponding FIFO queue according to the main memory identifier (host ID). 5. When there is data in the FIFO, a scheduling request is sent to the scheduler, and data is read from each queue in a round-robin fashion. 6. When there are four hosts, select the 4_host_fifo_ctrl control mode, which is similar to the 2_host_fifo_ctrl control mode.

[0123] Optionally, the MultiHost FIFO only requires four FIFO controllers, with one FIFO controller for each host. Optionally, to save resource overhead, FIFO 1, FIFO 2a, and FIFO 4a share one FIFO controller; FIFO 2b and FIFO 4b share one FIFO controller.

[0124] In this way, by replacing the solution of instantiating a cache independently for each host with a time-sharing multiplexing mechanism, the occupation of redundant hardware resources is reduced. While ensuring the concurrent needs of multiple hosts, the chip area and power consumption costs are reduced, thus lowering hardware resource costs.

[0125] In summary, the cache memory provided in this application adopts a set-associative structure, which can receive access requests from multiple hosts in parallel. Asynchronous temporary access requests are scheduled and then divided into sets according to the host dimension using a hash algorithm (ensuring even grouping of the set-associative cache). When an access request hits a cache line, there is no need to access the external host. If the access request is a write request, the cache line is marked as dirty, and the dirty data is written to the external host when it is replaced; if the access request is a read request, the data of the cache line is returned. If the access request misses a cache line, the dma_ctrl module, which isolates the access points between hosts, processes the request before sending it to the external host. Specifically, if the access request is a write request, it is directly written to the external host (write does not occupy space); if it is a read request, the read data from the external host is returned to the cache memory via dma_ctrl, written to the cache line, and the external read data is returned to the CPU. Furthermore, the software can configure the number of hosts, thereby achieving single-host exclusive resource allocation and resource sharing in multi-host mode. Layered isolation between different hosts can ensure that data and requests between hosts do not interfere with each other.

[0126] Specifically, MultiHost Cache, designed for high-concurrency multi-host access scenarios, provides a comprehensive and efficient cache management solution covering access control, address mapping, data scheduling, and response return. 1. Multi-host cache requests enter the corresponding host queue in the access_ctrl module in parallel, achieving high-concurrency request cache_req through asynchronous temporary storage. 2. After scheduling, the requests enter the hash module, where a hash algorithm is used to divide the data into indexes and groups based on the host dimension. Hit / miss checks are performed in tag_ctrl, and aging is carried out using the LRU algorithm based on the host dimension. If the data is hit, it is returned via resp_ctrl; if the data is missed, the cache_req is sent to dma_ctrl. 3. If the cache_req is a write request, it is directly written to the corresponding host; if the cache_req is a read request, it waits for the read data to be returned from the corresponding host before returning via resp_ctrl. For the complete access request processing flow, refer to [link to complete access request processing flow]. Figure 12 As shown in the example.

[0127] Based on the same inventive concept, this application also provides a request processing method, which is executed by the cache memory in the above embodiments. The solution to the problem provided by this method is similar to the solution described in the cache memory above. Therefore, the specific limitations of one or more request processing method embodiments provided below can be found in the limitations of the cache memory above, and will not be repeated here.

[0128] In one embodiment, such asFigure 13 A request processing method is provided, which is used in a cache memory as described in any of the above embodiments, and the method includes:

[0129] Step 1301: Receive the access request sent by the central processing unit.

[0130] Step 1302: Determine the target cache group number corresponding to the access request based on the identifier of the target main memory corresponding to the access request.

[0131] Step 1303: Determine whether the access request hits a cache line in the target cache group indicated by the target cache group number.

[0132] The target cache group is used to cache data corresponding to the target main memory.

[0133] Step 1304: Write the missed access request into the target request scheduling queue corresponding to the identifier of the target main memory, and schedule the access request to be dequeued to send the access request to the target main memory.

[0134] In one embodiment, a computer system is provided, the computer system including a central processing unit, at least one main memory, and a cache memory as in any of the above embodiments.

[0135] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0136] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A high-speed cache memory, characterized in that, The cache memory includes: The access module is used to receive a target access request sent by the central processing unit; and to determine the target cache group number corresponding to the target access request based on the identifier of the target main memory corresponding to the target access request. The judgment module is used to determine whether the target access request hits a cache line in the target cache group indicated by the target cache group number; wherein, the target cache group is used to cache data corresponding to the target main memory; The response module is used to write the unsuccessful target access request into a target request scheduling queue corresponding to the identifier of the target main memory, and schedule the target access request to be dequeued to send the target access request to the target main memory.

2. The cache memory according to claim 1, characterized in that, The access module is specifically used for: Using a hash algorithm, the target index included in the target access request is hashed to obtain the target hash value; When there are multiple main memories accessible to the central processing unit, the target cache group number is determined based on the target hash value and the identifier of the target main memory included in the target access request.

3. The cache memory according to claim 2, characterized in that, The access module is specifically used for: Based on the amount of main memory accessible to the central processing unit, a reference hash value is extracted from the target hash value; The identifier of the target main memory and the reference hash value are concatenated to obtain the target cache group number with a preset bit width; wherein the preset bit width is determined according to the depth of the cache memory.

4. The cache memory according to claim 3, characterized in that, The access module is specifically used for: When the depth of the cache memory is not a power of 2, the target hash value is uniformly reduced to obtain candidate hash values, and the reference hash value is extracted from the candidate hash values ​​according to the amount of main memory accessible to the central processing unit.

5. The cache memory according to claim 1, characterized in that, The access module is specifically used for: The system receives a target access request sent by the central processing unit and writes the target access request into a request receiving queue corresponding to the identifier of the target main memory. After scheduling the target access request to be dequeued, the system executes the step of determining the target cache group number corresponding to the target access request based on the identifier of the target main memory corresponding to the target access request.

6. The cache memory according to any one of claims 1 to 5, characterized in that, The judgment module is specifically used for: If the target access request is a write request, and the target access request does not hit the cache line in the target cache group corresponding to the target cache group number, then the target access request is transmitted to the response module, or an available cache line is determined from the target cache group, and the data corresponding to the write request is written to the available cache line; If the target access request is a read request, and the target access request does not hit the cache line in the target cache group corresponding to the target cache group number, then the target access request is transmitted to the response module.

7. The cache memory according to claim 6, characterized in that, The judgment module is specifically used for: Determine whether there are any free cache lines in the target cache group; If so, then one of the free first cache lines will be designated as the available cache line; If not, the available cache line is determined from the target cache group based on the LRU algorithm.

8. The cache memory according to claim 7, characterized in that, The judgment module is specifically used for: Identify candidate cache lines in the target cache group that store the least recently used data; Remove the data from the candidate cache line and use the candidate cache line as the available cache line.

9. The cache memory according to claim 8, characterized in that, The judgment module is specifically used for: Maintain the group register corresponding to the target cache group; the group register stores the identifier of each cache line in the target cache group; If there is a free cache line, write the identifier of the free first cache line into the highest bit of the group register, and shift all cache line identifiers currently stored in the group register to the lower bits by a preset number of bits; If no free cache line exists, remove the data from the second cache line corresponding to the least significant cache line identifier in the group register, and use the second cache line as the candidate cache line.

10. The cache memory according to any one of claims 1 to 5, characterized in that, The response module is specifically used for: For a target access request that is scheduled out of the target request scheduling queue, if the target access request is a read request, the target index included in the target access request is stored in the index queue corresponding to the identifier of the target main memory.

11. The cache memory according to claim 10, characterized in that, The response module is specifically used for: Receive the target read data returned by the target main memory in response to the target access request, and determine that the target read data has been received when the read response end signal is received from the target main memory; After the target read data is received, the identifier of the target main memory attached to the target read data is stored in the identifier queue corresponding to the identifier of the target main memory, and the target read data and the read response end signal are stored in the data queue corresponding to the identifier of the target main memory.

12. The cache memory according to claim 11, characterized in that, The response module is specifically used for: Dequeue the main memory identifiers from the identifier queue corresponding to the identifiers of each target main memory. For the identifier of the target main memory, according to the identifier of the target main memory, the target index corresponding to the identifier of the target main memory is dequeued from the index queue corresponding to the identifier of the target main memory, and the target read data and the read response end signal corresponding to the identifier of the target main memory are dequeued from the data queue corresponding to the identifier of the target main memory. The target index and the target read data are written back to the central processing unit, and / or the target index and the target read data are written to a free cache line in the target cache group.

13. A request processing method, characterized in that, The method is used in a cache memory as described in any one of claims 1 to 12, the method comprising: Receive target access requests sent by the central processing unit; Based on the identifier of the target main memory corresponding to the target access request, determine the target cache group number corresponding to the target access request; Determine whether the target access request hits a cache line in the target cache group indicated by the target cache group number; wherein, the target cache group is used to cache data corresponding to the target main memory; The unsuccessful target access request is written into the target request scheduling queue corresponding to the identifier of the target main memory, and the target access request is scheduled to be dequeued to send the target access request to the target main memory.

14. A computer system, characterized in that, The computer system includes a central processing unit, at least one main memory, and a cache memory as described in any one of claims 1 to 12.

Images