Cache, cache management method and electronic equipment
Patent Information
- Application Number
- CN202380011855.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-27
- Publication Date
- 2025-05-27
AI Technical Summary
The physical distance between cache and memory is relatively long, resulting in large data transmission delays, affecting cache processing efficiency, especially when the processor frequently updates cached data.
By setting a plurality of cache lines, a first read request queue and a second read request queue in the cache, a first read request exceeding the number of cache lines is sent to the memory controller in advance, and data is stored in the memory controller in advance. When the cache line is idle, data is directly obtained from the memory controller to reduce the waiting time.
It effectively reduces the time to wait for data to be transferred from memory to cache, improves the processing efficiency of cache, and reduces the impact of write first and then read on cache processing efficiency.
Smart Images
Figure CN120051767A_ABST
Abstract
Description
Cache, cache management method and electronic device Technical Field
[0001] The present application relates to the field of chips, and more specifically, to a cache, a cache management method, and an electronic device. Background Art
[0002] In order to improve the processing speed of the processor, a cache is set between the processor and the memory. When the processor needs to read a piece of data, it first searches from the cache. If the data is saved in the cache, it is directly returned to the processor for processing; if the data is not saved in the cache, it is read from the memory and returned to the processor for processing, and the data is saved in the cache at the same time, so that the data can be obtained directly from the cache later without having to call the memory again.
[0003] Because cache storage space is relatively small, processors need to frequently update the data stored in the cache when processing large amounts of data. During this data update, the processor sends a data request to the cache, which then requests the data from the memory. The memory then returns the data to the cache, which then returns it to the processor. Due to the long physical distance between the cache and memory, transferring data from memory to the cache typically requires hundreds of clock cycles. Therefore, the cache spends most of its time waiting for the memory to transfer data, significantly impacting cache processing efficiency.
[0004] Summary of the Invention
[0005] The purpose of the embodiments of the present application is to provide a cache, a cache management method and an electronic device to improve cache processing efficiency.
[0006] In a first aspect, the present application provides a cache, comprising: multiple cache lines, a first read request queue and a second read request queue; the first read request queue is configured to store and send a first read request to a memory controller; the first read request is used to request data from the memory and store the data in the memory controller; the number of first read requests that can be stored in the first read request queue is greater than the number of the multiple cache lines; the second read request queue is configured to store and send a second read request to the memory controller; the second read request corresponds one-to-one to the first read request, and the second read request is used to request the data corresponding to the first read request from the memory controller when the cache line corresponding to the first read request is idle.
[0007] In the embodiment of the present application, by sending a first read request exceeding the number of cache lines to the memory controller in advance, data is stored in the memory controller in advance. When the cache line is free, the data is directly retrieved from the memory controller, reducing the waiting time for data to be transferred from the memory to the cache, thereby improving the processing efficiency of the cache.
[0008] In an optional embodiment, the cache also includes: a third read request queue, configured to store the cache line number corresponding to the first read request, and store the data corresponding to the first read request in the cache line corresponding to the first read request according to the cache line number of the first read request.
[0009] In an optional embodiment, the cache also includes: a cache controller, used to allocate cache lines to requests sent by the processor; the request is a request that does not hit the cache; a first buffer, used to determine whether the cache line corresponding to the request has a write-first-then-read situation; if the cache line corresponding to the request does not have a write-first-then-read situation, generating a first read request and sending it to the first read request queue; if the cache line corresponding to the request has a write-first-then-read situation, saving the request, and determining whether the cache line corresponding to the next request of the request has a write-first-then-read situation.
[0010] In an embodiment of the present application, when a request encounters a write-first-read-later situation, the request is saved in a first buffer zone, and the request will not generate a first read request. The cache can continue to process subsequent requests for the request without waiting for the write-first-read-later situation to be resolved before processing subsequent requests, thereby reducing the impact of the write-first-read-later situation on the cache processing efficiency and further improving the cache processing efficiency.
[0011] In an optional embodiment, the first buffer is further configured to generate a first read request corresponding to the request stored in the first buffer when a write-before-read situation corresponding to the request is resolved.
[0012] In the embodiment of the present application, when a request encounters a write-before-read situation, the request is stored in the first buffer, and subsequent requests to the request are processed by the cache. However, after the read-before-write situation of the request is resolved, a first read request corresponding to the request is generated, so that requests that encounter the read-before-write situation can continue to be processed by the cache.
[0013] In an optional embodiment, the first buffer includes a trigger group with the same number as the cache lines, each trigger corresponds to a cache line, the trigger is used to store requests with a write-before-read situation, and the cache line corresponding to the request is the same as the cache line corresponding to the trigger that stores the request.
[0014] In an optional embodiment, the cache controller is specifically configured to allocate cache lines for a request sent by the processor from cache lines other than cache lines corresponding to a target request; wherein the target request is a request stored in the first buffer.
[0015] In an embodiment of the present application, when the cache controller determines a cache line for a request, it allocates a cache line for the request sent by the processor from a cache line where no write-before-read situation exists, thereby avoiding the occurrence of a new write-before-read situation and further improving the processing efficiency of the cache.
[0016] In an optional embodiment, the cache also includes: a second buffer and a sending queue; the second buffer is used to store the write requests, the number of pending requests and the number of completed requests of each cache line; when the number of pending requests corresponding to a cache line is the same as the number of completed requests corresponding to the cache line, the write request corresponding to the cache line is sent to the sending queue.
[0017] In an embodiment of the present application, the number of pending requests and the number of completed requests of the cache line are judged. When the two are the same, the write request corresponding to the cache line is sent to the send queue to write the data in the cache line back to the memory.
[0018] In an optional embodiment, the second buffer includes the same number of trigger groups as the cache lines, each trigger corresponds to a cache line, and the trigger is used to store the write request, the number of pending requests and the number of completed requests of the corresponding cache line.
[0019] In an optional embodiment, the cache further includes: an arbitration module; the arbitration module is configured to determine the cache line to be processed according to the status of each cache line and a preset rule; and\or
[0020] The cache further includes: a multi-thread queue; the multi-thread queue includes a plurality of threads, each thread corresponds to a cache line, and each thread is used to store pending requests of its corresponding cache line.
[0021] In a second aspect, the present application provides a cache management method, which is applied to the cache of the first aspect mentioned above, wherein the cache includes multiple cache lines, a first read request queue and a second read request queue, and the method includes: the first read request queue sends a first read request to a memory controller; the first read request is used to request data from the memory and store the data in the memory controller; the second read request queue sends a second read request to the memory controller when the cache line corresponding to the first read request is idle; the second read request is used to request the data corresponding to the first read request from the memory controller.
[0022] In an optional embodiment, the cache further includes a third read request queue, and the method further includes: the third read request queue storing the data corresponding to the first read request in the cache line corresponding to the first read request according to the cache line number of the first read request.
[0023] In an optional embodiment, the cache also includes a cache controller and a first buffer, and the method also includes: the cache controller allocates a cache line for the request sent by the processor; the request is a request that does not hit the cache; the first buffer determines whether the cache line corresponding to the request has a write-first-then-read situation; if the cache line corresponding to the request does not have a write-first-then-read situation, a first read request is generated; if the cache line corresponding to the request has a write-first-then-read situation, the request is saved in the first buffer, and it is determined whether the cache line corresponding to the next request of the request has a write-first-then-read situation.
[0024] In an optional implementation, the method further includes: when a write-before-read situation corresponding to a request stored in the first buffer is released, the first buffer generates a first read request corresponding to the request.
[0025] In an optional embodiment, the first buffer includes a trigger group with the same number as the cache lines, each trigger corresponds to a cache line, the trigger is used to store requests with a write-before-read situation, and the cache line corresponding to the request is the same as the cache line corresponding to the trigger that stores the request.
[0026] In an optional embodiment, allocating cache lines for requests sent by the processor includes: the cache controller allocating cache lines for requests sent by the processor from cache lines other than cache lines corresponding to target requests; wherein the target request is a request stored in the first buffer.
[0027] In an optional embodiment, the cache also includes: a second buffer and a sending queue, and the method also includes: the second buffer stores the write requests, the number of pending requests and the number of completed requests for each cache line; when the number of pending requests corresponding to a cache line is the same as the number of completed requests corresponding to the cache line, the write request corresponding to the cache line is sent to the sending queue, so that the sending queue sends the write request to the memory controller.
[0028] In an optional embodiment, the second buffer includes the same number of trigger groups as the cache lines, each trigger corresponds to a cache line, and the trigger is used to store the write request, the number of pending requests and the number of completed requests of the corresponding cache line.
[0029] In an optional embodiment, the cache further includes: an arbitration module; and the method further includes:
[0030] The arbitration module determines the cache line to be processed according to the status of each cache line and a preset rule; and\or
[0031] The cache further includes: a multi-thread queue, the multi-thread queue includes multiple threads, each thread corresponds to a cache line; the method further includes: each thread stores a pending request of its corresponding cache line.
[0032] In a third aspect, the present application provides an electronic device comprising: a processor and the cache described in any one of the first aspects above. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.
[0034] FIG1 is a structural block diagram of an electronic device provided in an embodiment of the present application;
[0035] FIG2 is a schematic diagram of the structure of a cache provided in an embodiment of the present application;
[0036] FIG3 is a schematic structural diagram of a first buffer zone provided in an embodiment of the present application;
[0037] FIG4 is a flowchart of a cache operation according to an embodiment of the present application;
[0038] FIG5 is a flowchart of a cache management method provided in an embodiment of the present application.
[0039] Icons: 100-electronic device; 101-processor; 102-cache; 103-memory controller; 104-memory; 201-first read request queue; 202-second read request queue; 203-third read request queue; 204-cache controller; 205-first buffer; 206-second buffer; 207-sending queue; 208-arbitration module; 209-multi-thread queue; 210-storage unit. DETAILED DESCRIPTION
[0040] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0041] In order to improve the processing efficiency of cache, the present application provides a cache, a cache management method and an electronic device.
[0042] Please refer to Figure 1, which is a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 includes: a processor 101, a cache 102, a memory controller 103 and a memory 104. The memory controller 103 is arranged between the cache 102 and the memory 104. The processor 101 can access the cache 102, and the cache 102 stores the data that the processor 101 needs to process. If the cache 102 does not store the data that the processor 101 needs to process, the cache 102 sends a read request to the memory controller 103, and the memory controller 103 obtains the data that the processor 101 needs to process from the memory 104 according to the read request. After the processor 101 processes the data, the cache 102 sends a write request to the memory controller 103, and the memory controller 103 updates the data in the memory 104 according to the write request.
[0043] The processor 101 has signal processing capabilities and may be a central processing unit (CPU), a graphics processing unit (GPU), a pixel processor (PP), a vertex processor (VP), etc.
[0044] Cache 102 includes multiple cache lines, which are the smallest unit of data transferred between cache 102 and memory controller 103. Each cache line has attributes such as a tag (Tag), an ID (Idx), and a validity (Valid). The Tag identifies the address of the data in the cache line in memory 104, the Idx identifies the location of the cache line in cache 102, and the Valid value indicates whether the data currently stored in the cache line is valid. Only data in cache lines with a valid value can be output by cache 102. When cache 102 receives a request from processor 101, the cache controller generates a Tag based on the requested address and compares it with the Tag of each cache line. If a cache line with an identical Tag exists, this is considered a cache hit, and cache 102 returns the data in the corresponding cache line directly to processor 101. If no cache line with an identical Tag exists, this is considered a cache miss. Cache 102 first requests the corresponding data from memory controller 103, stores the data in the cache line, and then returns the data to processor 101. After the cache 102 is initialized, the default valid attribute of all cache lines is invalid. Each cache line is set to valid after receiving data returned by the memory controller 103.
[0045] The electronic device 100 may be, but is not limited to, a physical device such as a desktop computer, a laptop computer, a smart phone, a smart wearable device, an in-vehicle device, etc. In addition, the electronic device is not necessarily a single device, but may be a combination of multiple devices, such as a server cluster, etc.
[0046] Please refer to FIG. 2 , which is a structural block diagram of a cache provided in an embodiment of the present application. The cache 102 includes a first read request queue 201 and a second read request queue 202 .
[0047] The first read request queue 201 is configured to store first read requests. The first read requests are used to request data from the memory 104 and store the data in the memory controller 103. The number of first read requests that can be stored in the first read request queue 201 is greater than the number of cache lines in the cache.
[0048] The second read request queue 202 is configured to store second read requests. The second read requests correspond one-to-one to the first read requests and are used to request the memory controller 103 for data corresponding to the first read request when the cache line corresponding to the first read request is idle.
[0049] In some embodiments, the first read request queue 201 and the second read request queue 202 may be first-in-first-out queues. This application does not limit the specific implementation of the first read request queue 201 and the second read request queue 202. The first read request queue 201 and the second read request queue 202 may be any hardware capable of storing data.
[0050] During data processing, processor 101 generates a large number of requests, which are sent to cache 102. When a request reaches cache 102 and does not hit a cache line in cache 102, a cache line is allocated for the request, and a first read request and a second read request are generated. The first read request is stored in first read request queue 201, and the second read request is stored in second read request queue 202. When memory controller 103 is ready to receive a first read request, first read request queue 201 sequentially sends the stored first read requests to memory controller 103. After receiving the first read request, memory controller 103 retrieves the corresponding data from memory 104 based on the first read request and stores the retrieved data in its own storage unit.
[0051] When the cache line corresponding to a first read request is idle, the second read request queue 202 sends the second read request corresponding to the first read request to the memory controller 103. After receiving the second read request, the memory controller 103 sends the data corresponding to the first read request stored in its own storage unit to the cache 102, thereby obtaining the data required by the processor 101 from the memory 104 and storing the data in the allocated cache line.
[0052] It should be noted that the cache line being free means that the cache line is in an initialized state or the currently stored data has been processed by the processor 101 and updated to the memory 104 , and the currently stored data in the cache line can be overwritten by other data.
[0053] When processor 101 generates a large number of requests and none of these requests hit a cache line in cache 102, each request is assigned a cache line. Since the number of cache lines in cache 102 is limited, when the number of requests that miss cache 102 is greater than the number of cache lines, one cache line may need to process multiple requests.
[0054] For example, cache 102 is provided with 16 cache lines (cache line 1-cache line 16), and processor 101 sends 32 requests with different tags to cache 102, and none of these 32 requests hits the cache lines in cache 102. Cache 102 assigns requests 1-16 to cache lines 1-16 in sequence, and assigns requests 17-32 to cache lines 1-16 in sequence. Cache line 1 processes the 1st request and the 17th request. Each request generates a first read request and a second read request. For ease of explanation, the first read request generated by the 1st request is first read request 1, and the second read request generated by the 1st request is second read request 1; the first read request generated by the 2nd request is first read request 2, and the second read request generated by the 2nd request is second read request 2, and so on. The first read request queue 201 stores the first read request 1 to the first read request 32. When the memory controller 103 can receive the first read request, the first read request queue 201 sends the first read request 1 to the first read request 32 to the memory controller 103 , and the memory controller 103 requests the data corresponding to the 1st-32nd requests from the memory and stores the data in the storage space of the memory controller 103 .
[0055] Assume that, in the initial state, each cache line is idle. The first request corresponds to cache line 1, and cache line 1 is idle. Then, second read request queue 202 sends second read request 1 to memory controller 103. Memory controller 103 returns the data obtained based on first read request 1 to the cache, and cache line 1 stores the data corresponding to the first request. Subsequently, processor 101 accesses cache line 1 and performs relevant data processing on the data corresponding to the first request. During this process, cache line 1 is occupied.
[0056] The 17th request corresponds to cache line 1. Since cache line 1 is processing the 1st request and cache line 1 is not idle, the second read request queue 202 will not send the second read request 17 to the memory controller 103, and the memory controller 103 will not return the data corresponding to the 17th request, to avoid the data corresponding to the 1st request stored in cache line 1 being overwritten by the data corresponding to the 17th request.
[0057] After the data corresponding to the first request is processed by processor 101, since the 17th request requires the use of cache line 1, second buffer 206 sends a write request to memory controller 103 to update the cache line data corresponding to the data of the first request to memory 104. After that, cache line 1 is idle, and second read request queue 202 sends a second read request 17 to memory controller 103. Memory controller 103 returns the data corresponding to the 17th request, overwriting the data corresponding to the 17th request originally stored in cache line 1 with the data corresponding to the 17th request. Subsequently, processor 101 accesses cache line 1 to perform relevant data processing on the data corresponding to the 17th request.
[0058] The physical distance between cache 102 and memory 104 is large, and the data transmission delay between the two is large, usually reaching hundreds of clock cycles. The physical proximity between memory controller 103 and cache 102 is small, and the data transmission delay between the two is small, usually only dozens of clock cycles. In the embodiment of the present application, data is stored in memory controller 103 in advance by sending a first read request exceeding the number of cache lines to memory controller 103 in advance. When a cache line is idle, data is directly obtained from memory controller 103, reducing the time waiting for data to be transferred from memory 104 to cache 102, thereby improving the processing efficiency of the cache.
[0059] As an optional implementation, the cache 102 further includes a third read request queue 203 .
[0060] The third read request queue 203 is configured to store the cache line number corresponding to the first read request. After the second read request queue 202 sends the second read request, it stores the corresponding cache line number (i.e., cache line idx) in the third read request queue 203. After the MC returns the data corresponding to the first read request, the third read request queue 203 will store the returned data in the cache line corresponding to the first read request according to the cache line number corresponding to the first read request.
[0061] In some embodiments, the first read request queue 201 is further configured to store the cache line IDX of the cache storing the read request data. After the first read request queue 201 sends the first read request to the memory controller 103 , it sends the cache line IDX of the cache 102 storing the data to the third read request queue 203 .
[0062] In some other embodiments, the second read request queue 202 is further configured to store the cache line idx of the cache storing the read request data. After the second read request queue 202 sends the second read request to the memory controller 103 , the cache line idx storing the data in the cache 102 is sent to the third read request queue 203 .
[0063] Furthermore, as an optional implementation, the cache 102 further includes: a cache controller 204 and a first buffer zone 205 .
[0064] The cache controller 204 is configured to allocate cache lines to requests sent by the processor 101 .
[0065] The first buffer 205 is used to determine whether the cache line corresponding to the request has a write-before-read situation; if the cache line corresponding to the request does not have a write-before-read situation, generate a first read request and send it to the first read request queue 201; if the cache line corresponding to the request has a write-before-read situation, save the request, and determine whether the cache line corresponding to the next request of the request has a write-before-read situation.
[0066] In an embodiment of the present application, after the processor 101 sends a request to the cache 102, the cache controller 204 generates a tag based on the address of the request and compares it with the tag of each cache line. If the cache hits, the cache 102 returns the data in the hit cache line to the processor 101, and the processor 101 processes the data.
[0067] If a cache miss occurs, it means that cache 102 does not store the data requested by processor 101, and cache 102 must retrieve the data from memory 104. To retrieve the data, cache controller 204 allocates a cache line for the request based on a preset replacement algorithm. If processor 101 frequently requests cache 102 and there are no free cache lines in cache 102, a write-before-read situation may occur.
[0068] The write-before-read situation means that a cache line in the cache 102 will only send a write request to the memory controller 103 to write the data stored in itself back to the memory 104 when a new request is assigned to the cache line. In order to ensure data consistency, after the memory 104 receives the data lineA sent by the cache line, it will reply with a write completion signal to the cache 102. When a new request requestB requests the cache line where the data lineA is located before the cache 102 receives the write completion signal of the data lineA, then after the cache 102 receives the write completion signal of the data lineA, the cache 102 can send the first read request corresponding to the new request requestB to the memory controller 103 to request data from the memory 104. This phenomenon is called write-before-read. The release of write-before-read means that the write completion signal associated with the write-before-read phenomenon reaches the cache 102, and the cache 102 is allowed to send the first read request corresponding to requestB. Due to the long communication delay between the cache 102 and the memory 104, when the write-before-read situation occurs, the cache 102 needs to wait for a long time before it can continue to work. In addition, cache 102 only has one cache line for data interaction at the same time. Once a write-before-read situation occurs, subsequent requests need to wait until the write-before-read situation is resolved before cache 102 can continue to process subsequent requests, resulting in low cache processing efficiency.
[0069] To address the above-mentioned problem, the embodiment of the present application sets a first buffer 205 in the cache. After the cache controller 204 allocates a cache line for a request, the first buffer 205 determines the cache line corresponding to the request. If the cache line corresponding to the request does not have a write-before-read situation, a first read request is generated and sent to the first read request queue 201. The first read request queue 201 retrieves the corresponding data from the memory 104. If the cache line corresponding to the request has a write-before-read situation, the request is saved in the first buffer 205. Then, it is determined whether the cache line corresponding to the next request of the request has a write-before-read situation. The next request is processed in the same manner as the above-mentioned processing method.
[0070] Through the above method, when a request has a write-before-read situation, the request is saved in the first buffer 205, and the request will not generate a first read request. The cache 102 can continue to process subsequent requests for the request without waiting for the write-before-read situation to be resolved before processing subsequent requests, thereby further improving the processing efficiency of the cache.
[0071] Furthermore, as an optional implementation, the first buffer 205 is further configured to generate a first read request corresponding to the request when the write-before-read situation corresponding to the request stored in the first buffer is released.
[0072] In an embodiment of the present application, when the write-first-read-later situation of the request stored in the first buffer 205 is lifted, the request is removed from the first buffer 205, a first read request is generated based on the request, and the first read request is sent to the first read request queue 201, and the data corresponding to the request is obtained from the memory 104 through the first read request queue 201.
[0073] Furthermore, in some embodiments, the first buffer 205 includes the same number of trigger groups as cache lines, each trigger corresponds to a cache line, the trigger is used to store requests with a write-before-read situation, and the cache line corresponding to the request is the same as the cache line corresponding to the trigger that stores the request.
[0074] As shown in Figure 3, the skid_buf in Figure 3 is a trigger group, and one cache line corresponds to one trigger. If the idx of the cache line allocated by a request is 1, and the cache line has a write-before-read situation, the tag corresponding to the request (i.e., the request tag in Figure 3) is saved in the L1 trigger. skid_buf_vld is used to indicate whether each trigger in the trigger group stores data, so that the subsequent arbitration module can determine the status of each cache line. RAW released means that the trigger group has received the release of the Read After Write (RAW) situation of a cache line. Release RAW's idx indicates the idx corresponding to the cache line that has released the write-before-read situation. Release tag means that after receiving the release of the write-before-read situation of a cache line, the tag stored in the trigger corresponding to the cache line is removed from the trigger.
[0075] Furthermore, as an optional implementation, the cache controller 204 allocates cache lines for the request sent by the processor in the following manner:
[0076] Allocating cache lines for a request sent by the processor from cache lines other than a cache line corresponding to a target request, wherein the target request is a request stored in the first buffer.
[0077] In an embodiment of the present application, if a cache line is written first and read later, in order to improve the processing efficiency of the cache, when allocating cache lines for subsequent requests, cache lines are allocated for subsequent requests from cache lines other than the cache line.
[0078] Specifically, a target queue is provided in the cache controller 204, and the target queue stores cache line information that can be allocated. When allocating a cache line, the cache controller 204 determines a cache line from the target queue according to a preset replacement algorithm. A group of triggers is also provided in the cache controller 204 for storing cache line information in a write-before-read situation. When a cache line has a write-before-read situation, the cache line information will be removed from the target queue and placed in the trigger group. The cache line in the trigger group will not be selected by the replacement algorithm; when the write-before-read situation of the cache line is resolved, the cache line information will be removed from the trigger group and added to the target queue.
[0079] In this way, when the cache controller 204 determines a cache line for a request, it allocates a cache line for the request sent by the processor from a cache line where a write-before-read situation does not exist, thereby avoiding a new write-before-read situation and further improving cache processing efficiency.
[0080] Furthermore, as an optional implementation, the cache 102 further includes: a second buffer 206 and a sending queue 207 .
[0081] The second buffer 206 is used to store the write requests, the number of pending requests and the number of completed requests of each cache line; when the number of pending requests corresponding to a cache line is the same as the number of completed requests corresponding to the cache line, the write request corresponding to the cache line is sent to the sending queue 207.
[0082] The sending queue 207 is used to store write requests to be sent and send the write requests to the memory controller 103. The memory controller 103 writes the data stored in the cache line back to the memory 104 according to the write request.
[0083] Furthermore, as an optional embodiment, the transmit queue 207 is further configured to store write requests that have been sent but have not received a write completion signal. The transmit queue 207 includes both pending write requests and write requests that have been sent but have not received a write completion signal. The first buffer 205 can determine whether a cache line has a write-before-read situation based on all write requests stored in the transmit queue 207. If the cache line corresponding to a newly generated read request is the same as the cache line corresponding to a write request stored in the transmit queue 207, then the cache line is determined to have a write-before-read situation.
[0084] In some embodiments, taking a cache line as an example, during the process of processor 101 requesting cache 102, after each request reaches cache 102, cache 102 determines whether the request hits. If the cache line hits, the number of pending requests for the cache line increases by 1. Each time a request is processed and completed for the cache line, the number of completed requests for the cache line increases by 1. When the number of pending requests and the number of completed requests for the cache line are the same, it means that processor 101 will no longer process the data stored in the cache line and needs to write the current data stored in the cache line back to memory 104. Therefore, the second buffer 206 sends the write request corresponding to the cache line to the send queue 207.
[0085] In other embodiments, when the number of pending requests and the number of completed requests for the cache line are the same and there is a read request for the cache line in the first read request queue 201 or the second read request queue 202, it indicates that the processor 101 has completed processing the data stored in the current cache line and needs to write the current data stored in the cache line back to the memory 104 so that the cache line can be allocated to a subsequent request. The second buffer 206 sends the write request corresponding to the cache line to the send queue 207.
[0086] Furthermore, in some embodiments, the second buffer 206 includes the same number of trigger groups as cache lines, each trigger corresponds to a cache line, and the trigger is used to store the write request, the number of pending requests, and the number of completed requests of the corresponding cache line.
[0087] For example, 16 cache lines are set in cache 102, and the idx of the 16 cache lines are 0-15. 16 triggers are set in second buffer 206, and the 16 triggers are numbered L0-L15. The idx of the cache line corresponding to trigger L0 is 0; the idx of the cache line corresponding to trigger L1 is 1, and so on. Trigger L0 stores the write request corresponding to cache line 0, the number of pending requests, and the number of completed requests. When it is determined that the number of pending requests and the number of completed requests for cache line 0 are the same, trigger L0 sends the write request corresponding to cache line 0 to the send queue 207.
[0088] Furthermore, as an optional implementation, the cache 102 further includes: an arbitration module 208 ; the arbitration module 208 is configured to determine a cache line to be processed according to the status of each cache line and a preset rule.
[0089] In the embodiment of the present application, the state of the cache line is the valid attribute of the cache line.
[0090] Cache 102 sets up a first read request queue 201 and a second read request queue 202 to pre-store data in memory controller 103. When a cache line is free, the data is directly retrieved from memory controller 103. After the cache line retrieves data from memory controller 103, the cache line's valid attribute becomes valid, allowing cache 102 to output the data, and cache 102 can process requests corresponding to the cache line. Therefore, multiple cache lines in cache 102 may have valid valid attributes at the same time.
[0091] Furthermore, due to the provision of first buffer 205, if a request encounters a write-before-read situation, the request will not generate a first read request, and cache 102 will process subsequent requests for the request. After the write-before-read situation is resolved, the request is removed from first buffer 205, a first read request is generated based on the request, and the first read request is sent to first read request queue 201. The data corresponding to the request is retrieved from memory 104 through first read request queue 201. This approach can frequently cause cache 102 to have multiple cache lines with valid attributes simultaneously.
[0092] At the same time, only one cache line of the cache 102 performs data interaction. In the embodiment of the present application, an arbitration module 208 is provided. The arbitration module 208 is configured with preset rules, and the cache line to be processed is determined from multiple cache lines with valid attributes according to the preset rules.
[0093] In some embodiments, the preset rule may be:
[0094] Priority 1: the valid attribute of the cache line is valid and the idx of the cache line is the same as the first idx to be output in the first read request queue 201 .
[0095] Priority 2: the valid attribute of the cache line is valid and the idx of the cache line is the same as the second idx to be output from the first read request queue 201 .
[0096] Priority 3: The valid attribute of the cache line is valid and a write-before-read situation exists in the cache line.
[0097] Priority 4: The valid attribute of the cache line is valid and there is a pending write request for the cache line (ie, the cache line is assigned to a subsequent request).
[0098] Priority 5: The valid attribute of the cache line is valid.
[0099] If there is a cache line that meets priority 1, the arbitration module 208 uses the cache line as the cache line to be processed; if there is no cache line that meets priority 1 but there is a cache line that meets priority 2, the arbitration module 208 uses the cache line as the cache line to be processed, and so on.
[0100] In the above preset rules, since it takes a long time for the cache 102 to request data from the memory 104, the cache line that sends the first read request to the memory 104 is processed first. When the valid attribute of the cache line corresponding to the first two first read requests to be output in the first read request queue 201 is invalid, the cache line with the write-before-read situation is processed first, so that the write-before-read situation is resolved as soon as possible. If there is no cache line with the write-before-read situation or the valid attribute of the cache line with the write-before-read situation is invalid, the cache line corresponding to the write request to be sent is processed first. If there is no cache line corresponding to the write request to be sent, the cache line with a valid attribute is processed.
[0101] In other embodiments, the preset rule may be:
[0102] Priority 1: the valid attribute of the cache line is valid and the idx of the cache line is the same as the first idx to be output in the first read request queue 201 .
[0103] Priority 2: the valid attribute of the cache line is valid and the idx of the cache line is the same as the second idx to be output from the first read request queue 201 .
[0104] Priority 3: the valid attribute of the cache line is valid and the idx of the cache line is the same as the third idx to be output from the first read request queue 201 .
[0105] Priority 4: The valid attribute of the cache line is valid and a write-before-read situation exists in the cache line.
[0106] Priority 5: The valid attribute of the cache line is valid and there is a pending write request for the cache line (that is, the cache line is assigned to a subsequent request).
[0107] Priority 6: The valid attribute of the cache line is valid.
[0108] It should be noted that the embodiments of the present application do not specifically limit the preset rules, and the preset rules can be set according to the actual application scenario of the cache 102. For example, if the actual application scenario of the cache 102 frequently encounters a write-before-read situation, the priority of the condition "the cache line's valid attribute is valid and the cache line encounters a write-before-read situation" can be increased; conversely, if the actual application scenario of the cache 102 rarely encounters a write-before-read situation, the priority of the condition "the cache line's valid attribute is valid and the cache line encounters a write-before-read situation" can be decreased.
[0109] Furthermore, as an optional implementation, the cache 102 further includes: a multi-thread queue 209; the multi-thread queue 209 includes multiple threads, each thread corresponds to a cache line, and each thread is used to store pending requests for its corresponding cache line.
[0110] Multithread queue 209 also includes a multithread controller and random access memory. Multithread queue 209 has the same number of threads as cache lines. When a request sent by processor 101 reaches cache 102, the multithread controller stores the request corresponding to cache line N in thread N. All threads share the random access memory. Requests between different threads are not ordered, but requests within the same thread (i.e., requests corresponding to the same cache line) are strictly ordered.
[0111] After the arbitration module 208 determines the cache line to be processed, the request corresponding to the cache line is output from the multi-thread queue 209 .
[0112] As an optional implementation, the cache 102 further includes a storage unit 210 . The storage unit 210 is divided into a plurality of cache lines for storing data obtained from the memory 104 .
[0113] Please refer to Figure 4, which is a flowchart of the cache workflow provided in an embodiment of the present application.
[0114] The cache 102 obtains a new request from the processor 101 , generates a tag according to the address of the new request, and compares it with the tag of each cache line to determine whether the tag of the new request hits the cache line.
[0115] If a cache line hit is found, the data in the cache line is returned to the processor 101. The processor 101 processes the data returned from the cache line and writes the processed data to the cache line. It then determines whether all requests for the cache line have been completed. If not, the data in the cache line is returned to the processor 101 again, and the above process repeats until all requests for the cache line have been completed. If all requests for the cache line have been completed, the data in the cache line is written back to the memory 104 via the send queue 207.
[0116] If the cache line is not hit, a cache line is allocated for the new request, and it is determined whether the cache line has a write-first-then-read situation. If the write-first-then-read situation exists, the new request is saved in the first buffer 205 until the write-first-then-read situation is resolved. After the write-first-then-read situation is resolved, a first read request is generated based on the request. If the write-first-then-read situation does not exist, a first read request is generated based on the request. The arbitration module 208 then determines the cache line to be processed. After the arbitration module 208 determines the cache line to be processed, it processes the cache line to be processed. The processing process is similar to the processing process after the tag of the aforementioned new request hits the cache line. In order to keep the specification concise, it will not be repeated here.
[0117] Based on the same inventive concept, the present application also provides a cache management method. Refer to Figure 5, which is a flow chart of a cache management method provided in the present application. This cache management method can be applied to the cache 102 in the aforementioned embodiment. The cache management method may include the following:
[0118] S501: A first read request queue sends a first read request to a memory controller.
[0119] S502: When the cache line corresponding to the first read request is idle, the second read request queue sends a second read request to the memory controller.
[0120] In the embodiment of the present application, the first read request is used to request data from the memory and store the data in the memory controller. The second read request is used to request the memory controller for the data corresponding to the first read request.
[0121] In an optional embodiment, the cache further includes a third read request queue, and the method further includes: the third read request queue storing the data corresponding to the first read request in the cache line corresponding to the first read request according to the cache line number of the first read request.
[0122] In an optional embodiment, the cache also includes a cache controller and a first buffer, and the method also includes: the cache controller allocates a cache line for the request sent by the processor; the request is a request that does not hit the cache; the first buffer determines whether the cache line corresponding to the request has a write-first-then-read situation; if the cache line corresponding to the request does not have a write-first-then-read situation, a first read request is generated; if the cache line corresponding to the request has a write-first-then-read situation, the request is saved in the first buffer, and it is determined whether the cache line corresponding to the next request of the request has a write-first-then-read situation.
[0123] In an optional implementation, the method further includes: when a write-before-read situation corresponding to a request stored in the first buffer is released, the first buffer generates a first read request corresponding to the request.
[0124] In an optional embodiment, the first buffer includes a trigger group with the same number as the cache lines, each trigger corresponds to a cache line, the trigger is used to store requests with a write-before-read situation, and the cache line corresponding to the request is the same as the cache line corresponding to the trigger that stores the request.
[0125] In an optional embodiment, allocating cache lines for requests sent by the processor includes: the cache controller allocating cache lines for requests sent by the processor from cache lines other than cache lines corresponding to target requests; wherein the target request is a request stored in the first buffer.
[0126] In an optional embodiment, the cache also includes: a second buffer and a sending queue, and the method also includes: the second buffer stores the write requests, the number of pending requests and the number of completed requests for each cache line; when the number of pending requests corresponding to a cache line is the same as the number of completed requests corresponding to the cache line, the write request corresponding to the cache line is sent to the sending queue, so that the sending queue sends the write request to the memory controller.
[0127] In an optional embodiment, the second buffer includes the same number of trigger groups as the cache lines, each trigger corresponds to a cache line, and the trigger is used to store the write request, the number of pending requests and the number of completed requests of the corresponding cache line.
[0128] In an optional embodiment, the cache further includes: an arbitration module; and the method further includes:
[0129] The arbitration module determines the cache line to be processed according to the status of each cache line and a preset rule; and\or
[0130] The cache further includes: a multi-thread queue, the multi-thread queue includes multiple threads, each thread corresponds to a cache line; the method further includes: each thread stores a pending request of its corresponding cache line.
[0131] It can be understood that the cache management method provided in this application corresponds to the working principle of the aforementioned cache 102. To keep the description concise, the same or similar parts can be referenced to each other and will not be repeated here.
[0132] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0133] In addition, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0134] Furthermore, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0135] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0136] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any actual relationship or order between these entities or operations.
[0137] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, various modifications and variations of the present application are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.
Claims
1. A cache, characterized in that: include: A plurality of cache lines, a first read request queue and a second read request queue; The first read request queue is configured to store and send a first read request to the memory controller; The first read request is used to request data from the memory and store the data in the memory controller; The number of first read requests that can be stored in the first read request queue is greater than the number of the plurality of cache lines; The second read request queue is configured to store and send a second read request to the memory controller; The second read request corresponds to the first read request one by one, and the second read request is used to request the memory controller for data corresponding to the first read request when a cache line corresponding to the first read request is idle.
2. The cache according to claim 1, characterized in that: The cache also includes: The third read request queue is configured to store the cache line number corresponding to the first read request, and store the data corresponding to the first read request in the cache line corresponding to the first read request according to the cache line number of the first read request.
3. The cache according to claim 1, characterized in that: The cache also includes: A cache controller, configured to allocate a cache line for a request sent by a processor; the request is a request that misses the cache; The first buffer is used to determine whether a cache line corresponding to a request has a write-before-read situation; if the cache line corresponding to the request does not have a write-before-read situation, generate a first read request and send it to the first read request queue; if the cache line corresponding to the request has a write-before-read situation, save the request, and determine whether a cache line corresponding to a next request of the request has a write-before-read situation.
4. The cache according to claim 3, characterized in that: The first buffer is further used to generate a first read request corresponding to the request stored in the first buffer when a write-before-read situation corresponding to the request is resolved.
5. The cache according to claim 3, characterized in that: The first buffer includes a trigger group with the same number as the cache lines, each trigger corresponds to a cache line, the trigger is used to store requests with a write-before-read situation, and the cache line corresponding to the request is the same as the cache line corresponding to the trigger storing the request.
6. The cache according to claim 3, characterized in that: The cache controller is specifically used to: allocate cache lines for requests sent by the processor from cache lines other than cache lines corresponding to target requests; wherein the target request is a request stored in the first buffer.
7. The cache according to claim 1, characterized in that: The cache also includes: a second buffer and a sending queue; The second buffer is used to store the write requests, the number of pending requests and the number of completed requests of each cache line; when the number of pending requests corresponding to a cache line is the same as the number of completed requests corresponding to the cache line, the write request corresponding to the cache line is sent to the sending queue.
8. The cache according to claim 7, characterized in that: The second buffer includes a trigger group having the same number as the cache lines, each trigger corresponds to a cache line, and the trigger is used to store write requests, the number of pending requests, and the number of completed requests of the corresponding cache line.
9. The cache according to claim 1, characterized in that: The cache further includes: an arbitration module; the arbitration module is used to determine the cache line to be processed according to the status of each cache line and a preset rule; and\or The cache also includes: a multi-thread queue; the multi-thread queue includes multiple threads, each thread corresponds to a cache line, and each thread is used to store pending requests for its corresponding cache line.
10. A cache management method, characterized in that: Applied to a cache as claimed in any one of claims 1 to 9, the cache comprising a plurality of cache lines, a first read request queue and a second read request queue, the method comprising: The first read request queue sends a first read request to the memory controller; the first read request is used to request data from the memory and store the data in the memory controller; The second read request queue sends a second read request to the memory controller when the cache line corresponding to the first read request is idle; the second read request is used to request the memory controller for data corresponding to the first read request.
11. An electronic device, characterized in that: include: A processor and a cache as claimed in any one of claims 1 to 9.