Cache devices, cache access methods, processors, chips and electronic devices
By introducing a cache unit and an output merging unit into the cache device, the problem that the cache only supports cache line-level memory access requests is solved, enabling memory access request processing at a larger request granularity and improving transmission efficiency.
Patent Information
- Application Number
- CN202411910509.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-23
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-12-23
AI Technical Summary
In existing technologies, caching can only support memory access requests with a maximum granularity of cache lines, resulting in low transmission efficiency in some scenarios.
The cache device introduces a cache unit and an output merging unit. The cache unit is used to send the split sub-requests or sub-request hit status information to the output merging unit. After receiving the sub-requests or hit status information, the output merging unit sends them to the downstream storage device, supporting memory access requests with larger request granularity.
Cache devices can handle memory access requests with larger request granularity, reduce the amount of auxiliary information transmitted by downstream transmission units, and improve transmission efficiency, especially in cross-die or cross-chip access scenarios.
Smart Images

Figure CN119862134B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of processors, and in particular to a cache device, a cache access method, a processor, a chip, and an electronic device. Background Technology
[0002] Cache is an essential component in modern processors. Located between the processor core and main memory, cache is used to compensate for the speed difference between the processor core and main memory.
[0003] In related technologies, the request initiating unit sends a memory access request to the cache. Typically, the granularity of the memory access request is no larger than the cacheline size. If the granularity of the memory access request is larger than the cacheline size, the request initiating unit will further split the original memory access request into multiple sub-requests with a maximum granularity of the cacheline size, and each sub-request will access the cache independently.
[0004] In related technologies, caching can only support memory access requests with a maximum granularity of cache line. Even memory access requests with a larger granularity will be broken down into sub-requests with a smaller granularity. In some scenarios, caching that can only support memory access requests with a maximum granularity of cache line results in low transmission efficiency. Summary of the Invention
[0005] This application provides a caching device, a cache access method, a processor, a chip, and an electronic device, wherein the caching device can support memory access requests with larger request granularity.
[0006] According to one aspect of this application, a cache device is provided, which includes a cache unit and an output merging unit;
[0007] The cache unit is used to send the hit status information of at least two sub-requests or at least two sub-requests to the output merging unit when at least two sub-requests are obtained. The at least two sub-requests are obtained in response to split memory access requests, and the request granularity of the memory access request is larger than the cache line granularity corresponding to the cache unit.
[0008] The output merging unit is used to send the memory access request to the downstream storage device when at least two sub-requests have been received, or to send the memory access request and the hit status information of at least two sub-requests to the downstream storage device when the hit status information of at least two sub-requests has been received.
[0009] According to one aspect of this application, a cache access method is provided, the method being applied to the cache device described above, the method comprising:
[0010] The cache unit determines the hit status information of at least two sub-requests. At least two sub-requests are obtained in response to split memory access requests. The request granularity of the memory access request is larger than the cache line granularity corresponding to the cache unit.
[0011] If the output merging unit receives at least two sub-requests, it sends the memory access request to the downstream storage device; or, if it receives the hit status information of at least two sub-requests, it sends the memory access request and the hit status information of at least two sub-requests to the downstream storage device.
[0012] According to one aspect of this application, a processor is provided, the processor including a cache device for implementing as described above.
[0013] According to one aspect of this application, a chip is provided, the chip including a processor, the processor including the cache device as described above.
[0014] According to one aspect of this application, a graphics card is provided, which includes a cache device as described above.
[0015] According to one aspect of this application, a server is provided, the server including the caching device as described above.
[0016] According to one aspect of this application, an electronic device is provided, comprising: a processor, the processor including the cache device described above. The aforementioned electronic device may be a terminal device. The terminal device may be a mobile phone, computer, intelligent voice interaction device, smart home appliance, vehicle terminal, etc., but is not limited thereto.
[0017] The beneficial effects of the technical solutions provided in this application include at least the following:
[0018] In this application, by incorporating a cache unit and an output merging unit within the cache device, the cache device, in response to a memory access request with a split request granularity larger than the cache line granularity, obtains at least two sub-requests. The cache unit is used to send the split sub-requests or their hit status information to the output merging unit. The output merging unit is used to send the memory access request to the downstream storage device when at least two sub-requests are received, or, when at least two sub-requests' hit status information is received, to send the memory access request and the hit status information of at least two sub-requests to the downstream storage device. This allows the cache device to support memory access requests with larger request granularity. The input to the cache device can be a memory access request with larger request granularity, and thus the output of the cache device can be larger-sized data, rather than only being able to output a maximum of one cache line granularity. For scenarios involving cross-die (the basic unit with full functionality on a chip) or cross-chip access, the larger-sized output results in a smaller amount of other auxiliary information transmitted by the downstream transmission unit (such as PCIe) (other auxiliary information is the auxiliary information required when transmitting the cache output), thereby enabling the downstream transmission unit to have greater transmission efficiency. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a schematic diagram of the structure of a cache device provided in one embodiment of this application.
[0021] Figure 2 This is a schematic diagram illustrating the processing procedure for a first read request provided in one embodiment of this application.
[0022] Figure 3 This is a schematic diagram illustrating the processing procedure for a first read request provided in another embodiment of this application.
[0023] Figure 4 This is a schematic diagram illustrating a method for determining the cache set for a normal mode memory access request, provided in one embodiment of this application.
[0024] Figure 5 This is a schematic diagram illustrating the processing procedure for a first write request provided in one embodiment of this application.
[0025] Figure 6 This is a schematic diagram illustrating the processing procedure for a first write request provided in another embodiment of this application.
[0026] Figure 7This is a schematic diagram illustrating the processing procedure for a second read request provided in one embodiment of this application.
[0027] Figure 8 This is a schematic diagram illustrating the processing procedure for a second read request provided in another embodiment of this application.
[0028] Figure 9 This is a schematic diagram illustrating a method for determining the cache set for memory access requests in bypass mode, provided in one embodiment of this application.
[0029] Figure 10 This is a schematic diagram illustrating the processing procedure for a second write request provided in one embodiment of this application.
[0030] Figure 11 This is a schematic diagram illustrating the processing procedure for a second write request provided in another embodiment of this application.
[0031] Figure 12 This is a structural block diagram of an electronic device provided in one embodiment of this application. Detailed Implementation
[0032] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0033] First, a brief introduction to the terms used in the embodiments of this application will be given.
[0034] Cache: A buffer for data exchange. When the processor core needs to read data, it first looks for the required data in the cache. If the target data is found in the cache, it is executed directly. If the target data is not found in the cache, it is looked for in main memory. The cache is located between the processor core and main memory. The cache operates faster than main memory and is used to compensate for the speed difference between the processor core and main memory, facilitating fast data access and helping the processor core run faster.
[0035] Cacheline size: This refers to the size of a cacheline. Optional cacheline size can be 32 bytes, 64 bytes, 128 bytes, etc. Request granularity refers to the size of data that a memory access request (read or write request) intends to access. In related technologies, caches only support memory access requests with a granularity no larger than one cacheline size. That is, the maximum granularity of a memory access request in related technologies is R (cacheline granularity), measured in bytes. This application will provide cache support for larger request granularities, which can be (1~N)*R bytes.
[0036] Cache Bank: To improve the efficiency of parallel cache access, the cache is divided into multiple cache banks. Each cache bank has its own read / write peripheral circuitry, allowing the processor core to access multiple cache banks in parallel. A cache bank internally includes at least one cache set and at least one cache path.
[0037] Hash operation: In this application, it refers to the operation of mapping the address of a memory access request to the identifier (Bank ID) of the target cache set in multiple cache sets. In this application, performing a hash operation based on the address of the memory access request aims to achieve load balancing across multiple cache sets. When a series of memory access requests access the cache, the hash operation obtains the cache set identifier corresponding to each of these requests, and then allocates them to their respective cache sets. That is, the purpose of the hash operation is to distribute these memory access requests as evenly as possible across multiple cache sets, allowing multiple cache sets to process these requests in parallel. If a hash operation is not performed and the requests are directly allocated to a cache set, it may lead to severe cache set conflicts; that is, only one cache set may be working at the same time, while the performance of other cache sets is wasted.
[0038] Figure 1 This illustration shows a schematic diagram of a caching device provided in an exemplary embodiment of this application. The caching device 100 includes a request splitting unit 101, a caching unit 102, an output merging unit 103, and a read data merging unit 104. The input terminal 1011 of the request splitting unit 101 is connected to the output terminal 1051 of an external request initiation unit 105. The output terminal 1012 of the request splitting unit 101 is connected to the input terminal 1021 of the caching unit 102. Optionally, the caching unit 102 includes at least one cache bank. The request splitting unit 101 is used to split a memory access request into at least two sub-requests, where the request granularity of the memory access request is larger than the cache line granularity corresponding to the caching unit 102. It should be noted that the term "connected" in this application can refer to a direct connection between two hardware units or an indirect connection, i.e., an electrical connection between two hardware units.
[0039] The first output terminal 1022 of the cache unit 102 is connected to the input terminal 1031 of the output merging unit 103. The output terminal 1032 of the output merging unit 103 is used to connect to an external downstream storage device 106, which can be a lower-level cache or main memory. The output merging unit 103 is used to collect the hit status information of at least two sub-requests or at least two sub-requests.
[0040] The first input terminal 1041 of the read data merging unit 104 is connected to the downstream storage device 106, the second input terminal 1042 of the read data merging unit 104 is connected to the second output terminal 1023 of the cache unit 102, and the output terminal 1043 of the read data merging unit 104 is connected to the input terminal 1052 of the request initiation unit 105. The summarized read data is returned to the request initiation unit 105. The read data merging unit 104 is used to merge the read data read from the cache unit 102 and / or the downstream storage device 106.
[0041] In this embodiment, when the cache unit 102 receives at least two sub-requests, it sends the at least two sub-requests or their hit status information to the output merging unit 103. The at least two sub-requests are obtained in response to split memory access requests, and the granularity of the memory access requests is larger than the cache line granularity corresponding to the cache unit. Optionally, the at least two sub-requests are obtained by the cache device 100 splitting memory access requests, or the at least two sub-requests are obtained by a device outside the cache device 100 splitting memory access requests.
[0042] The output merging unit 103, upon receiving at least two sub-requests, sends the memory access request to the downstream storage device 106. It is understood that in this case, the cache device 100 does not respond to the memory access request by performing read / write operations, but instead sends the memory access request to the downstream storage device 106 for response. Alternatively, upon receiving hit status information for at least two sub-requests, the output merging unit 103 sends both the memory access request and the hit status information for at least two sub-requests to the downstream storage device 106. It is understood that in this case, the cache device 100 sends the hit status information to the downstream storage device 106, which then responds to sub-requests in a missing state by performing read / write operations, while the cache device 100 responds to sub-requests in a hit state by performing read / write operations.
[0043] For read requests in normal mode:
[0044] based on Figure 1 The cache device 100 shown is... Figure 2 This illustration shows a schematic diagram of a memory access request processing method provided in an exemplary embodiment of this application. Figure 2 The method for handling the first read request is shown.
[0045] The request initiating unit 105 sends a memory access request to the request splitting unit 101.
[0046] The request splitting unit 101 determines that the received memory access request is a normal mode read request, and the data requested by the memory access request is distributed across M cache lines, where M is a positive integer greater than 1. In other words, the request splitting unit 101 determines that the memory access request is the first read request.
[0047] In one embodiment, the request splitting unit 101 determines that the received read request in normal mode is the first read request based on the starting address and request length. Illustratively, the request splitting unit 101 determines whether the sum of the starting address and request length of the received read request in normal mode exceeds the address range of the cache line corresponding to the starting address. If it does, it indicates that the data requested by the read request is distributed across multiple cache lines, i.e., the read request is the first read request. If it does not exceed the range, the read request is processed according to relevant technologies, and the request granularity of the read request is always smaller than the cache line granularity.
[0048] The request splitting unit 101 splits the first read request into M (positive integers greater than 1) sub-requests, with each of the M sub-requests corresponding to one of the M cache lines. The request splitting unit 101 then sends the M sub-requests to the cache unit 102.
[0049] Reference Figure 2 The caching unit 102 performs a hit test on the M sub-requests, sending the hit or missing status of each sub-request to the output merging unit 103. The caching unit 102 includes at least one cache bank, each cache bank being a group-associative cache (multi-way group-associative cache) organization. Each cache bank includes multiple cache groups (often simply called "groups") and multiple cache paths (often simply called "paths"). The intersection of a cache group and a cache path constitutes a cache line. For each of the M sub-requests, the caching unit 102 performs a hit test on that sub-request, determining whether the caching unit 102 stores the corresponding cache line. If the caching unit 102 stores the corresponding cache line, the sub-request is determined to be in a hit state; if the caching unit 102 does not store the corresponding cache line, the sub-request is determined to be in a missing state.
[0050] The cache unit 102 sends the hit status or missing status of each of the M sub-requests to the output merging unit 103. The output merging unit 103 is used to collect the hit status or missing status of the M sub-requests, and when the hit status or missing status of the M sub-requests is collected, it sends the first read request and the hit status or missing status of the M sub-requests to the downstream storage device 106.
[0051] For example, the output merging unit 103 checks whether the received first read request corresponds to M sub-requests. If so, it waits until the hit status or missing status corresponding to each of the M sub-requests is successfully received.
[0052] When the hit or missing status of M sub-requests is collected, the output merging unit 103 generates a read need data mask. The read need data mask contains M bits, and the i-th bit of the M bits corresponds to the i-th sub-request. The value of each bit indicates whether the corresponding sub-request is in a hit or missing status. For example, a bit value of 1 indicates that the corresponding sub-request is in a hit status, and a bit value of 0 indicates that the corresponding sub-request is in a missing status.
[0053] The output merging unit 103 sends a first read request and a read data mask to the downstream storage device 106.
[0054] Downstream storage device 106 returns the cache line corresponding to the missing sub-request to read data merging unit 104, i.e., the first data, based on the hit or missing status of the M sub-requests. Read data merging unit 104 aggregates the first and second data to obtain merged data, and returns the merged data to request initiation unit 105. Cache unit 102 sends the cache line corresponding to the hit sub-request to read data merging unit 104, i.e., the second data. Read data merging unit 104 aggregates the first and second data to obtain merged data, and returns the merged data to request initiation unit 105.
[0055] Based on the above embodiments, the data requested by the first read request supported by the caching device 100 is distributed across M cache lines, where M is a positive integer greater than 1. This application uses a request splitting unit 101 to split the first read request into M sub-requests corresponding one-to-one with the M cache lines, ensuring that the granularity of each sub-request is no greater than the cache line granularity. That is, this application provides a caching device 100 that supports a larger request granularity, where the granularity of the first read request can be greater than one cache line granularity. For scenarios involving cross-die (the basic unit with complete functionality on a chip) or cross-chip access, a larger output size results in a smaller amount of data for other auxiliary information transmitted by downstream transmission units (such as PCIe), thereby increasing the transmission efficiency of the downstream transmission units.
[0056] based on Figure 2 In the processing method for the first read request shown, optional, Figure 2 The cache unit 102 includes B cache banks, where B is a positive integer of 1 or greater than 1, and optionally, B is four.
[0057] For the case where B is a positive integer greater than 1:
[0058] Reference Figure 3After splitting the request into M sub-requests, the request splitting unit 101 sends the first sub-request (which can be any one of the M sub-requests) to the first cache set 301 of the B cache sets. The first cache set 301 is the cache set corresponding to the first sub-request. The first cache set 301 is used to perform a hit test on the first sub-request. The other sub-requests in the M sub-requests are all sent to their respective corresponding cache sets.
[0059] During the hit test, if the first cache set 301 stores a cache line corresponding to the first sub-request, the first sub-request is determined to be in a hit state; if the first cache set 301 does not store a cache line corresponding to the first sub-request, the first sub-request is determined to be in a missing state. The first cache set 301 sends the hit or missing state of the first sub-request to the output merging unit 103.
[0060] In an optional embodiment, the first cache set 301 is further configured to mark the first cache line corresponding to the first sub-request as locked when the first sub-request is in a hit state; and to mark the first cache line as unlocked when the read data merging unit 104 aggregates the merged data. In the locked state, if other memory access requests attempt to access the first cache line, they must wait for the first cache line to unlock before they can access it. By locking the first cache line beforehand and unlocking it after aggregation, data consistency in the first cache line can be guaranteed (the cached data read is the previously written cached data, not the initialization data in main memory), preventing other memory access requests from accessing the first cache line before it was read.
[0061] In an optional embodiment, the first cache set 301 is further configured to, when the first sub-request is in a missing state and the first sub-request carries an indication to allocate a cache line, backfill the first cache line requested by the first sub-request returned by the downstream storage device 106. Illustratively, if the first sub-request is in a missing state, a choice is made whether to allocate a cache line for it based on the read allocate hint information carried by the first sub-request. If an indication to allocate a cache line exists, the first cache line returned by the downstream storage device 106 is backfilled into the first cache set 301. The backfilling operation refers to the operation of obtaining the first cache line corresponding to the first read request from the downstream storage device 106 and storing the first cache line in the first cache set 301 when the first cache set 301 is missing the first cache line corresponding to the first read request.
[0062] Next, we will introduce how to determine the first cache set 301 corresponding to the first sub-request.
[0063] After request splitting unit 101 splits the request into M sub-requests, it also needs to determine the cache set corresponding to each sub-request. For the first sub-request, request splitting unit 101 is further used to perform a hash operation based on the base address of the first cache line to obtain the identifier of the first cache set 301; based on the identifier of the first cache set 301, the first sub-request is sent to the first cache set 301. Here, the first cache line is the cache line corresponding to the first sub-request. The hash operation is the operation of extracting bits from the base address. For example, the base addresses of the M cache lines correspond sequentially to Addr0, Addr1, ..., Addr(M-1).
[0064] In one embodiment, the request splitting unit 101 is further configured to obtain the identifier of the first cache set 301 based on the first bit sequence. The starting bit of the first bit sequence is determined in the base address of the first cache line by a hash operation based on the first parameter R and the second parameter B. The first parameter R is the cache line size in bytes, and the second parameter B is the number of cache sets in the cache unit 102.
[0065] In one possible implementation, the starting bit of the first bit sequence is sorted from low to high in the base address of the first cache line as a first value. The first value is the sum of the base-2 logarithm of the first parameter R and the base-2 logarithm of the second parameter B minus one. The starting value of the sorting position is zero, that is, the first sorting position is bit 0.
[0066] The end bit of the first bit sequence is determined by a hash operation based on the first parameter R in the base address of the first cache line. In one possible implementation, the sorting position of the end bit of the first bit sequence in the base address of the first cache line from low to high is a second value, which is the base-2 logarithm of the first parameter R, where the starting value of the sorting position is zero, i.e., the first sorting position is bit 0.
[0067] Reference Figure 4 , Figure 4 The base address of the cache line corresponding to different sub-requests is shown. Figure 4 In the case shown, the cache line granularity R is 32, that is, the base address interval between two cache lines is 32 bytes (0x20), and B is the number of cache sets. In this case, the first value is [log2R]+[log2B]-1==6, and the second value is log2R==5. Figure 4 This illustrates a scenario where a read request is split into four sub-requests (i.e., the data requested by the read request is distributed across four cache lines). Figure 4 Sub-requests 0, 1, 2, and 3 in the table are derived from a single read request, while sub-requests 4, 5, 6, and 7 are derived from another read request.
[0068] The hexadecimal base address of sub-request 0 is 10000, and the binary base address is 0001 0000 0000 00000000. The 6th bit (i.e. the 7th bit) from low to high is 0, and the 5th bit (i.e. the 6th bit) is 0. Therefore, the cache set identifier corresponding to sub-request 0 is 00.
[0069] The hexadecimal base address of sub-request 1 is 10020, and the binary base address is 0001 0000 0000 00100000. The 6th bit (i.e. the 7th bit) from low to high is 0, and the 5th bit (i.e. the 6th bit) is 1. Therefore, the cache set identifier corresponding to sub-request 1 is 01, and so on.
[0070] It can be seen that, according to the above method for determining the cache set identifier, every N (optional, N is 4) sub-requests will be evenly distributed across B (optional, B is 4) cache sets. This application provides a cache set balancing scheme for memory access requests with larger request granularity, where the split M sub-requests are evenly distributed across B cache sets.
[0071] The above describes the handling method for the first read request in normal mode, where the data requested by the first read request is distributed across M cache lines, where M is a positive integer greater than 1. For the third read request in normal mode, where the data requested by the third read request is distributed across one cache line, the handling method is as follows:
[0072] For a third read request whose requested data is distributed on a cache line, the request splitting unit 101 sends the third read request to the cache unit 102. The cache unit 102 performs a hit test on the third read request. If the cache unit 102 stores the cache line corresponding to the third read request, it is determined that the third read request is in a hit state; if the cache unit 102 does not store the cache line corresponding to the third read request, it is determined that the third read request is in a missing state.
[0073] The cache unit 102 is used to send the hit or missing status of the third read request to the output merging unit 103. The output merging unit 103 is used to send the missing status of the third read request to the downstream storage device 106 when the third read request is in a missing status. The downstream storage device 106 sends the cache line corresponding to the third read request to the read data merging unit 104, and the read data merging unit 104 returns the cache line corresponding to the third read request to the request initiating unit 105.
[0074] Optionally, cache unit 102 includes B cache sets, where B is a positive integer of 1 or greater than 1. For cases where B is a positive integer greater than 1, request splitting unit 101 sends the third read request to the corresponding cache set. Request splitting unit 101 performs a hash operation based on the starting address of the third read request to obtain the identifier of the corresponding cache set; then, it sends the third read request to the corresponding cache set. This method is similar to the method used above to calculate the identifier of the first cache set corresponding to the first sub-request, and will not be repeated here.
[0075] It is worth noting that, in this application, regardless of whether the granularity of the input memory access request is greater than or less than one cache line, the cache device 100 provided in this application can support the memory access request. Specifically, the request splitting unit 101, the cache unit 102, the output merging unit 103, and the read data merging unit 104 will each provide a certain processing function for the input memory access request.
[0076] For write requests in normal mode:
[0077] based on Figure 1 The cache device 100 shown is... Figure 5 This illustration shows a schematic diagram of a memory access request processing method provided in an exemplary embodiment of this application. Figure 5 The method for handling the first write request is shown.
[0078] The request initiating unit 105 sends a memory access request to the request splitting unit 101.
[0079] The request splitting unit 101 determines that the received memory access request is a write request in normal mode, and the data requested by the memory access request is distributed across P cache lines, where P is a positive integer greater than 1. In other words, the request splitting unit 101 determines that the memory access request is the first write request.
[0080] The request splitting unit 101 determines that the write request is the first write request based on the starting address and request length of the received write request in normal mode. Illustratively, the request splitting unit 101 determines whether the sum of the starting address and request length of the received write request exceeds the address range of the cache line corresponding to the starting address. If it does, it indicates that the data requested by the write request is distributed across multiple cache lines, meaning the write request is the first write request. If it does not exceed the range, the request granularity of the write request is definitely smaller than the cache line granularity.
[0081] The request splitting unit 101 splits the first write request into P (positive integers greater than 1) sub-requests, with each of the P sub-requests corresponding to one of the P cache lines. The request splitting unit 101 then sends the P sub-requests to the cache unit 102.
[0082] The caching unit 102 performs a hit test on P sub-requests, sending the hit or missing status of each sub-request to the output merging unit 103. For a sub-request that is in a hit state, the corresponding cache line is set to invalid, and the cache line is not written.
[0083] By setting the cache line corresponding to a sub-request that is in a hit state to an invalid state and not writing to that cache line, the data consistency of that cache line can be guaranteed (the cached data read is the cached data that was previously written, not the initial data in main memory). In this application, even if a cache line in cache unit 102 is hit, the data of that cache line will not be changed. When the next memory access request arrives, if the memory access request hits that cache line, since the cache line is set to an invalid state in cache unit 102, the next memory access request will access the downstream storage device 106 and perform read and write operations based on the cache line in the downstream storage device 106.
[0084] Reference Figure 6 The cache unit 102 sends P sub-requests to the output merging unit 103. The output merging unit 103, upon receiving all P sub-requests, sends a first write request to the downstream storage device 106. The output merging unit 103 checks whether the received first write request corresponds to P sub-requests; if so, it waits until all P sub-requests are successfully received. Upon receiving all P sub-requests, it sends the first write request to the downstream storage device 106. The downstream storage device 106 sends a first write response corresponding to the first write request to the cache device 100, and the cache device 100 feeds back the first write response to the request initiating unit 105. The cache device 100 does not process the first write response but bypasses it and transmits it to the request initiating unit 105.
[0085] Based on the above embodiments, the data requested by the first write request supported by the caching device 100 is distributed across P cache lines, where P is a positive integer greater than 1. This application uses a request splitting unit 101 to split the first write request into P sub-requests corresponding one-to-one with the P cache lines, ensuring that the granularity of each sub-request is no greater than the cache line granularity. That is, this application provides a caching device 100 that supports a larger request granularity, where the granularity of the first write request can be greater than one cache line granularity. For scenarios involving cross-die (the basic unit with complete functionality on a chip) or cross-chip access, the downstream transmission unit has greater transmission efficiency.
[0086] based on Figure 5 The processing method for the first write request shown is optional. Figure 5 The cache unit 102 includes B cache banks, where B is a positive integer of 1 or greater than 1, and optionally, B is four.
[0087] For the case where B is a positive integer greater than 1:
[0088] Reference Figure 6 After splitting the request into P sub-requests, the request splitting unit 101 sends the second sub-request (which is any one of the P sub-requests) to the second cache set 601 (one of the B cache sets). The second cache set 601 is the cache set corresponding to the second sub-request. The second cache set 601 is used to perform a hit test on the second sub-request. The second cache set 601 is also used to send the second sub-request to the output merging unit 103. The other sub-requests in the P sub-requests are all sent to their respective corresponding cache sets.
[0089] Next, we will introduce how to determine the second cache set 601 corresponding to the second sub-request.
[0090] After splitting the request into P sub-requests, the request splitting unit 101 needs to determine the cache set corresponding to each sub-request. For the second sub-request, the request splitting unit 101 is also used to perform a hash operation based on the base address of the second cache line to obtain the identifier of the second cache set; based on the identifier of the second cache set, the second sub-request is sent to the second cache set. Here, the second cache line is the cache line requested by the second sub-request. The hash operation is the operation of extracting bits from the base address. For example, the base addresses of the P cache lines correspond sequentially to Addr0, Addr1, ..., Addr(P-1).
[0091] In one embodiment, the request splitting unit is further configured to obtain the identifier of the second cache set based on the second bit sequence; wherein the starting bit of the second bit sequence is determined in the base address of the second cache line by a hash operation based on the first parameter R and the second parameter B, the first parameter R is the cache line size in bytes, and the second parameter B is the number of cache sets in the cache unit 102.
[0092] In one possible implementation, the starting bit of the second bit sequence is sorted from low to high in the base address of the second cache line as a first value. The first value is the sum of the base-2 logarithm of the first parameter R and the base-2 logarithm of the second parameter B minus one. The starting value of the sorting position is zero, that is, the first sorting position is bit 0.
[0093] The end bit of the second bit sequence is determined by a hash operation on the first parameter R in the base address of the second cache line. In one possible implementation, the sorting position of the end bit of the second bit sequence in the base address of the second cache line from low to high is a second value, which is the base-2 logarithm of the first parameter R, where the starting value of the sorting position is zero, i.e., the first sorting position is bit 0.
[0094] Reference Figure 4 Similar to the case of a first read request, Figure 4 It can also be viewed as the process of calculating the cache set identifiers corresponding to the multiple base addresses split from the first write request.
[0095] Figure 4 The base addresses of the different sub-requests are shown. Figure 4 In the case shown, the cache line granularity R is 32, that is, the base address interval between the two sub-requests is 32 bytes (0x20), and B is the number of cache sets. In this case, the first value is [log2R]+[log2B]-1==6, and the second value is log2R==5. Figure 4 This illustrates a scenario where a write request is split into four sub-requests (i.e., the data requested by the write request is distributed across four cache lines). Figure 4 Sub-requests 0, 1, 2, and 3 are derived from a single write request, while sub-requests 4, 5, 6, and 7 are derived from another write request.
[0096] The hexadecimal base address of sub-request 0 is 10000, and the binary base address is 0001 0000 0000 00000000. The 6th bit (i.e. the 7th bit) from low to high is 0, and the 5th bit (i.e. the 6th bit) is 0. Therefore, the cache set identifier corresponding to sub-request 0 is 00.
[0097] The hexadecimal base address of sub-request 1 is 10020, and the binary base address is 0001 0000 0000 00100000. Counting from the lowest to the highest bit (i.e., the 7th bit), the 6th bit is 0, and the 5th bit (i.e., the 6th bit) is 1. Therefore, the cache set identifier corresponding to sub-request 1 is 01, and so on. It can be seen that, according to the above method of determining the cache set identifier, every N (optional, N is 4) sub-requests will be evenly distributed across B (optional, B is 4) cache sets. This application provides a cache set balancing scheme for memory access requests with larger request granularity, where the P sub-requests are evenly distributed across B cache sets.
[0098] The above describes the handling method for the first write request in normal mode, where the data requested by the first write request is distributed across M cache lines, where M is a positive integer greater than 1. For the third write request in normal mode, where the data requested by the third write request is distributed across one cache line, the handling method is as follows:
[0099] For a third write request whose requested data is distributed across a single cache line, request splitting unit 101 sends the third write request to cache unit 102. Cache unit 102 performs a hit test on the third write request. If cache unit 102 stores the cache line corresponding to the third write request, the third write request is determined to be in a hit state, and the cache line corresponding to the third write request is processed according to the write hit policy of the third write request. If cache unit 102 does not store the cache line corresponding to the third write request, the third write request is determined to be in a missing state, and cache unit 102 processes the cache line corresponding to the third write request according to the write miss policy of the third write request.
[0100] The cache unit 102 is used to send the third write request and its hit or miss status to the output merging unit 103. The output merging unit 103 is used to send the third write request and its hit or miss status to the downstream storage device 106. The downstream storage device 106 processes the cache line corresponding to the third write request according to the write hit policy of the third write request. Alternatively, the downstream storage device 106 processes the cache line corresponding to the third write request according to the write miss policy of the third write request.
[0101] Optionally, cache unit 102 includes B cache sets, where B is a positive integer of 1 or greater than 1. For cases where B is a positive integer greater than 1, request splitting unit 101 sends the third write request to the corresponding cache set. Request splitting unit 101 performs a hash operation based on the starting address of the third write request to obtain the identifier of the corresponding cache set; then, it sends the third write request to the corresponding cache set. This process is similar to calculating the identifier of the second cache set corresponding to the second sub-request described above, and will not be repeated here.
[0102] It is worth noting that in this application, the cache device 100 provided in this application can support both input memory access requests with a request granularity greater than or equal to one cache line. Specifically, the request splitting unit 101, cache unit 102, output merging unit 103, and read data merging unit 104 will each provide certain processing functions for the input memory access requests.
[0103] For read requests in bypass mode:
[0104] based on Figure 1 The cache device 100 shown is... Figure 1This illustration shows a schematic diagram of a memory access request processing method provided in an exemplary embodiment of this application. Figure 7 The method for handling the second read request is shown.
[0105] The request initiating unit 105 sends a memory access request to the request splitting unit 101.
[0106] The request splitting unit 101 determines that the received memory access request is a second read request, which is a bypass request.
[0107] The request splitting unit 101 bypasses and transmits the second read request to the cache unit 102; the cache unit 102 is used to bypass and transmit the second read request to the output merging unit 103; the output merging unit 103 is used to bypass and transmit the second read request to the downstream storage device 106; the read data merging unit 104 is used to bypass and transmit the data returned by the downstream storage device 106 to the request initiating unit 105.
[0108] Bypass transmission refers to a transmission method that passes data directly without processing it.
[0109] In this application, for read requests in bypass mode, the request is not split, but is transmitted to the downstream storage device 106 in a bypass transmission manner to read the cached data in the downstream storage device 106. At this time, for requests in bypass mode, it is necessary to ensure the data consistency of requests in bypass mode in advance, that is, to ensure that requests in bypass mode do not have duplicate addresses with requests in normal mode.
[0110] In the above embodiments, the request granularity of the second read request in the bypass mode can be (1~N)*R bytes. In the above embodiments, the second read request in the bypass mode is sent to the downstream storage device 106 through bypass to read the cached data in the downstream storage device 106. At this time, it can be considered that the cache device 100 provided by this application can support a larger request granularity. The cache device 100 supports processing read requests with a request granularity greater than 1 cache line granularity.
[0111] based on Figure 7 The method for processing the second read request shown is optional. Figure 7 The intermediate cache unit 102 includes B cache banks, where B is a positive integer of 1 or greater than 1, and optionally, B is four.
[0112] For the case where B is a positive integer greater than 1:
[0113] Reference Figure 8The request splitting unit 101 is also used to bypass the transmission of the second read request to the third cache set 801, which is a cache set corresponding to the second read request; the third cache set 801 is also used to bypass the transmission of the second read request to the output merging unit 103.
[0114] Next, we will introduce how to determine the third cache set 801 corresponding to the second read request.
[0115] After receiving the second read request, the request splitting unit 101 determines the identifier of the cache set (i.e., the third cache set 801) corresponding to the second read request based on the starting address (memory access address) of the second read request, and bypasses the second read request to the third cache set 801. The third cache set 801 then bypasses the second read request to the output merging unit 103.
[0116] In one embodiment, the request splitting unit 101 is further configured to perform a hash operation based on the starting address of the second read request to obtain the identifier of the third cache set; and based on the identifier of the third cache set, to bypass and transmit the second read request to the third cache set. A hash operation refers to the operation of retrieving bits from the starting address.
[0117] In one embodiment, the request splitting unit 101 is further configured to perform a bit sequence XOR operation based on the third bit sequence and the fourth bit sequence to obtain the identifier of the third cache set;
[0118] The bit sequence XOR operation refers to the XOR operation performed between the i-th third bit in the third bit sequence and the i-th fourth bit in the fourth bit sequence, where i is a positive integer. In the bit sequence XOR operation, every third bit in the third bit sequence and every fourth bit in the fourth bit sequence participates.
[0119] The starting bit of the third bit sequence is determined by a hash operation based on the first parameter R and the second parameter B in the starting address of the second read request. The first parameter R is the cache line granularity in bytes, and B is the number of cache sets. In one possible implementation, the starting bit of the third bit sequence is the first value in the ascending order of the starting address of the second read request. The first value is the sum of the base-2 logarithm of the first parameter R and the base-2 logarithm of the second parameter B, minus one. The initial value of the sorting position is zero, that is, the first sorting position is bit 0.
[0120] The end bit of the third bit sequence is determined by a hash operation based on the first parameter R in the starting address of the second read request. In one possible implementation, the end bit of the third bit sequence is sorted from low to high in the starting address of the second read request as a second value, which is the base-2 logarithm of the first parameter R, where the starting value of the sorting position is zero, i.e., the first sorting position is bit 0.
[0121] The starting bit of the fourth bit sequence is determined by a hash operation based on the first parameter R, the second parameter B, and the third parameter N in the starting address of the second read request. The third parameter N is the ratio of the largest request granularity to the cache line granularity. In one possible implementation, the starting bit of the fourth bit sequence is the third value in the ascending order of the starting address of the second read request. The third value is the sum of the base-2 logarithm of the first parameter R, twice the base-2 logarithm of the second parameter B, and the base-2 logarithm of the third parameter N, minus one. The initial value of the sorting position is zero, i.e., the first sorting position is bit 0.
[0122] The end bit of the fourth bit sequence is determined by a hash operation on the first parameter R, the second parameter B, and the third parameter N in the starting address of the second read request. In one possible implementation, the end bit of the fourth bit sequence is the fourth value in the ascending order of the starting address of the second read request. The fourth value is the sum of the base-2 logarithms of the first parameter R, the base-2 logarithms of the second parameter B, and the base-2 logarithms of the third parameter N. The starting value of the sorting position is zero, that is, the first sorting position is bit 0.
[0123] Reference Figure 9 , Figure 9 The starting addresses of different bypass requests are shown.
[0124] Figure 9 In the illustrated case, the cache line granularity R is 32, meaning the starting address interval between the two bypass requests is N*R = 128 bytes (0x80), B is the number of cache sets (four), and N is four. In this case, the first value is log2(R*B) - 1 = 6, the second value is log2(R) = 5, the third value is log2(R*B*N) + log2(B) - 1 = 10, and the fourth value is log2(R*B*N) = 9.
[0125] Figure 9 The example shows 16 bypass requests, with the starting address interval between two bypass requests being 0x80.
[0126] Taking bypass request 0 as an example, the hexadecimal address of bypass request 0 is 10000, and the binary address is 0001 00000000 0000 0000. Counting from low to high, the 10th bit is 0, the 9th bit is 0, the 6th bit is 0, and the 5th bit is 0. The XOR result of the 10th bit and the 6th bit is 0, and the XOR result of the 9th bit and the 5th bit is 0. Therefore, the identifier of the cache set corresponding to bypass request 0 is 00.
[0127] Similarly, for bypass request 1 (hexadecimal address 10080), bypass request 2 (hexadecimal address 10100), and bypass request 3 (hexadecimal address 10180), based on a similar XOR method, the cache set identifier can be obtained as 00.
[0128] Taking bypass request 4 as an example again, the hexadecimal address of bypass request 4 is 10200, and the binary address is 00010000 0010 0000 0000. Counting from the low to the high, the 10th bit is 0, the 9th bit is 1, the 6th bit is 0, the 5th bit is 0, the XOR result of the 10th bit and the 6th bit is 0, and the XOR result of the 9th bit and the 5th bit is 1. Therefore, the identifier of the cache set corresponding to bypass request 4 is 01.
[0129] And so on, without further explanation.
[0130] It can be observed that, through the cache set identifier determination method provided in this application, when the request granularity is N*R bytes, in the contiguous address mode, every N (N is the ratio of the maximum request granularity to the cache line granularity, optional, N is 4) bypass requests fall on the same cache set, and the corresponding cache set will be changed every N bypass requests. Therefore, this application provides a cache set balancing method for bypass requests, achieving the effect of balancing between cache sets.
[0131] Write requests for bypass mode:
[0132] based on Figure 1 The cache device 100 shown is... Figure 10 This illustration shows a schematic diagram of a memory access request processing method provided in an exemplary embodiment of this application. Figure 10 The method for handling the second write request is shown.
[0133] The request initiating unit 105 sends a memory access request to the request splitting unit 101.
[0134] The request splitting unit 101 determines that the received memory access request is a second write request, which is a bypass request.
[0135] The request splitting unit 101 bypasses and transmits the second write request to the cache unit 102; the cache unit 102 is used to bypass and transmit the second write request to the output merging unit 103; the output merging unit 103 is used to bypass and transmit the second write request to the downstream storage device 106. The downstream storage device 106 returns the second write response corresponding to the second write request to the cache device 100, and the cache device 100 bypasses and transmits the second write response to the request initiating unit 105. The cache device 100 does not process the second write response and bypasses and transmits the second write response to the request initiating unit 105.
[0136] Bypass transmission refers to a transmission method that passes data directly without processing it.
[0137] In this application, write requests in bypass mode are not split, but are transmitted to the downstream storage device 106 in a bypass transmission manner to perform write operations on the cached data in the downstream storage device 106. At this time, for requests in bypass mode, it is necessary to ensure the data consistency of requests in bypass mode in advance, that is, to ensure that requests in bypass mode do not have duplicate addresses with normal requests.
[0138] In the above embodiments, the request granularity of the second write request in the bypass mode is (1~N)*R bytes. In the above embodiments, the second write request in the bypass mode is sent to the downstream storage device 106 through bypass to write the cached data in the downstream storage device 106. At this time, it can be considered that the cache device 100 provided by this application can support a larger request granularity. The cache device 100 supports processing write requests with a request granularity greater than 1 cache line granularity.
[0139] based on Figure 10 The processing method for the second write request shown is optional. Figure 10 The intermediate cache unit 102 includes B cache banks, where B is a positive integer of 1 or greater than 1, and optionally, B is four.
[0140] For the case where B is a positive integer greater than 1:
[0141] Reference Figure 11 The request splitting unit 101 is also used to bypass the transmission of the second write request to the fourth cache set 1101, which is a cache set corresponding to the second write request; the fourth cache set 1101 is also used to bypass the transmission of the second write request to the output merging unit 103.
[0142] Next, we will introduce how to determine the fourth cache set 1101 corresponding to the second write request.
[0143] The request splitting unit 101 is also used to perform a hash operation based on the starting address of the second write request to obtain the identifier of the fourth cache set 1101; and to bypass the second write request to the fourth cache set 1101 based on the identifier of the fourth cache set 1101.
[0144] In one embodiment, the request splitting unit 101 is further configured to perform a bit sequence XOR operation based on the fifth bit sequence and the sixth bit sequence to obtain the identifier of the fourth cache set;
[0145] The bit sequence XOR operation refers to the operation of performing an XOR calculation between the i-th fifth bit in the fifth bit sequence and the i-th sixth bit in the sixth bit sequence, where i is a positive integer; in the bit sequence XOR operation, each fifth bit in the fifth bit sequence and each sixth bit in the sixth bit sequence participate.
[0146] The starting bit of the fifth bit sequence is determined by a hash operation based on the first parameter R and the second parameter B in the starting address of the second write request. The first parameter R is the cache line granularity, measured in bytes. In one possible implementation, the starting bit of the fifth bit sequence is the first value, ordered from low to high in the starting address of the second write request. The first value is the sum of the base-2 logarithm of the first parameter R and the base-2 logarithm of the second parameter B, minus one. The initial value of the sorting position is zero, i.e., the first sorting position is bit 0.
[0147] The end bit of the fifth bit sequence is determined by a hash operation based on the first parameter R in the starting address of the second write request. In one possible implementation, the end bit of the fifth bit sequence is a second value in the ascending order of the starting address of the second write request. The second value is the base-2 logarithm of the first parameter R, where the starting value of the sorting position is zero, i.e., the first sorting position is bit 0.
[0148] The starting bit of the sixth bit sequence is determined by a hash operation based on the first parameter R, the second parameter B, and the third parameter N in the starting address of the second write request. The third parameter N is the ratio of the largest request granularity to the cache line granularity. In one possible implementation, the starting bit of the sixth bit sequence is the third value in the ascending order of the starting address of the second write request. The third value is the sum of the base-2 logarithm of the first parameter R, twice the base-2 logarithm of the second parameter B, and the base-2 logarithm of the third parameter N, minus one. The initial value of the sorting position is zero, i.e., the first sorting position is bit 0.
[0149] The end bit of the sixth bit sequence is determined by a hash operation based on the first parameter R, the second parameter B, and the third parameter N in the starting address of the second write request. In one possible implementation, the end bit of the sixth bit sequence is the fourth value in the starting address of the second write request, which is the sum of the base-2 logarithm of the first parameter R, the base-2 logarithm of the second parameter B, and the base-2 logarithm of the third parameter N. The starting value of the sorting position is zero, that is, the first sorting position is bit 0.
[0150] Reference Figure 9 Similar to the case of a second read request, Figure 9 It can also be viewed as the process of calculating the cache set identifier corresponding to the second write request based on the starting address of the second write request.
[0151] Reference Figure 9 , Figure 9 The starting addresses of different bypass requests are shown.
[0152] Figure 9 In the illustrated case, the cache line granularity R is 32, meaning the starting address interval between the two bypass requests is N*R = 128 bytes (0x80), B is the number of cache sets (four), and N is four. In this case, the first value is log2(R*B) - 1 = 6, the second value is log2(R) = 5, the third value is log2(R*B*N) + log2(B) - 1 = 10, and the fourth value is log2(R*B*N) = 9.
[0153] Figure 9 The example shows 16 bypass requests, with the starting address interval between two bypass requests being 0x80.
[0154] Taking bypass request 0 as an example, the hexadecimal address of bypass request 0 is 10000, and the binary address is 0001 00000000 0000 0000. Counting from low to high, the 10th bit is 0, the 9th bit is 0, the 6th bit is 0, and the 5th bit is 0. The XOR result of the 10th bit and the 6th bit is 0, and the XOR result of the 9th bit and the 5th bit is 0. Therefore, the identifier of the cache set corresponding to bypass request 0 is 00.
[0155] Similarly, for bypass request 1 (hexadecimal address 10080), bypass request 2 (hexadecimal address 10100), and bypass request 3 (hexadecimal address 10180), based on a similar XOR method, the cache set identifier can be obtained as 00.
[0156] Taking bypass request 4 as an example again, the hexadecimal address of bypass request 4 is 10200, and the binary address is 00010000 0010 0000 0000. Counting from the low to the high, the 10th bit is 0, the 9th bit is 1, the 6th bit is 0, the 5th bit is 0, the XOR result of the 10th bit and the 6th bit is 0, and the XOR result of the 9th bit and the 5th bit is 1. Therefore, the identifier of the cache set corresponding to bypass request 4 is 01.
[0157] And so on, without further explanation.
[0158] It can be observed that, through the cache set identifier determination method provided in this application, when the request granularity is N*R bytes, in the contiguous address mode, every N (N is the ratio of the maximum request granularity to the cache line granularity, optional, N is 4) bypass requests fall on the same cache set, and the corresponding cache set will be changed every N bypass requests. Therefore, this application provides a cache set balancing method for bypass requests, achieving the effect of balancing between cache sets.
[0159] The above section introduced four methods for handling memory access requests: read requests in normal mode, write requests in normal mode, read requests in bypass mode, and write requests in bypass mode. The caching device provided in this application supports handling at least one of the above four types of memory access requests; for example, it supports all four types of memory access requests.
[0160] In one embodiment, the request initiating unit 105 adds a request type indication to the memory access request. For example, a bit value of 1 indicates that the memory access request is a bypass request (a request in bypass mode), and a bit value of 0 indicates that the memory access request is a normal request. The request splitting unit 101 receives the memory access request and determines the type of the memory access request based on the indication.
[0161] Figure 12 A structural block diagram of an electronic device 1200 provided in an exemplary embodiment of this application is shown. The electronic device 1200 may be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The electronic device 1200 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.
[0162] Typically, electronic device 1200 includes a processor 1201 and a memory 1202.
[0163] Processor 1201 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1201 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1201 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1201 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1201 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.
[0164] The processor 1201 includes the cache device provided in the above embodiments.
[0165] The memory 1202 may include one or more computer-readable storage media, which may be non-transitory. The memory 1202 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices.
[0166] In some embodiments, the computer device 1200 may also optionally include: a peripheral device interface 1203 and at least one peripheral device. Those skilled in the art will understand that... Figure 12 The structure shown does not constitute a limitation on the computer device 1200, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.
[0167] This application also provides a processor that includes a cache device as described in the embodiments above.
[0168] This application also provides a chip, which includes a processor, and the processor includes a cache device as described in the above embodiments.
[0169] This application also provides a graphics card that includes a cache device as described in the above embodiments.
[0170] This application also provides a server that includes a caching device as described in the embodiments above.
[0171] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0172] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0173] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A cache device, characterized in that, The caching device includes a request splitting unit, a caching unit, and an output merging unit; The caching unit is used to send the at least two sub-requests or the hit status information of the at least two sub-requests to the output merging unit when at least two sub-requests are obtained. The at least two sub-requests are obtained by splitting the memory access request in response to the prompt information of the request splitting unit that the memory access request carries a normal request. The request granularity of the memory access request is larger than the cache line granularity corresponding to the caching unit. The output merging unit is configured to send the memory access request to the downstream storage device when the at least two sub-requests are received, or to send the memory access request and the hit status information of the at least two sub-requests to the downstream storage device when the hit status information of the at least two sub-requests is received. The caching unit is used to bypass the transmission of requests in bypass mode to the output merging unit. The output merging unit is used to bypass the request of the bypass mode to the downstream storage device. The bypass mode request carries a bypass request prompt message so that the request splitting unit can identify the bypass mode request and transmit the bypass mode request to the cache unit.
2. The device according to claim 1, characterized in that, The request splitting unit is used to obtain the memory access request sent by the external request initiating unit, split the memory access request into at least two sub-requests, and send the at least two sub-requests to the cache unit.
3. The device according to claim 1, characterized in that, The cache device also includes a read data merging unit; The read data merging unit is used to merge read data read from the cache unit and / or the downstream storage device, and output the merged read data to an external request initiation unit.
4. The device according to any one of claims 1 to 3, characterized in that, The cache device also includes a read data merging unit; The request splitting unit is used to receive a first read request sent by the request initiating unit, wherein the data requested by the first read request is distributed across M cache lines, where M is a positive integer greater than 1; split the first read request into M sub-requests, wherein the M sub-requests correspond one-to-one with the M cache lines; and send the M sub-requests to the cache unit. The caching unit is used to perform a hit test on the M sub-requests and send the hit status or missing status of each sub-request in the M sub-requests to the output merging unit. The output merging unit is used to collect the hit status or missing status of the M sub-requests and send the first read request and the hit status or missing status of the M sub-requests to the downstream storage device. The read data merging unit is used to summarize the first data and the second data to obtain merged data, and return the merged data to the request initiation unit. The first data includes the cache line corresponding to the missing state among the M sub-requests returned by the downstream storage device, and the second data includes the cache line corresponding to the hit state among the M sub-requests.
5. The device according to claim 2, characterized in that, The memory access request includes a first read request, and the at least two sub-requests include a first sub-request, wherein the first sub-request is any one of the at least two sub-requests; the cache unit includes B cache sets, where B is a positive integer greater than 1, and the B cache sets include a first cache set; The request splitting unit is further configured to perform a hash operation based on the base address of the first cache line to obtain the identifier of the first cache set; Based on the identifier of the first cache set, the first sub-request is sent to the first cache set; The first cache line is the cache line requested by the first sub-request.
6. The device according to claim 5, characterized in that, The request splitting unit is also used to obtain the identifier of the first cache set based on the first bit sequence; Wherein, the starting bit of the first bit sequence is determined in the base address of the first cache line by a hash operation based on the first parameter R and the second parameter B, and the ending bit of the first bit sequence is determined in the base address of the first cache line by a hash operation based on the first parameter R. The first parameter R is the granularity of the cache line, and the unit is bytes.
7. The device according to claim 6, characterized in that, The first bit of the first bit sequence is ordered from low to high in the base address of the first cache line as a first value. The first value is the sum of the base-2 logarithm of the first parameter R and the base-2 logarithm of the second parameter B minus one. The starting value of the sorting position is zero. The second value is the sorting position of the end bit of the first bit sequence in the base address of the first cache line from low to high. The second value is the base-2 logarithm of the first parameter R, and the starting value of the sorting position is zero.
8. The device according to any one of claims 5 to 7, characterized in that, The cache device also includes a read data merging unit; The first cache set is further configured to mark the first cache line corresponding to the first sub-request as locked when the first sub-request is in a hit state; When the merged data is obtained by the read data merging unit, the first cache line is marked as unlocked.
9. The device according to any one of claims 1 to 3, characterized in that, The request splitting unit is used to receive the first write request sent by the request initiation unit. The data requested by the first write request is distributed across P cache lines, where P is a positive integer greater than 1. The first write request is split into P sub-requests, and each of the P sub-requests corresponds one-to-one with one of the P cache lines; Send the P sub-requests to the cache unit; The caching unit is used to perform a hit test on the P sub-requests. For the sub-requests that are in a hit state, the cache line corresponding to the sub-request is set to an invalid state, and the cache line is not written. And, send the P sub-requests to the output merging unit; The output merging unit is used to collect the P sub-requests, send the first write request to the downstream storage device, the downstream storage device returns the first write response corresponding to the first write request to the cache device, and the cache device feeds back the first write response to the request initiation unit.
10. The device according to claim 2, characterized in that, The memory access request includes a first write request, and the at least two sub-requests include a second sub-request, wherein the second sub-request is any one of the at least two sub-requests; the cache unit includes B cache sets, where B is a positive integer greater than 1, and the B cache sets include the second cache set; The request splitting unit is also used to perform a hash operation based on the base address of the second cache line to obtain the identifier of the second cache set; Based on the identifier of the second cache set, the second sub-request is sent to the second cache set; The second cache line is the cache line requested by the second sub-request.
11. The device according to claim 10, characterized in that, The request splitting unit is also used to obtain the identifier of the second cache set based on the second bit sequence; The starting bit of the second bit sequence is determined by a hash operation on the base address of the second cache line based on the first parameter R and the second parameter B. The ending bit of the second bit sequence is determined by a hash operation on the base address of the second cache line based on the first parameter R. The first parameter R is the granularity of the cache line, and the unit is bytes.
12. The device according to claim 11, characterized in that, The starting bit of the second bit sequence is sorted from low to high in the base address of the second cache line by a first value. The first value is the sum of the base-2 logarithm of the first parameter R and the base-2 logarithm of the second parameter B minus one. The starting value of the sorting position is zero. The second bit sequence is ordered from low to high in the base address of the second cache line as a second value, which is the base-2 logarithm of the first parameter R, and the starting value of the sorting position is zero.
13. The device according to any one of claims 1 to 3, characterized in that, The cache device also includes a read data merging unit; The request splitting unit is used to receive a second read request sent by the request initiation unit, wherein the second read request is a request in bypass mode; The second read request is bypassed and transmitted to the cache unit; The cache unit is used to bypass the transmission of the second read request to the output merging unit; The output merging unit is used to bypass the second read request and transmit it to the downstream storage device; The read data merging unit is used to bypass the data returned by the downstream storage device and transmit it to the request initiation unit.
14. The device according to claim 13, characterized in that, The memory access request includes a second read request, which is a bypass mode request; the cache unit includes B cache sets, and the B cache sets include a third cache set; The request splitting unit is further configured to bypass the transmission of the second read request to the third cache set, wherein the third cache set is the cache set corresponding to the second read request; The third cache set is also used to bypass the second read request and transmit it to the output merging unit.
15. The device according to claim 14, characterized in that, The request splitting unit is further configured to perform a hash operation based on the starting address of the second read request to obtain the identifier of the third cache set; and to bypass and transmit the second read request to the third cache set based on the identifier of the third cache set.
16. The device according to claim 15, characterized in that, The request splitting unit is also used to perform a bit sequence XOR operation based on the third bit sequence and the fourth bit sequence to obtain the identifier of the third cache set; Wherein, the bit sequence XOR operation refers to the operation of performing XOR calculation between the i-th third bit in the third bit sequence and the i-th fourth bit in the fourth bit sequence, where i is a positive integer; The starting bit of the third bit sequence is determined by a hash operation on the starting address of the second read request based on the first parameter R and the second parameter B. The ending bit of the third bit sequence is determined by a hash operation on the starting address of the second read request based on the first parameter R. The first parameter R is the cache line granularity, in bytes. The starting bit of the fourth bit sequence is determined by hashing the first parameter R, the second parameter B, and the third parameter N in the starting address of the second read request. The ending bit of the fourth bit sequence is determined by hashing the first parameter R, the second parameter B, and the third parameter N in the starting address of the second read request. The third parameter N is the ratio of the largest request granularity to the cache line granularity.
17. The device according to claim 16, characterized in that, The starting bit of the third bit sequence is sorted from low to high in the starting address of the second read request by a first value. The first value is the sum of the base-2 logarithm of the first parameter R and the base-2 logarithm of the second parameter B minus one. The starting value of the sorting position is zero. The end bit of the third bit sequence is sorted from low to high in the starting address of the second read request by a second value, which is the base-2 logarithm of the first parameter R, and the starting value of the sorting position is zero. The starting bit of the fourth bit sequence is in the third value in the order of low to high in the starting address of the second read request. The third value is the sum of the base-2 logarithm of the first parameter R, twice the base-2 logarithm of the second parameter B, and the base-2 logarithm of the third parameter N, minus one. The starting value of the sorting position is zero. The end bit of the fourth bit sequence is the fourth value in the ascending order of the starting address of the second read request. The fourth value is the sum of the base-2 logarithm of the first parameter R, the base-2 logarithm of the second parameter B, and the base-2 logarithm of the third parameter N. The starting value of the sorting position is zero.
18. The device according to any one of claims 1 to 3, characterized in that, The request splitting unit is used to receive a second write request sent by the request initiation unit, wherein the second write request is a bypass mode request; and to bypass the transmission of the second write request to the cache unit. The caching unit is used to bypass the second write request and transmit it to the output merging unit; The output merging unit is further configured to bypass the second write request and transmit it to the downstream storage device, the downstream storage device returns the second write response corresponding to the second write request to the cache device, and the cache device bypasses the second write response and transmits it to the request initiation unit.
19. The device according to claim 18, characterized in that, The cache unit includes B cache sets, and the B cache sets include a fourth cache set, where B is a positive integer greater than 1; The request splitting unit is further configured to bypass the transmission of the second write request to the fourth cache set, the fourth cache set being the cache set corresponding to the second write request; The fourth cache set is also used to bypass the second write request and transmit it to the output merging unit.
20. The device according to claim 19, characterized in that, The request splitting unit is further configured to perform a hash operation based on the starting address of the second write request to obtain the identifier of the fourth cache set; and to bypass and transmit the second write request to the fourth cache set based on the identifier of the fourth cache set.
21. The device according to claim 20, characterized in that, The request splitting unit is also used to perform a bit sequence XOR operation based on the fifth bit sequence and the sixth bit sequence to obtain the identifier of the fourth cache set; Wherein, the bit sequence XOR operation refers to the operation of performing XOR calculation between the i-th fifth bit in the fifth bit sequence and the i-th sixth bit in the sixth bit sequence, where i is a positive integer; The starting bit of the fifth bit sequence is determined by a hash operation on the starting address of the second write request based on the first parameter R and the second parameter B. The ending bit of the fifth bit sequence is determined by a hash operation on the starting address of the second write request based on the first parameter R. The first parameter R is the cache line granularity, in bytes. The starting bit of the sixth bit sequence is determined by hashing the first parameter R, the second parameter B, and the third parameter N in the starting address of the second write request. The ending bit of the sixth bit sequence is determined by hashing the first parameter R, the second parameter B, and the third parameter N in the starting address of the second write request. The third parameter N is the ratio of the largest request granularity to the cache line granularity.
22. The device according to claim 21, characterized in that, The starting bit of the fifth bit sequence is sorted from low to high in the starting address of the second write request by a first value. The first value is the sum of the base-2 logarithm of the first parameter R and the base-2 logarithm of the second parameter B minus one. The starting value of the sorting position is zero. The ending bit of the fifth bit sequence is sorted from low to high in the starting address of the second write request by a second value, which is the base-2 logarithm of the first parameter R, and the starting value of the sorting position is zero. The starting bit of the sixth bit sequence is the third value in the order of low to high in the starting address of the second write request. The third value is the sum of the base-2 logarithm of the first parameter R, twice the base-2 logarithm of the second parameter B, and the base-2 logarithm of the third parameter N, minus one. The starting value of the sorting position is zero. The end bit of the sixth bit sequence is the fourth value in the ascending order of the starting address of the second write request. The fourth value is the sum of the base-2 logarithm of the first parameter R, the base-2 logarithm of the second parameter B, and the base-2 logarithm of the third parameter N. The starting value of the sorting position is zero.
23. A cache access method, characterized in that, Applied to the cache device of claim 1, the method includes: The cache unit determines the hit status information of at least two sub-requests. The at least two sub-requests are obtained by splitting the memory access request in response to the prompt information that the memory access request carries a normal request. The request granularity of the memory access request is larger than the cache line granularity corresponding to the cache unit. The output merging unit sends the memory access request to the downstream storage device when it has received all at least two sub-requests, or sends the memory access request and the hit status information of the at least two sub-requests to the downstream storage device when it has received all at least two sub-requests. The caching unit bypasses and transmits the bypass mode request to the output merging unit; the output merging unit bypasses and transmits the bypass mode request to the downstream storage device; the bypass mode request carries a bypass request prompt message so that the request splitting unit can identify the bypass mode request and transmit the bypass mode request to the caching unit.
24. A processor, characterized in that, The processor includes a cache device as described in any one of claims 1 to 22.
25. A chip, characterized in that, The chip includes a processor, and the processor includes a cache device as described in any one of claims 1 to 22.
26. A server, characterized in that, The server includes a caching device as described in any one of claims 1 to 22.
27. An electronic device, characterized in that, The electronic device includes a processor, the processor including a cache device as described in any one of claims 1 to 22.
Citation Information
Patent Citations
Method and device for accessing fine-grained cache by GPU (Graphics Processing Unit) multi-granularity memory access request
CN114661352A
Methods and systems for distributing memory requests
US10558573B1
