Memory access request processing method, apparatus, device, storage medium, and program product

Abstract

Embodiments of the present disclosure disclose a memory access request processing method and device, equipment, a storage medium and a program product, wherein the method comprises: obtaining a first memory access instruction to be processed currently; the first memory access instruction comprises a plurality of first memory access requests; based on the matching relationship between the access addresses of the plurality of first memory access requests and the cache lines, the plurality of first memory access requests are merged to obtain at least one merged request; based on the corresponding relationship between the first memory access request and the merged request, the return type of the first memory access instruction, and the merged request, the first memory access instruction is processed; the return type of the first memory access instruction represents the return mode of the access data between the first memory access instructions and the return mode of the access data between the first memory access requests in the first memory access instruction. In this way, the upstream module of various memory access requirements can be supported, the application range is wide, and the hardware overhead is saved.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of computer technology, and in particular to a memory access request processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] In the design of parallel computing chips such as graphics processing units (GPUs) and artificial intelligence (AI) chips, a single parallel execution unit executes the tasks of multiple threads simultaneously. To reduce the number of memory accesses to downstream memory modules (such as Double Data Rate Synchronous Dynamic Random Access Memory, DDR), a memory coalescing module is typically used to merge memory access requests from multiple threads at a certain granularity.

[0003] Existing memory merging modules can only process memory access request return data using either in-order or out-of-order methods, and cannot support both methods simultaneously. Therefore, the existing solution cannot meet the diverse needs of different business scenarios for return data processing methods and suffers from poor applicability. Summary of the Invention

[0004] In view of the above, embodiments of this disclosure provide at least one memory access request processing method, apparatus, device, storage medium, and program product.

[0005] The technical solution of this disclosure embodiment is implemented as follows:

[0006] On one hand, embodiments of this disclosure provide a memory access request processing method, which includes:

[0007] Obtain the first memory access instruction to be processed; the first memory access instruction includes multiple first memory access requests;

[0008] Based on the matching relationship between the access addresses of multiple first memory access requests and cache lines, the multiple first memory access requests are merged to obtain at least one merged request;

[0009] Based on the correspondence between the first memory access request and the merge request, the return type of the first memory access instruction, and the merge request, the first memory access instruction is processed; the return type of the first memory access instruction represents the return method of the access data between the first memory access instructions and the return method of the access data between the first memory access requests within the first memory access instruction.

[0010] On the other hand, embodiments of this disclosure provide a memory access request processing apparatus, which includes:

[0011] The module is configured to acquire the first memory access instruction to be processed; the first memory access instruction includes multiple first memory access requests.

[0012] The merging module is configured to merge multiple first memory access requests based on the matching relationship between the access addresses of multiple first memory access requests and cache lines, to obtain at least one merged request;

[0013] The processing module is configured to process the first memory access instruction based on the correspondence between the first memory access request and the merge request, the return type of the first memory access instruction, and the merge request; the return type of the first memory access instruction represents the return method of the access data between the first memory access instructions and the return method of the access data between the first memory access requests within the first memory access instruction.

[0014] In another aspect, embodiments of this disclosure provide a computer device, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0015] In another aspect, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements some or all of the steps in the above-described method.

[0016] In another aspect, embodiments of this disclosure provide a computer program including computer-readable code, which, when executed in a computer device, causes a processor in the computer device to perform some or all of the steps in the above-described method.

[0017] In another aspect, embodiments of this disclosure provide a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method.

[0018] In this embodiment, merging is performed based on the matching relationship between the access addresses of multiple first memory access requests and cache lines. This allows multiple memory access requests within the same cache line to be merged together, achieving a better merging effect and maximizing request processing efficiency. Based on the correspondence between the first memory access requests and the merge request, the return type of the first memory access instruction, and the processing of the first memory access instruction by the merge request, appropriate processing can be performed according to the return type of the first memory access instruction. Thus, regardless of whether the return type of the first memory access instruction is in order, partially in order, or out of order, it can be processed accordingly using the memory access request processing method provided in this embodiment. This method supports upstream modules with multiple memory access requirements, has a wide range of applications, and saves hardware overhead by implementing upstream modules supporting multiple memory access requirements through a single method.

[0019] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0021] Figure 1 A schematic diagram of the implementation flow of a memory access request processing method provided in this embodiment of the disclosure. Figure 1 ;

[0022] Figure 2 A schematic diagram of the implementation flow of a memory access request processing method provided in this embodiment of the disclosure. Figure 2 ;

[0023] Figure 3 A schematic diagram illustrating the composition of a memory access request processing method provided in an embodiment of this disclosure;

[0024] Figure 4 This is a schematic diagram of the composition structure of a memory access request processing device provided in an embodiment of the present disclosure;

[0025] Figure 5 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0027] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] The terms “first / second / third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this disclosure.

[0030] This disclosure provides a memory access request processing method, which can be executed by a processor of a computer device. The computer device refers to a device with data processing capabilities, such as a server, laptop, tablet, desktop computer, smart TV, set-top box, or mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device). Figure 1 As shown, the method may include the following steps 101 to 103:

[0031] Step 101: Obtain the first memory access instruction to be processed; the first memory access instruction includes multiple first memory access requests.

[0032] The first memory access instruction refers to one or more memory access instructions to be processed. Each memory access instruction can include M memory access requests. Memory access requests can be read requests or write requests. Each memory access request can refer to the memory access request of a thread, also known as a thread task.

[0033] Each memory access request includes at least the following information: return type, request length, and memory address. The {request type; request length} can be shared by M memory access requests within a single memory access instruction.

[0034] Step 102: Based on the matching relationship between the access addresses of the multiple first memory access requests and the cache lines, merge the multiple first memory access requests to obtain at least one merged request.

[0035] A merge request refers to a request that combines multiple first memory access requests for the same cache line. A merge request must include at least: return type, address length, and the cache line address of the merge request.

[0036] To reduce the number of memory accesses to downstream modules (such as Double Data Rate SDRAM), multiple memory access requests are typically merged according to a preset granularity. However, the embodiments of this disclosure merge requests based on the matching relationship between the access addresses of multiple first memory access requests and cache lines. This allows multiple memory access requests within the same cache line to be combined together, achieving a better merging effect and thus maximizing request processing efficiency.

[0037] In some implementations, step 102 can be specifically implemented by merging multiple first memory access requests according to whether the access addresses of the first memory access requests are in the same cache line, to obtain at least one merged request.

[0038] Step 103: Process the first memory access instruction based on the correspondence between the first memory access request and the merge request, the return type of the first memory access instruction, and the merge request.

[0039] The return type of the first memory access instruction represents the return method of the access data between the first memory access instructions, and the return method of the access data between the first memory access requests within the first memory access instruction.

[0040] The correspondence between the first memory access request and the merge request is to ensure that the access data obtained through the merge request is correctly returned to the upstream module.

[0041] The return type of a memory access instruction indicates how the accessed data is returned between memory access instructions, as well as how the accessed data is returned between memory access requests within a memory access instruction.

[0042] In one feasible implementation, the return type of memory access instructions can include a first type, a second type, and a third type. The first type indicates that the return order of data accessed between memory access instructions is independent of the order in which they are received, and the return order of data accessed between memory access requests is independent of the order in which they are received; that is, both within and between instructions are out of order. The second type indicates that the return order of data accessed between memory access instructions is independent of the order in which they are received, and the return order of data accessed between memory access requests is the same as the order in which they are received; within instructions, it is in order, while between instructions it is out of order, also known as partially in order. The third type indicates that the return order of data accessed between memory access instructions is the same as the order in which they are received, and the return order of data accessed between memory access requests is the same as the order in which they are received; both within and between instructions are in order.

[0043] Both within and between instructions, the order of access data returned to the upstream module is indicative of the order in which the memory access instructions and requests sent by the upstream module are received. In other words, access data from multiple instructions is collected before being returned to the upstream module together. This reduces the number of interactions with the upstream module; however, the trade-off is that if one request in the M threads results in a cache miss, the data from other cache hits must also wait, creating a bottleneck effect. Furthermore, because the order of instructions also applies, even if a later-received instruction collects all read data first, it must wait for the preceding instructions to complete before returning, further exacerbating the bottleneck effect.

[0044] "In order within an instruction, out of order between instructions" means that the order in which the access data is returned to the upstream module is exactly the same as the order in which the memory access requests sent by the upstream module are received, but it is not related to the order in which the memory access instructions sent by the upstream module are received; that is, once all the access data of a certain instruction has been received, it can be returned to the upstream module together.

[0045] The "out of order" approach, both within and between instructions, means that the order in which access data is returned to the upstream module is independent of the order in which memory access instructions and requests are received from the upstream module. In other words, if access data for any memory access request is obtained, the access data can be directly returned to the upstream module. Thus, there is no data collection action within the instruction, resulting in a higher number of interactions with the upstream module and wasted interface bandwidth. However, because there is no data collection action and no waiting logic in the "in order" approach, there is no bottleneck effect, reducing path latency to some extent.

[0046] In some implementations, step 103 can be specifically implemented as follows: according to the return type of the first memory access instruction, the access data obtained through the merge request is processed differently based on the correspondence between the first memory access request and the merge request, so as to realize the processing of the first memory access instruction.

[0047] In this embodiment, merging is performed based on the matching relationship between the access addresses of multiple first memory access requests and cache lines. This allows multiple memory access requests within the same cache line to be merged together, achieving a better merging effect and maximizing request processing efficiency. Based on the correspondence between the first memory access requests and the merge request, the return type of the first memory access instruction, and the processing of the first memory access instruction by the merge request, appropriate processing can be performed according to the return type of the first memory access instruction. Thus, regardless of whether the return type of the first memory access instruction is in order, partially in order, or out of order, it can be processed accordingly using the memory access request processing method provided in this embodiment. This method supports upstream modules with multiple memory access requirements, has a wide range of applications, and saves hardware overhead by implementing upstream modules supporting multiple memory access requirements through a single method.

[0048] This disclosure provides a memory access request processing method, which can be executed by the processor of a computer device. For example... Figure 2 As shown, the method includes the following steps 201 to 207:

[0049] Step 201: Receive memory access instructions sent by multiple objects.

[0050] An object can refer to an upstream module, which can be a parallel execution unit. Different objects send memory access instructions with different return types.

[0051] In some implementations, step 201 can be implemented as follows: each of the multiple objects can send one or more memory access instructions; these multiple memory access instructions can be sent sequentially or arranged in a certain order and sent together.

[0052] Step 202: Determine the target return type to be processed from the return types of the multiple memory access instructions.

[0053] The target return type refers to the return type that is currently being processed.

[0054] Since this embodiment supports three return types—in order, partial in order, and out of order—after obtaining memory access instructions sent by multiple objects, one of these three return types can be selected for processing.

[0055] In some implementations, step 202 can be specifically implemented by: employing a selection strategy to determine the target return type from the return types of multiple memory access instructions. The selection strategy can be a random selection strategy, a round-robin selection strategy, or a weight-based selection strategy, etc.; wherein, the weight can be determined based on at least one of the priority of the return type and the priority of the object.

[0056] Step 203: Take the memory access instruction corresponding to the target return type as the first memory access instruction to be processed; the first memory access instruction includes multiple first memory access requests.

[0057] Here, steps 201 to 203 correspond to the aforementioned step 101, and can be implemented with reference to the specific implementation method of the aforementioned step 101.

[0058] In some implementations, step 203 can be specifically implemented by taking one or more memory access instructions corresponding to the target return type as the first memory access instruction to be processed.

[0059] Step 204: Based on the matching relationship between the access addresses of the multiple first memory access requests and the cache lines, merge the multiple first memory access requests to obtain at least one merged request.

[0060] Here, step 204 corresponds to the aforementioned step 102, and the specific implementation of step 204 can be referred to during implementation.

[0061] In some implementations, step 204 can be achieved through the following steps 2041 to 2042:

[0062] Step 2041: According to the access address of the first memory access request, divide the plurality of first memory access requests into at least one request group in the same cache line.

[0063] In some implementations, step 2041 can be specifically implemented as follows: using a comparator to determine at least one first memory access request whose access address is in the same cache line from multiple first memory access requests; grouping at least one first memory access request whose access address is in the same cache line as a request group to obtain multiple request groups.

[0064] In some implementations, step 2041 can be specifically implemented as follows: performing comparator matching on the access addresses of the plurality of first memory access requests until all first memory access requests have been matched to obtain at least one request group; the comparator matching includes: determining the base address of the comparator based on the access addresses of the plurality of first memory access requests; matching the access addresses of the plurality of first memory access requests with the base address of the comparator; if the matching is successful, grouping the second memory access requests that are in the same cache line as the base address of the comparator into a request group in the same cache line; if the matching fails, determining a new base address of the comparator based on the access addresses of the unmatched first memory access requests.

[0065] The base address of the comparator can be determined in at least three ways:

[0066] The first method uses the access address of the first memory access request with the largest unmatched identifier as the base address of the comparator.

[0067] The identifier of the first memory access request is used to uniquely identify the first memory access request. For example, the thread IDs of M threads in a memory access instruction are 0 to M-1 respectively. Here, thread represents the memory access request, and threadid represents the identifier of the memory access request.

[0068] In one implementation method, the access address of the first memory access request with the largest identifier among multiple first memory access requests can be used as the base address of the first set of comparators. The access addresses of multiple first memory access requests are matched with the base address of the first set of comparators to obtain a request group that is in the same cache line as the base address of the first set of comparators. The access address of the first memory access request with the largest identifier among multiple unmatched first memory access requests is used as the base address of the second set of comparators. The access addresses of multiple unmatched first memory access requests are matched with the base address of the second set of comparators to obtain a request group that is in the same cache line as the base address of the second set of comparators. This process is repeated until all first memory access requests have been matched.

[0069] The second method uses the access address of the first memory access request with the smallest unmatched identifier as the base address of the comparator.

[0070] In the second implementation method, the access address of the first memory access request with the smallest identifier among multiple first memory access requests can be used as the base address of the first set of comparators. The access addresses of multiple first memory access requests are matched with the base address of the first set of comparators to obtain a request group that is in the same cache line as the base address of the first set of comparators. The access address of the first memory access request with the smallest identifier among multiple unmatched first memory access requests is used as the base address of the second set of comparators. The access addresses of multiple unmatched first memory access requests are matched with the base address of the second set of comparators to obtain a request group that is in the same cache line as the base address of the second set of comparators. This process is repeated until all first memory access requests have been matched.

[0071] The third approach is to use the address of the first memory access request with the smallest unmatched identifier as the base address of the i*2-1th comparator group, and the address of the first memory access request with the largest unmatched identifier as the base address of the i*2th comparator group; where i is a positive integer.

[0072] In the third implementation method, the access address of the first memory access request with the smallest identifier among multiple first memory access requests can be used as the base address of the first set of comparators. The access addresses of the multiple first memory access requests are matched with the base address of the first set of comparators to obtain a request group that is in the same cache line as the base address of the first set of comparators. Then, the access address of the first memory access request with the largest identifier among the multiple unmatched first memory access requests is used as the base address of the second set of comparators. The access addresses of the multiple unmatched first memory access requests are matched with the base address of the second set of comparators to obtain a request group that is in the same cache line as the base address of the second set of comparators. This process is repeated until all first memory access requests have been matched. At this point, the base address of the odd-numbered comparators is the minimum address, and the base address of the even-numbered comparators is the maximum address.

[0073] It should be noted that the matching operations of the i*2-1th comparator and the i*2th comparator can be performed simultaneously. In this way, the matching of the minimum identifier and the matching of the maximum identifier can be achieved at the same time, maximizing the matching rate and thus improving the merging rate.

[0074] In some implementations, the specific implementation of "matching the access addresses of the plurality of first memory access requests with the base address of the comparator" in step 2041 can be as follows: based on the address length of the first memory access request and the size of a single cache line, determine the first number of first memory access requests that the single cache line can contain; match the access addresses of the plurality of first memory access requests with the base address of the comparator to obtain a first number of second memory access requests that are in the same cache line as the base address of the comparator.

[0075] The first number refers to the number of memory access requests that a single cache line can contain. For example, if the address length of the first memory access request is 16 bytes and the size of a single cache line is 128 bytes, then the first number of memory access requests that a single cache line can contain is 8.

[0076] In some embodiments, the memory access request processing method provided in this disclosure may further include: setting a matching identifier for each of the plurality of first memory access requests; and updating the matching identifier of any first memory access request from a first identifier to a second identifier when any of the plurality of first memory access requests is matched.

[0077] The matching identifier is used to indicate whether a memory access request has been matched. If the matching identifier of a memory access request is the first identifier, then the memory access request has not been matched; if the matching identifier of the memory access request is the second identifier, then the memory access request has been matched.

[0078] Step 2042: Merge multiple first memory access requests in each request group into one request to obtain at least one merged request.

[0079] Multiple first memory access requests in each request group are merged into one request; that is, multiple first memory access requests in each cache line are merged into one request.

[0080] In some embodiments, the memory access request processing method provided in this disclosure may further include: when there is a target memory access request whose access address is the base address in any set of comparators and matches the base address of other sets of comparators besides the any set of comparators, determining that the matching result of the any set of comparators is duplicated with the matching result of the other comparators; and selecting the matching result of the comparator with the lower group from the matching result of the any set of comparators and the matching result of the other sets of comparators.

[0081] A target memory access request refers to a memory access request whose access address is the base address in any comparator and matches the base address of all other comparators except for any one of them. In other words, if a memory access request is simultaneously the base address of one comparator and a memory access request that matches the base address of another comparator, it means that the matching results of the two comparators are duplicated. In this case, the matching result of the comparator with the lower group is selected. The reason is that the smaller the group of the comparator, the more comprehensive the memory access requests included in its matching results.

[0082] Suppose a memory access instruction includes M memory access requests, and one cycle can merge at least W merged requests (W>=1, M is an integer multiple of W, and W<=M / 2). In this case, the merging process requires W sets of comparators. One set of address comparators selects the access address of one memory access request as the base address, and the access addresses of other memory access requests are matched against the base address. If a match is found, these matched memory access requests do not participate in the merging of subsequent cycles; otherwise, the unmatched memory access requests need to be matched against the base addresses of other comparators, or wait for matching in subsequent cycles.

[0083] For odd-numbered comparators (1, 3, 5...), the smaller address is used as the base address; for even-numbered comparators (2, 4, 6...), the larger address is used as the base address. Table 1 below shows the cycle1 based threadid (base address identifier), cycle1 match threadid (identifier of the matched memory access request), and the number of comparators in each comparator group (1, 2, 3, 4). There are a total of W comparators, requiring a total of M-1 + M-2 + ... = W*M – (1 + W)*W / 2 comparators.

[0084]

[0085]

[0086] Table 1

[0087] A coalesce_matched_bit (matching identifier) ​​is set, initially set to 0 (first identifier). When a memory access request is matched in a cycle, the matching identifier of that memory access request is set to 1 (0 and 1 are example values, but other values ​​can be used). The coalesce_matched_bit is updated after each cycle and used as input for the next cycle. Subsequent cycles only merge memory access requests where the coalesce_matched_bit is 0.

[0088] Starting with Cycle2, the odd-numbered comparators for the 1st, 3rd, 5th... odd-numbered arrays select the smallest (the smallest for the 1st group, the 2nd smallest for the 3rd group, the 3rd smallest for the 5th group, and so on) memory access requests that are 0 from the output of the previous cycle's `coalesce_matched_bit` as the base address. Other memory access requests are then matched against the base address. The number of remaining memory access requests that are 0 is always less than the number of comparators in the current group, and any excess comparators can be left idle.

[0089] Starting with Cycle2, the even-numbered array comparators in the 2nd, 4th, 6th... cycles select the largest (the 2nd group corresponds to the largest, the 4th group to the 2nd largest, the 6th group to the 3rd largest, and so on) memory access request with a value of 0 from the coalesce_matched_bit output of the previous cycle, using it as the base address. Other memory access requests are then matched against this base address. When all coalesce_matched_bits in a given cycle are 0, it indicates that the coalesce operation for this instruction is complete.

[0090] Since W comparators perform merging operations in one cycle, a thread ID can be both a based thread ID and a match thread ID in the same cycle. Consequently, the cachelinesize based memory address output by multiple comparators may be duplicated. In this case, the matching result with the smaller comparator group can be taken, because a smaller comparator group includes a more comprehensive total number of thread IDs.

[0091] Step 205: Store the merge request and record the storage address of the merge request.

[0092] Recording the storage address of the merge request is to determine which cache line to return the access data to when the downstream module returns the access data.

[0093] In some implementations, step 205 can be specifically implemented as follows: if the number of merge requests matches the number of cache lines in each group of the first storage module, the merge request is stored in the first storage module; the first storage module is a group-related cache; the bank identifier, group identifier, and road identifier of the cache line containing the merge request are used as the storage address of the merge request; the storage address of the merge request and the multiple first memory access requests corresponding to the merge request are recorded as entries in the first data cache corresponding to the merge request; each bank in the first storage module corresponds to one first data cache.

[0094] The first storage module can be a group-dependent cache. For example, the first storage module can be a normal memory cache to improve data access efficiency. A normal cache uses {bank id, set id, way id} to uniquely identify a cache line; where bank id is the bank identifier, set id is the group identifier, and way id is the way identifier. A bank refers to an independently operable memory area. A cache can be divided into multiple banks to support simultaneous read and write operations; for example, if a cache has 4 banks, it can perform 4 independent read and write operations simultaneously. A set is an index group in the cache that contains multiple possible cache lines; caches are typically organized into multiple sets, each containing a certain number of cache lines. A way refers to the number of replicas of a cache line; for example, a four-way cache means that each set has 4 cache lines, which can store different data.

[0095] The number of cache lines in each group refers to the number of paths contained in each set. The number of merge requests matches the number of cache lines in each group in the first storage module, meaning that the number of merge requests Q after merging M memory access requests is less than or equal to the number of paths U. In this case, if Q merge requests happen to fall into the same set in the same bank, then the normal cache can also receive all the merge requests.

[0096] The first data cache is the data cache of the first storage module, used to record the storage address of the merge request and the multiple memory access requests corresponding to the merge request. Each bank in the first storage module has a data cache. For example, the first data cache can be a request FIFO, used to record information such as the storage address of the cacheline set / way of the merge request stored in the first storage module and the corresponding memory access requests. The request FIFO includes multiple entries, each corresponding to a merge request. The entries in the request FIFO are stored sequentially (push), but can be released out of order (release).

[0097] In some implementations, the specific way to "record the storage address of the merge request and the multiple first memory access requests corresponding to the merge request in the entries of the first data cache corresponding to the merge request" can be: according to the order in which the merge requests are obtained, record the storage address of at least one merge request and the multiple first memory access requests corresponding to each merge request in the entries of the first data cache corresponding to the merge request.

[0098] In some implementations, step 205 may be further implemented as follows: if the number of merge requests does not match the number of cache lines in each group of the first storage module, a second number of first merge requests are stored in the first storage module, and a third number of second merge requests are stored in the second storage module; the first storage module is a group-related cache, and the second storage module includes multiple entries, each entry being able to store data of the same size as a cache line; the second number of first merge requests and the third number of second merge requests are equal to the merge request; the storage address of the first merge request is recorded in the first data cache corresponding to the first storage module, and the storage address of the second merge request is recorded in the second data cache corresponding to the second storage module; each bank in the first storage module corresponds to one first data cache; and the second storage module corresponds to one second data cache.

[0099] In some implementations, the specific way to "record the storage address of the first merge request in the first data cache corresponding to the first storage module, and record the storage address of the second merge request in the second data cache corresponding to the second storage module" can be as follows: use the bank identifier, group identifier, and road identifier of the cache line storing the first merge request as the first storage address of the merge request, and use the identifier of the entry storing the second merge request as the second storage address of the merge request; record the first storage address and the multiple first memory access requests corresponding to the first merge request as entries in the first data cache; and record the second storage address and the multiple first memory access requests corresponding to the second merge request as entries in the second data cache.

[0100] The number of merge requests does not match the number of cache lines in each group of the first storage module, meaning that the number of merge requests Q > the number of paths U. In this case, if Q merge requests happen to fall into the same set in the same bank, then the normal cache cannot receive all the merge requests.

[0101] Here, the first storage module is used to store a second number of merge requests, and the second storage module is used to store a third number of merge requests. The first storage module is a normal cache, and the second storage module is a redundant static random-access memory (SRAM).

[0102] Redundancy SRAM contains multiple entries, and each entry can store data for one cacheline. It can be considered a cache with bank num = 1, set num = 1, and way num = the total number of entries. Redundancy SRAM is organized at the cacheline level and uses a similar encoding method to the normal cache to uniquely identify entries within it. For example, if U is not a power of 2, the way id encoding can be used to number the entries in the redundancy SRAM; if U is a power of 2, all bits of the {bank id, set id, way id} in the normal cache are occupied, and one additional bit is needed to identify the entry in the redundancy SRAM.

[0103] Taking B=4, S=4, U=16 as an example, since U is 2 to the power of n, one bit needs to be added to uniquely identify a cacheline, such as using {redundancy id, bank id, set id, way id}, where redundancy id is 1 bit and bank id is 2 bits; if redundancy id is 1 and bank id is 00, then combining {redundancy id, bank id} can be considered as redundancy SRAM is bank4(100).

[0104] The second data cache is the data cache for the second storage module. Similar to the first data cache, it is used to record the storage address of the merge request and the multiple memory access requests corresponding to the merge request. One data cache is set up on the second storage module. For example, the second data cache can also be a request FIFO. Each normal cache bank has a request FIFO set up to record information such as the storage address of the merge request cacheline set / way stored in the second storage module and the corresponding memory access requests.

[0105] In one feasible implementation, if Q merge requests of a memory access instruction happen to fall into the same set in the same bank of the normal cache, and Q > U, then U merge requests are allowed to enter the normal cache, and the remaining Q–U merge requests are allowed to enter the redundancy SRAM and occupy QU entries. After these Q–U access data are returned, they will be temporarily stored in the redundancy SRAM, which can avoid deadlock problems.

[0106] Step 206: Obtain the access data of the merge request, and store the access data of the merge request in the corresponding cache line based on the storage address of the merge request.

[0107] In some implementations, the specific implementation of "obtaining the access data of the merge request" in step 206 can be as follows: determine the fourth number of merge requests that can be merged in one batch; determine the number of batches required for the merge request based on the fourth number and the number of merge requests; and send the merge requests to the downstream module in batches based on the number of batches to obtain the access data of the merge request.

[0108] If a memory access instruction includes M memory access requests, and one cycle can merge at least W merge requests (W>=1, M is an integer multiple of W, and W<=M / 2), then one instruction requires Celling(Q / W) cycles to send the merge requests to the downstream module; where Celling means rounding up.

[0109] Step 207: Based on the storage address of the merge request, the correspondence, and the return type of the first memory access instruction, return the access data of the first memory access instruction to the corresponding object.

[0110] The return type of the first memory access instruction represents the return method of the access data between the first memory access instructions, and the return method of the access data between the first memory access requests within the first memory access instruction.

[0111] Here, steps 205 to 207 correspond to the aforementioned step 103, and the specific implementation of the aforementioned step 103 can be referred to during implementation.

[0112] In some implementations, when the return type is the first type (out of order), step 207 can be implemented by step A as follows:

[0113] Step A: When the return type of the first memory access instruction is the first type and there is a third merge request with access data ready, perform the first return operation on the access data for the third merge request until all access data of the first memory access instruction has been returned to the corresponding object; the first type indicates that the return order of access data between the first memory access instructions is independent of the receiving order between the first memory access instructions, and the return order of access data between the first memory access requests is independent of the receiving order of the first memory access requests;

[0114] The first return operation includes: determining multiple third memory access requests corresponding to the third merge request based on the correspondence; and returning the access data of the multiple third memory access requests to the corresponding object based on the storage address of the merge request.

[0115] A third merge request refers to a merge request for which access to data is ready. A third memory access request refers to one or more memory access requests corresponding to a third merge request.

[0116] In one feasible implementation, the correspondence can be recorded in a request bitmap. Specifically, the request bitmap may include: object identifier, instruction identifier, return type, and other instruction information; the number of merge requests for each bank in the first storage module and whether the accessed data is ready; and the number of merge requests in the second storage module and whether the accessed data is ready. Thus, multiple third memory access requests corresponding to the third merge request can be determined from the request bitmap.

[0117] In some implementations, the specific way to "return the access data of the plurality of third memory access requests to the corresponding object based on the storage address of the merge request" can be: directly return the access data of the plurality of third memory access requests to the corresponding object based on the storage address of the merge request.

[0118] In some implementations, the specific implementation of "returning the access data of the plurality of third memory access requests to the corresponding objects based on the storage address of the merge request" can also be as follows: based on the storage address of the merge request, perform a second return operation on the access data for each third memory access request until the access data of the plurality of third memory access requests have been returned to the corresponding objects; the second return operation includes: based on the storage address of the merge request, transferring the access data of each third memory access request from the cache line corresponding to each third memory access request to the third data cache corresponding to each third memory access request, so as to return the access data of each third memory access request to the corresponding object through the third data cache; each bank in the first storage module corresponds to one third data cache; release the data about each third memory access request in the first data cache.

[0119] The third data buffer is used to temporarily store third memory access requests so that the accessed data from the third memory access request can be returned to the corresponding object. The third data buffer can be an out-of-order return FIFO. The out-of-order return FIFO can have two layers: the first layer stores the i-th selection result, and the second layer stores the (i+1)-th selection result; in this way, the pressure of returning accessed data can be reduced by using the third data buffer.

[0120] In some implementations, the order in which the third memory access request performs the second return operation can be random, or determined based on at least one of the following: the return order of the accessed data, the access order of the request, or the order determined based on weights.

[0121] In some implementations, when the return type is the second type (partial in order), step 207 can be implemented by step B as follows:

[0122] Step B: When the return type of the first memory access instruction is the second type and there is a third merge request with access data ready, perform a third return operation on the third merge request until all access data of the first memory access instruction has been returned to the corresponding object; the second type indicates that the return order of access data between the first memory access instructions is independent of the receiving order between the first memory access instructions, and the return order of access data between the first memory access requests is the same as the receiving order between the first memory access requests.

[0123] The third return operation includes: setting the entry of the third merge request in the first data buffer as a third identifier; if there is a second memory access instruction whose access data for all included memory access requests is ready, setting the identifier of the second memory access instruction in the request bitmap as a fourth identifier; the fourth identifier indicates that the access data of the second memory access instruction is ready; and returning the access data of the second memory access instruction to the corresponding object according to the receiving order of the second memory access instructions.

[0124] The first memory access request refers to the order in which the upstream module sends the first memory access requests. The third identifier indicates that the access data for the corresponding merged request is ready. The second memory access request refers to a memory access instruction whose access data for all included memory access requests is ready. The fourth identifier indicates that the access data for all included memory access requests in the second memory access instruction is ready.

[0125] In some implementations, when the return type is third type (in order), step 207 can be implemented by the following steps C1 to C3:

[0126] Step C1: If the return type of the first memory access instruction is the third type and there is a third merge request for which access data is ready, set the entry of the third merge request in the first data buffer to the third identifier; the third type indicates that the return order of access data between the first memory access instructions is the same as the receiving order between the first memory access instructions, and the return order of access data between the first memory access requests is the same as the receiving order between the first memory access requests.

[0127] Step C2: If there is a second memory access instruction for which the access data of all memory access requests are ready, set the identifier of the second memory access instruction in the request bitmap to a fourth identifier; the fourth identifier indicates that the access data of the second memory access instruction is ready.

[0128] Step C3: If it is determined based on the request bitmap that the access data of all memory access instructions in the first memory access instructions are ready, return the access data of the first memory access instructions to the corresponding objects according to the receiving order of the first memory access instructions and the receiving order of the first memory access requests.

[0129] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0130] In this embodiment, merging is performed based on the matching relationship between the access addresses of multiple first memory access requests and cache lines. This allows multiple memory access requests within the same cache line to be merged together, achieving a better merging effect and maximizing request processing efficiency. Based on the correspondence between the first memory access requests and the merge request, the return type of the first memory access instruction, and the processing of the first memory access instruction by the merge request, appropriate processing can be performed according to the return type of the first memory access instruction. Thus, regardless of whether the return type of the first memory access instruction is in order, partially in order, or out of order, it can be processed accordingly using the memory access request processing method provided in this embodiment. This method supports upstream modules with multiple memory access requirements, has a wide range of applications, and saves hardware overhead by implementing upstream modules supporting multiple memory access requirements through a single method.

[0131] The following describes the application of the memory access request processing method provided in the embodiments of this disclosure in a real-world scenario.

[0132] In the design of parallel computing chips such as GPUs and AI, a parallel execution unit will execute the tasks of multiple threads simultaneously (assuming that the number of threads executing simultaneously is M). In order to reduce the number of memory accesses to downstream memory modules (such as DDR), a memory merging module is generally set up to merge the memory access requests (such as read / write requests) of multiple threads at a certain granularity. At the same time, in order to improve the bandwidth of memory access and reduce access latency, a cache is also set up.

[0133] The functions of the coalesce module and cache can be implemented separately by two modules, or they can be merged into one module. The functions of the coalesce module and cache will be collectively referred to as memory coalescing cache below. The memory access request before merging will be referred to as init_thread_request, and the merged request will be referred to as coalesced_cache_request. The M thread tasks executed by a parallel execution unit at one time are referred to as one instruction. This disclosure focuses on memory access instructions; non-memory access instructions (such as computation instructions) are not the subject of this disclosure.

[0134] The init_thread_request typically contains at least {request type; request length; M memory addresses}, where {request type; request length} is shared by the M threads.

[0135] A coalesced_cache_request typically contains at least {request type; request length; one cacheline size based memory address}.

[0136] One `init_thread_request` will correspond to Q `coalesced_cache_request`s based on the merging of M memory addresses. The minimum value of Q is 1, and the maximum value of Q is referred to as `Q_MAX` below. `Q_MAX` is predetermined based on M, R, and the allowed request length. For example, if R = 128 bytes, the maximum allowed request length is 128 bytes, and M = 32 bytes; then considering that a thread can span at most one cache line of R, a thread can correspond to a maximum of 2 `coalesced_cache_requests`, and `Q_MAX` is equal to 2 * M = 64.

[0137] For memory access requests like read requests that require accessing data, the mapping between init_thread_request and coalesced_cache_request (hereinafter referred to as requestmap) needs to be recorded in the coalesce module (some types of requests, such as write requests, do not require a response from the downstream module). This is so that the accessed data returned by the cache or the downstream module can be correctly returned to the upstream module (parallel execution unit).

[0138] Currently, there are two main ways to handle accessed data in memory merged caches: in order and out of order.

[0139] In order means that the order in which the coalesce module returns access data to the upstream module is exactly the same as the order in which it receives the init_thread_request from the upstream module.

[0140] "Out of order" means that the order in which the coalesce module returns access data to the upstream module is unrelated to the order in which it receives the init_thread_request from the upstream module. After obtaining the access data, the access data can be returned to the upstream module directly.

[0141] In-order processing ensures that all read data from the M threads within an instruction is collected before being returned to the accessed object, saving the number of interactions with the accessed object. However, the trade-off is that if there is a cache miss among the M threads, the data from other cache-hit threads also needs to wait, resulting in a bottleneck effect. Furthermore, since instructions are also in-order, even if a later instruction collects all read data first, it must wait for the preceding instruction to complete before returning, further exacerbating the bottleneck effect.

[0142] Out-of-order returns data directly to the upstream module in the order it can be accessed by the downstream module, without any data collection action within the instruction. This results in more interactions with the upstream module, leading to wasted interface bandwidth. However, because there is no data collection action and no waiting logic for in-order requests, there is no bottleneck effect, reducing path latency to some extent. Both return methods have their advantages and disadvantages, and both require coordination with the accessed object. In other words, the processing method of the upstream module generally determines which data access return method the memory merging cache should use.

[0143] Existing memory merge caches typically only support one of two methods for handling return read data: in-order or out-of-order. If a memory merge cache corresponds to two upstream modules, one requiring in-order and the other out-of-order, current technology cannot meet the requirements. This necessitates using two separate memory merge caches, increasing hardware resource overhead. Therefore, this disclosure proposes a novel memory merge cache that can support both in-order and out-of-order access to objects using a single cache.

[0144] Both read data return methods have their advantages and disadvantages. To be compatible with the advantages of both methods, this disclosure proposes a new data access return method: in-order within instructions and out-of-order between instructions, hereinafter referred to as partial in-order. Partial in-order ensures that the read data corresponding to the M threads within an instruction is collected before being returned to the upstream module together, saving the number of interactions with the upstream module; at the same time, since the out-of-order between instructions avoids the bottleneck effect between instructions.

[0145] like Figure 3 As shown, the overall block diagram of the novel memory merging cache provided in this embodiment includes: N upstream modules (N>=1), which have both in-order / partial in-order requirements and out-of-order requirements.

[0146] The memory coalescing cache mainly consists of sub-modules such as request coalesce (coalescing module), normal cache (first storage module), redundancy SRAM (second storage module), and return data handler (access data controller, including request map and in-order return data pack). Among them, redundancy SRAM is an optional sub-module.

[0147] Before entering the request coalesce stage, the init_thread_requests of N upstream modules need to be arbitrated by the input req MUX. The return type of the accessed data of each upstream module (hereinafter referred to as order_type, including: in order, partial in order, out of order) is predetermined.

[0148] The arbitration unit (Input req MUX) can know the order_type of the current init_thread_request based on the source of the init_thread_request. The init_thread_request_add_info sent to the request coalesce sub-module adds at least the identifiers {master_id, instruction_id, order_type} based on the init_thread_request. Among them, the master id is the identifier of N upstream modules; the instruction_id is used to identify different instructions issued by the same master. Generally, the init_thread_request itself carries the instruction id information. {master_id, instruction_id} can uniquely identify a received init_thread_request in the memory merge cache; the order_type can be obtained by looking up a table based on the master_id.

[0149] The merge module (request coalesce) is used to merge the M memory addresses in the init_thread_request_add_info into Q cacheline size based memory addresses according to whether the request addresses (base address + address length) are in the same cache line. According to the egress bandwidth requirement, it can be designed to merge at least W coalesced_cache_requests in 1 cycle (W >= 1. Generally, M is an integer multiple of W, and W <= M / 2). Then, 1 instruction needs Celling(Q / W) cycles to send the merged coalesced_cache_request to the downstream module; where Celling means rounding up. Since 1 <= Q <= Q_MAX, the minimum value of the total number of cycles T required for 1 instruction in the request coalesce sub-module is 1, and the maximum value is Q_MAX / W (the cycle numbers are numbered starting from 1 below, and cycle1 represents the first cycle).

[0150] The merging process requires the use of W sets of address comparators. One set of address comparators selects the address of one based thread ID as the base address, and compares the addresses of other thread IDs with the base address. If they match, these thread IDs do not participate in the matching of subsequent cycles; otherwise, they need to be matched with the base addresses of other sets of address comparators, or wait for the matching of subsequent cycles.

[0151] For odd-numbered comparators (groups 1, 3, 5, ...), the smaller thread ID is used as the based thread ID for cycle 1; for even-numbered comparators (groups 2, 4, 6, ...), the larger thread ID is used as the based thread ID for cycle 1. Table 1 shows the cycle 1 based thread ID, cycle 1 match thread ID, and the number of comparators in each group for comparators 1, 2, 3, and 4. The request coalesce submodule has a total of W comparators, requiring a total of M-1 + M-2 + ... = W*M – (1 + W)*W / 2 comparators.

[0152] A coalesce_matched_bit is set to M bits, initially all of which are 0. When a thread is matched in a cycle, the coalesce_matched_bit is set to 1 (0 and 1 are example values, other values ​​can be used). The based thread ID of each group is also included in the comparison, so its corresponding coalesce_matched_bit is also set to 1. The coalesce_matched_bit is updated after each cycle and used as input for the next cycle. Subsequent cycles only perform merging operations on threads with a coalesce_matched_bit of 0.

[0153] Starting with Cycle2, the 1st, 3rd, 5th... odd-numbered comparators select the smallest (the smallest in group 1, the second smallest in group 3, the third smallest in group 5, and so on) thraed ID that is 0 from the coalesce_matched_bit output of the previous cycle as the based thread ID. The other thread IDs are used as the match thraed IDs. The number of remaining thraed IDs that are 0 is always less than the number of comparators in this group. The extra comparators can be left idle.

[0154] Starting with Cycle2, the even-array comparators in the 2nd, 4th, 6th... even-arrays select the largest (the largest in the 2nd group, the second largest in the 4th group, the third largest in the 6th group, and so on) thraed ID that is 0 from the coalesce_matched_bit output of the previous cycle and use it as the based thread ID. The other thread IDs are used as the match thraed IDs. When all the coalesce_matched_bits output by a cycle are 0, it indicates that the merging operation for this instruction is complete.

[0155] Since W comparators perform merging operations in one cycle, a thread ID can be both a based thread ID and a match thread ID in the same cycle. Consequently, the cachelinesize based memory address output by multiple comparators may be duplicated. In this case, the merging information of the smaller comparator group can be used, because a smaller comparator group contains a more comprehensive total number of thread IDs.

[0156] Example 1:

[0157] The advantage of the above-described thread-id-based configuration is that it allows for faster coalescing of consecutive memory address patterns. The following example illustrates this with W (number of comparators configured in hardware) = 2, M = 32, cacheline size R = 128 bytes, and request length = 16 bytes.

[0158] If the M memory addresses are 0x10000, 0x10010, 0x10020, ..., 0x101F0, and R = 128 bytes, then every 8 threads can be merged together, resulting in a total of Q = 4 cacheline-size based memory addresses, which are 0x10000, 0x10080, 0x10100, and 0x10180.

[0159] Following the above thread ID-based configuration, the comparator compares thread 0 with all threads. Cycle1 will find that the addresses of thread 0 and threads 1, 2, 3, 4, 5, 6, and 7 can be matched in the same cache line, and the output cache line address is 0x10000; the addresses of thread 31 and threads 30, 29, 28, 27, 26, 25, and 24 can be matched in the same cache line, and the output cache line address is 0x10180. The coalesce_matched_bit input to cycle2 is 00000000 11111111 111111100000000; therefore, the based thread IDs of the two comparators in cycle2 are thread8 and thread23 respectively. We find that the addresses of thread 8 and threads 9, 10, 11, 12, 13, 14, and 15 can be matched in the same cache line, with the output cache line address being 0x10080; the addresses of thread 23 and threads 22, 21, 20, 19, 18, 17, and 16 can be matched in the same cache line, with the output cache line address being 0x10100. Since all coalesce_matched_bits output by cycle2 are 0, the merging operation ends, and T = 2.

[0160] If the thread ID is not set according to the above method, T is likely to be greater than 2, resulting in low merging efficiency.

[0161] Example 2:

[0162] If W=2, M=32, cacheline size R=128 bytes, and request length=4 bytes; and the M memory addresses are 0x10000, 0x10004, 0x10008, ..., according to the above-mentioned thread-id-based configuration, cycle1 will find that thread 0 and threads 1-31 all match in the same cache line, with the output cache line address being 0x10000; thread 31 and threads 30-1 all match in the same cache line, with the output cache line address also being 0x10000. In this case, the merged information with the smaller comparator group can be used, which is thread 0-31.

[0163] The request coalesce module sends the merged coalesced_cache_request to the normal cache and redundancy SRAM. The normal cache is a common group-related cache, assuming there are B banks, S sets, and U ways. The redundancy SRAM is mainly used to solve deadlock problems in in-order and partially in-order return methods; if all accessed objects in the system are out-of-order, this module is not needed.

[0164] A normal cache line can be uniquely identified using {bank id, set id, way id}.

[0165] Redundancy SRAM contains multiple entries, and each entry can store data for one cacheline. It can be considered as a cache with bank num=1, set num=1, and way num=the total number of entries. Redundancy SRAM is organized at the cacheline level and uses a similar encoding method to normal cache to uniquely identify entries in redundancy SRAM.

[0166] For example, if U is not a power of 2, the way id encoding can be used to number the entries in the redundancy SRAM; if U is a power of 2, all bits of the {bank id, set id, way id} in the normal cache are occupied, and one bit needs to be added to identify the entries in the redundancy SRAM.

[0167] Taking B=4, S=4, U=16 as an example, since U is 2 to the power of n, one bit needs to be added to uniquely identify a cacheline, such as using {redundancy id, bank id, set id, way id}, where redundancy id is 1 bit and bank id is 2 bits; if redundancy id is 1 and bank id is 00, then combining {redundancy id, bank id} can be considered as redundancy SRAM is bank4(100).

[0168] The coalesced_cache_request_add_info sent to downstream modules needs to be supplemented with the unique identifier information (storage address of the merge request) of the cache line of the normal cache / edundancy SRAM mentioned above, so that when the memory merge cache receives the access data returned by the downstream module, it knows which cache line to return the access data to.

[0169] Deadlock issue description regarding in-order and partial-in-order return methods: Since in-order and partial-in-order return methods require that all read data from the M threads within the instruction be collected before being returned to the accessed object, for a read request, if the number of cacheline size-based memory addresses Q after merging the M threads is greater than the way number U, and the Q coalesced_cache_requests happen to fall into the same set in the same bank, then the normal cache in the request direction cannot receive all coalesced_cache_requests, and the return direction cannot return to the upstream because not all read data has been collected, leading to a deadlock. The conventional approach is to ensure that Q <= U for any read instruction, thus avoiding deadlock.

[0170] The redundancy SRAM proposed in this embodiment can solve the deadlock problem mentioned above. If the above problem occurs (Q coalesced_cache_requests of a read instruction happen to fall into the same set of the same bank of the normal cache, and Q>U), then the remaining Q–U coalesced_cache_requests are allowed to enter the redundancy SRAM to occupy QU entries. After these Q–U access data are returned, they will be temporarily stored in the redundancy SRAM, thus solving the deadlock problem mentioned above.

[0171] The number of entries in redundancy SRAM should be at least Q_MAX–U. The more entries there are, the more instructions that can be covered in the above scenarios. The number of entries can be set according to the frequency of the above scenarios in actual applications. When all cacheline entries of redundancy SRAM are occupied, backpressure access to objects is required.

[0172] Each normal cache's bank and redundant SRAM are configured with a request FIFO to record cacheline set / way information and corresponding raw init_thread_request_add_info information for received coalesced_cache_requests. The request FIFO configured for the normal cache is... Figure 3 The first data buffer in the redundancy SRAM is set to request FIFO. Figure 3 The second data cache in the system. Simultaneously, if a master with out-of-order return type exists in the system, an out-of-order return FIFO (two levels deep are sufficient; the normal cache arbitrates multiple ready memory access requests to determine which request's data to return; the first level stores the i-th matching result; the second level stores the (i+1)-th matching result) needs to be set in each bank of the normal cache to temporarily store the arbitration results of the coalesced_cache_requests with out-of-order return type from the S sets. The out-of-order return FIFO is... Figure 3 The third data cache in the system.

[0173] In a request FIFO, entries are allocated sequentially (push), but can be released out of order (release).

[0174] For read requests with out-of-order return type, when the accessed data is ready (cache hit or access data returned by the downstream module), the accessed data is allocated to the out-of-order return FIFO, and at this time the corresponding entry in the request FIFO can be released.

[0175] For read requests with in-order / partial in-order return types, when the accessed data is ready, the corresponding entry in the request FIFO (request bitmap) is first notified that the accessed data is ready. Simultaneously, the accessed data readiness information of the corresponding instruction in the request map of the return data handler is updated. Since a cache line may be used by multiple instructions, all entries in the request FIFO that use this cache line need to be notified, and all instructions in the request map of the return data handler that use this cache line need to be updated.

[0176] The return data handler (which includes the request map and the in-order return data pack) is used to process the accessed data of the read request.

[0177] The request map is used to record the correspondence between `init_thread_request_add_info` and `coalesced_cache_request`. Specifically, it includes at least: instruction information for `{master_id, instruction_id, order_type}`, and information on the number and read data ready of `coalesced_cache_request`s in each bank for this instruction, as well as information on the number and read data ready of `coalesced_cache_request`s in redundancy SRAM.

[0178] The in-order return data pack is used to collect and assemble access data for in-order / partial in-order return types. This step is initiated after all return data for an instruction is ready: it reads the access data for the corresponding instruction from each bank in the normal cache and the redundancy SRAM, packages it together, and returns it to the upstream module; after the corresponding bank and redundancy SRAM send the cached line to the return data handler, the corresponding entry in the request FIFO can be released.

[0179] For out-of-order return type access data, the return data handler can directly arbitrate the access data of a request from the out-of-order return FIFO of the B banks in the normal cache and return it to the upstream module, without going through the in-order return data pack step.

[0180] For access data with in-order or partial in-order return types, the read data ready information for all coalesced_cache_requests recorded in the request map for this instruction must be set to valid before the step of returning access data to the upstream module (entering the in-order return data pack) can be initiated. Specifically, in-order return type instructions must ensure complete order preservation among instructions, while partial in-order return type instructions can enter the in-order return data pack step as long as all access data is ready.

[0181] This disclosure includes at least the following points to be protected:

[0182] 1. The memory coalescing cache mainly consists of modules such as request coalesce, normal cache, redundancy SRAM, and return data handler. Among them, redundancy SRAM is an optional module.

[0183] 2. The init_thread_request_add_info sent to the request coalesce module is based on init_thread_request and at least adds the identifiers {master_id, instruction_id, order_type}.

[0184] 3. For the 1st, 3rd, 5th... odd-numbered comparators, the smaller thread ID is used as the based thread ID for cycle 1; for the 2nd, 4th, 6th... even-numbered comparators, the larger thread ID is used as the based thread ID for cycle 1. The request coalesce submodule has a total of W comparators, requiring a total of M-1 + M-2 + ... = W*M – (1 + W)*W / 2 comparators.

[0185] 4. Set an M-bit `coalesce_matched_bit`, initially all values ​​are 0. When a thread is matched in a cycle, `coalesce_matched_bit` is marked as 1 (0 and 1 are example values; other values ​​can be used). Since the based thread ID of each group also participates in the comparison, its corresponding `coalesce_matched_bit` is also marked as 1. When all `coalesce_matched_bit` values ​​output in a cycle are 0, it indicates that the merging operation for this instruction is complete.

[0186] 5. After each cycle, `coalesce_matched_bit` is updated, and it becomes the input for the next cycle. Subsequent cycles only perform merging operations on threads with `coalesce_matched_bit` equal to 0. Starting from Cycle2, the 1st, 3rd, 5th... odd-numbered comparators select the smallest (the smallest in group 1, the second smallest in group 3, the third smallest in group 5, and so on) thraed ID that is 0 from the `coalesce_matched_bit` output of the previous cycle as the based thread ID. The other thread IDs are used as the match thraed IDs. The number of remaining thraed IDs that are 0 is always less than the number of comparators in the current group. The extra comparators can be left idle. Starting with Cycle2, the even array comparators for the 2nd, 4th, 6th... cycles select the largest (the 2nd group corresponds to the largest, the 4th group corresponds to the 2nd largest, the 6th group corresponds to the 3rd largest, and so on) thraed ID that is 0 from the coalesce_matched_bit output of the previous cycle as the based thread ID, and use the other thread IDs as the matchthraed ID.

[0187] 6. If a deadlock occurs where "Q coalesced_cache_requests from a read instruction happen to fall into the same set in the same bank of the normal cache, and Q > U", then the remaining Q–U coalesced_cache_requests are moved into the redundancy SRAM, occupying QU cacheline entries. These Q–U read data will be temporarily stored in the redundancy SRAM after they are returned, thus resolving the deadlock problem. The number of cacheline entries in the redundancy SRAM must be at least Q_MAX-U.

[0188] 7. A request FIFO is set up in each normal cache bank and redundant SRAM to record the cacheline set / way information of received coalesced_cache_requests and the corresponding original init_thread_request_add_info information. Additionally, if an out-of-order return type master exists in the system, an out-of-order return FIFO (two levels deep) needs to be set up in each bank of the normal cache to temporarily store the arbitration results of out-of-order return type coalesced_cache_requests in S sets where access data is ready.

[0189] 8. In the request FIFO, entries are pushed sequentially, but can be released out of order. For read requests with out-of-order return types, when the accessed data is ready (cache hit or access data returned by the downstream module), the read data is pushed to the out-of-order return FIFO, at which point the corresponding entry in the request FIFO can be released.

[0190] 9. For read requests with in-order / partial in-order return types, when the accessed data is ready, first notify the corresponding entry in the request FIFO that the accessed data is ready, and simultaneously update the accessed data ready information of the corresponding instruction in the request map of the return data handler. Since a cache line may be used by multiple instructions, it is necessary to notify all entries in the request FIFO that use this cache line, and update all instructions in the request map of the return data handler that use this cache line.

[0191] 10. The return data handler (which includes the request map and the in-order return data pack) is used to process the accessed data of the read request.

[0192] 11. The request map is used to record the correspondence between init_thread_request_add_info and coalesced_cache_request; specifically, it includes at least the instruction information of {master_id, instruction_id, order_type}, {the number of coalesced_cache_requests and read data ready information of each bank where this instruction falls, and the number of coalesced_cache_requests and read data ready information of those falling in redundancy SRAM}.

[0193] 12. The in-order return data pack is used to collect and assemble access data of in-order / partial in-order return types. This step is initiated after all access data for an instruction is ready: the access data of the corresponding instruction is read from each bank and redundancy SRAM in the normal cache, packaged together and returned to the upstream module; after the corresponding bank and redundancy SRAM send the access data to the return data handler, the corresponding entry in the request FIFO can be released.

[0194] 13. For out-of-order return type access data, the return data handler can directly arbitrate the access data of a request from the out-of-order return FIFO of the B banks in the normal cache and return it to the upstream module, without going through the in-order return data pack step.

[0195] 14. For in-order / partial in-order return type access data, the readiness information of all coalesced_cache_requests for this instruction recorded in the requestmap must be set to valid before the step of returning access data upstream (entering the in-order return data pack) can be initiated. Specifically, in-order return type instructions must ensure complete order preservation among instructions, while partial in-order return type instructions can enter the in-order return data pack step as long as all data is ready.

[0196] The embodiments disclosed herein can achieve at least the following technical effects:

[0197] 1. Partial in order ensures that all access data corresponding to M threads within an instruction is collected before being returned to the access object, saving the number of interactions with the access object; at the same time, since the instructions are out of order, there is no bottleneck effect between instructions.

[0198] 2. This disclosure proposes a novel memory merging cache, which can use a single memory merging cache to support upstream modules with multiple memory access requirements, including in-order, partial in-order, and out-of-order, thus saving hardware overhead.

[0199] 3. The based thread ID setting method of this embodiment can complete the coalesce operation faster for consecutive memory address patterns.

[0200] Based on the foregoing embodiments, this disclosure provides a memory access request processing device, which includes the included units and the modules included in each unit, and can be implemented by a processor in a computer device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0201] Figure 4 This is a schematic diagram of the composition structure of a memory access request processing device provided in an embodiment of the present disclosure, as shown below. Figure 4 As shown, the memory access request processing device 400 includes: an obtaining module 410, a merging module 420, and a processing module 430, wherein:

[0202] The obtaining module 410 is configured to obtain a first memory access instruction to be processed; the first memory access instruction includes a plurality of first memory access requests;

[0203] The merging module 420 is configured to merge the multiple first memory access requests based on the matching relationship between the access addresses of the multiple first memory access requests and the cache lines, to obtain at least one merged request;

[0204] The processing module 430 is configured to process the first memory access instruction based on the correspondence between the first memory access request and the merge request, the return type of the first memory access instruction, and the merge request; the return type of the first memory access instruction represents the return method of the access data between the first memory access instructions and the return method of the access data between the first memory access requests within the first memory access instruction.

[0205] In some embodiments, the obtaining module 410 is specifically configured to: receive memory access instructions sent by multiple objects; determine the target return type to be processed from the return types of the multiple memory access instructions; and take the memory access instruction corresponding to the target return type as the first memory access instruction to be processed.

[0206] In some embodiments, the merging module 420 is specifically configured to: divide the plurality of first memory access requests into at least one request group in the same cache line according to the access address of the first memory access request; and merge the plurality of first memory access requests in each request group into one request to obtain at least one merged request.

[0207] In some embodiments, the merging module 420 is specifically configured to: perform comparator matching on the access addresses of the plurality of first memory access requests until all first memory access requests have been matched to obtain at least one request group; the comparator matching includes: determining the base address of the comparator based on the access addresses of the plurality of first memory access requests; matching the access addresses of the plurality of first memory access requests with the base address of the comparator; if the matching is successful, grouping the second memory access requests that are in the same cache line as the base address of the comparator into a request group in the same cache line; if the matching fails, determining a new base address of the comparator based on the access addresses of the unmatched first memory access requests.

[0208] In some embodiments, the method for determining the base address of the comparator includes at least: using the access address of the first memory access request with the largest unmatched identifier as the base address of the comparator; using the access address of the first memory access request with the smallest unmatched identifier as the base address of the comparator; using the access address of the first memory access request with the smallest unmatched identifier as the base address of the i*2-1 group of comparators; and using the access address of the first memory access request with the largest unmatched identifier as the base address of the i*2 group of comparators; where i is a positive integer.

[0209] In some embodiments, the merging module 420 is specifically configured to: determine a first number of first memory access requests that a single cache line can contain based on the address length of the first memory access request and the size of a single cache line; match the access addresses of the plurality of first memory access requests with the base address of the comparator to obtain a first number of second memory access requests that are in the same cache line as the base address of the comparator.

[0210] In some embodiments, the merging module 420 is specifically configured to: set a matching identifier for each of the plurality of first memory access requests; and update the matching identifier of any first memory access request from the first identifier to the second identifier when any of the plurality of first memory access requests is matched.

[0211] In some embodiments, the merging module 420 is specifically configured to: when there is a target memory access request whose access address is the base address in any group of comparators and matches the base address of other groups of comparators besides the any group of comparators, determine that the matching result of the any group of comparators is duplicated with the matching result of the other comparators; and select the matching result of the comparator with the lower group from the matching result of the any group of comparators and the matching result of the other groups of comparators.

[0212] In some embodiments, the processing module 430 is specifically configured to: store the merge request and record the storage address of the merge request; obtain the access data of the merge request and store the access data of the merge request into the corresponding cache line based on the storage address of the merge request; and return the access data of the first memory access instruction to the corresponding object based on the storage address of the merge request, the correspondence and the return type of the first memory access instruction.

[0213] In some embodiments, the processing module 430 is specifically configured to: store the merge request in the first storage module when the number of merge requests matches the number of cache lines in each group of the first storage module; the first storage module is a group-related cache; use the bank identifier, group identifier, and road identifier of the cache line storing the merge request as the storage address of the merge request; record the storage address of the merge request and the multiple first memory access requests corresponding to the merge request as entries in the first data cache corresponding to the merge request; each bank in the first storage module corresponds to one first data cache.

[0214] In some embodiments, the processing module 430 is specifically configured to: when the number of merge requests does not match the number of cache lines in each group of the first storage module, store a second number of first merge requests in the first storage module and a third number of second merge requests in the second storage module; the first storage module is a group-related cache, and the second storage module includes multiple entries, each entry being able to store data of the same size as a cache line; the second number of first merge requests and the third number of second merge requests are equal to the merge request; the storage address of the first merge request is recorded in the first data cache corresponding to the first storage module, and the storage address of the second merge request is recorded in the second data cache corresponding to the second storage module; each bank in the first storage module corresponds to one first data cache; and the second storage module corresponds to one second data cache.

[0215] In some embodiments, the processing module 430 is specifically configured to: use the bank identifier, group identifier, and road identifier of the cache line storing the first merge request as the first storage address of the merge request, and use the identifier of the entry storing the second merge request as the second storage address of the merge request; record the first storage address and the multiple first memory access requests corresponding to the first merge request as entries in the first data cache; and record the second storage address and the multiple first memory access requests corresponding to the second merge request as entries in the second data cache.

[0216] In some embodiments, the processing module 430 is specifically configured to: when the return type of the first memory access instruction is a first type and there is a third merge request with access data ready, perform a first return operation on the access data of the third merge request until all access data of the first memory access instruction has been returned to the corresponding object; the first type indicates that the return order of access data between the first memory access instructions is independent of the receiving order between the first memory access instructions, and the return order of access data between the first memory access requests is independent of the receiving order between the first memory access requests; the first return operation includes: determining multiple third memory access requests corresponding to the third merge request based on the correspondence; and returning the access data of the multiple third memory access requests to the corresponding object based on the storage address of the merge request.

[0217] In some embodiments, the processing module 430 is specifically configured to: perform a second return operation on access data for each third memory access request based on the storage address of the merge request, until the access data of the plurality of third memory access requests has been returned to the corresponding object; the second return operation includes: transferring the access data of each third memory access request from the cache line corresponding to each third memory access request to the third data cache corresponding to each third memory access request based on the storage address of the merge request, so as to return the access data of each third memory access request to the corresponding object through the third data cache; each bank in the first storage module corresponds to a third data cache, and the first type indicates that the return order of access data is independent of the receiving order of the first memory access request; releasing the data about each third memory access request in the first data cache.

[0218] In some embodiments, the processing module 430 is specifically configured to: when the return type of the first memory access instruction is a second type and there is a third merge request for which access data is ready, perform a third return operation on the third merge request until all access data of the first memory access instruction has been returned to the corresponding object; the second type indicates that the return order of access data between the first memory access instructions is independent of the receiving order between the first memory access instructions, and the return order of access data between the first memory access requests is the same as the receiving order between the first memory access requests; the third return operation includes: setting the entry of the third merge request in the first data buffer to a third identifier; when there is a second memory access instruction for which access data of all included memory access requests is ready, setting the identifier of the second memory access instruction in the request bitmap to a fourth identifier; the fourth identifier indicates that the access data of the second memory access instruction is ready; and returning the access data of the second memory access instruction to the corresponding object according to the receiving order of the second memory access instructions.

[0219] In some embodiments, the processing module 430 is specifically configured to: when the return type of the first memory access instruction is a third type and there is a third merge request where the access data is ready, set the entry of the third merge request in the first data buffer to a third identifier; the third type indicates that the return order of the access data between the first memory access instructions is the same as the receiving order between the first memory access instructions, and the return order of the access data between the first memory access requests is the same as the receiving order between the first memory access requests; when there is a second memory access instruction where the access data of all included memory access requests is ready, set the identifier of the second memory access instruction in the request bitmap to a fourth identifier; the fourth identifier indicates that the access data of the second memory access instruction is ready; when it is determined from the request bitmap that the access data of all memory access instructions in the first memory access instruction is ready, return the access data of the first memory access instruction to the corresponding object according to the receiving order of the first memory access instruction and the receiving order of the first memory access request.

[0220] The descriptions of the apparatus embodiments above are similar to those of the method embodiments above, and have similar beneficial effects. In some embodiments, the functions or modules included in the apparatus provided in this disclosure can be used to perform the methods described in the method embodiments above. For technical details not disclosed in the apparatus embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0221] It should be noted that, in the embodiments of this disclosure, if the above-described memory access request processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this disclosure are not limited to any specific hardware, software, or firmware, or any combination of hardware, software, and firmware.

[0222] This disclosure provides a computer device including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements some or all of the steps in the above-described method.

[0223] This disclosure provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the above-described method. The computer-readable storage medium may be transient or non-transient.

[0224] This disclosure provides a computer program including computer-readable code, wherein when the computer-readable code is executed in a computer device, a processor in the computer device performs some or all of the steps in the above-described method.

[0225] This disclosure provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0226] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referenced interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0227] It should be noted that, Figure 5 This is a schematic diagram of a hardware entity of a computer device in an embodiment of this disclosure, such as... Figure 5 As shown, the hardware entity of the computer device 500 includes: a processor 501, a communication interface 502, and a memory 503, wherein:

[0228] Processor 501 typically controls the overall operation of computer device 500.

[0229] Communication interface 502 enables computer devices to communicate with other terminals or servers via a network.

[0230] The memory 503 is configured to store instructions and applications executable by the processor 501, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 501 and various modules in the computer device 500. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 501, the communication interface 502, and the memory 503 can be performed via bus 504.

[0231] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above embodiments of this disclosure are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0232] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0233] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0234] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0235] In addition, each functional unit in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0236] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0237] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0238] The above description is merely an embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A memory access request processing method, characterized by, The memory access request processing method includes: Obtain the first memory access instruction to be processed; the first memory access instruction includes multiple first memory access requests; Based on the fact that the access addresses of the multiple first memory access requests are in the same cache line, the multiple first memory access requests corresponding to the same cache line are merged to obtain at least one merged request; Based on the correspondence between the first memory access request and the merge request, the return type of the first memory access instruction, and the merge request, the first memory access instruction is processed; the return type of the first memory access instruction indicates that the return order of the access data between the first memory access instructions is independent of or the same as the receiving order between the memory access instructions, and that the return order of the access data between the first memory access requests within the first memory access instruction is independent of or the same as the receiving order between the memory access requests.

2. The memory access request processing method of claim 1, wherein, The step of obtaining the first memory access instruction to be processed includes: Receive memory access instructions sent by multiple objects; Determine the target return type to be processed from the return types of the multiple memory access instructions; The memory access instruction corresponding to the target return type is taken as the first memory access instruction to be processed.

3. The memory access request processing method according to claim 1 or 2, characterized by, The step of merging multiple first memory access requests corresponding to the same cache line according to their access addresses to obtain at least one merged request includes: According to the access address of the first memory access request, the plurality of first memory access requests are divided into at least one request group in the same cache line; Multiple first memory access requests in each of the request groups are merged into one request to obtain at least one merged request.

4. The memory access request processing method according to claim 3, wherein The step of dividing the plurality of first memory access requests into at least one request group in the same cache line according to the access address of the first memory access request includes: A comparator is used to match the access addresses of the plurality of first memory access requests until all first memory access requests have been matched, resulting in at least one request group. The comparator's matching includes: The base address of the comparator is determined based on the access addresses of the plurality of first memory access requests; The access addresses of the plurality of first memory access requests are matched with the base address of the comparator; If a match is found, the second memory access request that is in the same cache line as the base address of the comparator is treated as a request group in the same cache line. In the event of a failed match, a new base address for the comparator is determined based on the access address of the first memory access request that was not matched.

5. The memory access request processing method according to claim 4, characterized in that, The method for determining the base address of the comparator includes at least: The access address of the first memory access request with the largest unmatched identifier is used as the base address of the comparator; The access address of the first memory access request with the smallest unmatched identifier is used as the base address of the comparator; The address of the first memory access request with the smallest unmatched identifier is used as the base address of the i*2-1th comparator group, and the address of the first memory access request with the largest unmatched identifier is used as the base address of the i*2th comparator group; i is a positive integer.

6. The memory access request processing method according to claim 4, characterized in that, The step of matching the access addresses of the plurality of first memory access requests with the base address of the comparator includes: Based on the address length of the first memory access request and the size of a single cache line, determine the first number of first memory access requests that the single cache line can contain; The access addresses of the plurality of first memory access requests are matched with the base address of the comparator to obtain a first number of second memory access requests that are in the same cache line as the base address of the comparator.

7. The memory access request processing method according to claim 4, characterized in that, The memory access request processing method further includes: A matching identifier is set for each of the plurality of first memory access requests; If any of the plurality of first memory access requests is matched, the matching identifier of the first memory access request is updated from the first identifier to the second identifier.

8. The memory access request processing method according to claim 5, characterized in that, The memory access request processing method further includes: If there is a target memory access request whose access address is the base address in any set of comparators and matches the base address of other sets of comparators besides the aforementioned set of comparators, it is determined that the matching result of the aforementioned set of comparators is duplicated with the matching result of the other comparators. From the matching results of any one set of comparators and the matching results of the other sets of comparators, select the matching result of the comparator with the lower group.

9. The memory access request processing method according to any one of claims 1, 2, or 4 to 8, characterized in that, The processing of the first memory access instruction based on the correspondence between the first memory access request and the merge request, the return type of the first memory access instruction, and the merge request includes: Store the merge request and record the storage address of the merge request; Obtain the access data of the merge request, and store the access data of the merge request in the corresponding cache line based on the storage address of the merge request; Based on the storage address of the merge request, the correspondence, and the return type of the first memory access instruction, the access data of the first memory access instruction is returned to the corresponding object.

10. The memory access request processing method according to claim 9, characterized in that, The storage of the merge request and the recording of the storage address of the merge request include: If the number of merge requests matches the number of cache lines in each group in the first storage module, the merge requests are stored in the first storage module; the first storage module is a group-related cache. The bank identifier, group identifier, and road identifier that store the cache line containing the merge request are used as the storage address of the merge request; The storage address of the merge request and the multiple first memory access requests corresponding to the merge request are recorded in the entries of the first data cache corresponding to the merge request; each bank in the first storage module corresponds to one first data cache.

11. The memory access request processing method according to claim 9, characterized in that, The storage of the merge request and the recording of the storage address of the merge request include: If the number of merge requests does not match the number of cache lines in each group of the first storage module, a second number of first merge requests are stored in the first storage module, and a third number of second merge requests are stored in the second storage module; the first storage module is a group-related cache, and the second storage module includes multiple entries, each entry being able to store data of the same size as a cache line; the second number of first merge requests and the third number of second merge requests are equal to the number of merge requests. The storage address of the first merge request is recorded in the first data cache corresponding to the first storage module, and the storage address of the second merge request is recorded in the second data cache corresponding to the second storage module; each bank in the first storage module corresponds to a first data cache; and each second storage module corresponds to a second data cache.

12. The memory access request processing method according to claim 11, characterized in that, The step of recording the storage address of the first merge request in the first data cache corresponding to the first storage module, and recording the storage address of the second merge request in the second data cache corresponding to the second storage module, includes: The bank identifier, group identifier, and road identifier of the cache line containing the first merge request are used as the first storage address of the merge request, and the identifier of the entry containing the second merge request is used as the second storage address of the merge request. The entries corresponding to the first storage address and the first merge request are recorded in the first data cache. The entries for the second storage address and the multiple first memory access requests corresponding to the second merge request are recorded in the second data cache.

13. The memory access request processing method according to claim 9, characterized in that, The step of returning the access data of the first memory access instruction to the corresponding object based on the storage address of the merge request, the correspondence, and the return type of the first memory access instruction includes: When the return type of the first memory access instruction is the first type and there is a third merge request with access data ready, the first return operation of access data is performed on the third merge request until all access data of the first memory access instruction has been returned to the corresponding object; the first type indicates that the return order of access data between the first memory access instructions is independent of the receiving order between the first memory access instructions, and the return order of access data between the first memory access requests is independent of the receiving order of the first memory access requests. The first return operation includes: Based on the correspondence, multiple third memory access requests corresponding to the third merge request are determined; Based on the storage address of the merge request, the access data of the plurality of third memory access requests are returned to the corresponding object.

14. The memory access request processing method according to claim 13, characterized in that, The step of returning the access data of the multiple third memory access requests to the corresponding object based on the storage address of the merge request includes: Based on the storage address of the merge request, perform a second return operation on the access data for each third memory access request until the access data of the multiple third memory access requests have been returned to the corresponding object; The second return operation includes: Based on the storage address of the merge request, the access data of each third memory access request is transferred from the cache line corresponding to each third memory access request to the third data cache corresponding to each third memory access request, so as to return the access data of each third memory access request to the corresponding object through the third data cache; each bank in the first storage module corresponds to one third data cache; Release the data in the first data cache related to each of the third memory access requests.

15. The memory access request processing method according to claim 9, characterized in that, The step of returning the access data of the first memory access instruction to the corresponding object based on the storage address of the merge request, the correspondence, and the return type of the first memory access instruction includes: If the return type of the first memory access instruction is the second type and there is a third merge request where the access data is ready, the third return operation of the access data is performed on the third merge request until all the access data of the first memory access instruction has been returned to the corresponding object; the second type indicates that the return order of the access data between the first memory access instructions is independent of the receiving order between the first memory access instructions, and the return order of the access data between the first memory access requests is the same as the receiving order between the first memory access requests. The third return operation includes: Set the entry of the third merge request in the first data cache to the third identifier; In the case of a second memory access instruction where the access data for all memory access requests is ready, the identifier of the second memory access instruction in the request bitmap is set to a fourth identifier; the fourth identifier indicates that the access data for the second memory access instruction is ready. According to the order in which the second memory access instructions are received, the access data of the second memory access instructions is returned to the corresponding object.

16. The memory access request processing method according to claim 9, characterized in that, The step of returning the access data of the first memory access instruction to the corresponding object based on the storage address of the merge request, the correspondence, and the return type of the first memory access instruction includes: If the return type of the first memory access instruction is the third type and there is a third merge request where the access data is ready, the entry of the third merge request in the first data buffer is set to the third identifier; the third type indicates that the return order of the access data between the first memory access instructions is the same as the receiving order between the first memory access instructions, and the return order of the access data between the first memory access requests is the same as the receiving order between the first memory access requests. In the case of a second memory access instruction where the access data for all memory access requests is ready, the identifier of the second memory access instruction in the request bitmap is set to a fourth identifier; the fourth identifier indicates that the access data for the second memory access instruction is ready. If, based on the request bitmap, it is determined that the access data for all memory access instructions in the first memory access instruction is ready, the access data for the first memory access instruction is returned to the corresponding object according to the receiving order of the first memory access instruction and the receiving order of the first memory access request.

17. A memory access request processing apparatus, characterized in that, The memory access request processing device includes: The module is configured to acquire the first memory access instruction to be processed; the first memory access instruction includes a plurality of first memory access requests. The merging module is configured to merge multiple first memory access requests corresponding to the same cache line according to the access addresses of the multiple first memory access requests, and obtain at least one merged request. The processing module is configured to process the first memory access instruction based on the correspondence between the first memory access request and the merge request, the return type of the first memory access instruction, and the merge request; the return type of the first memory access instruction indicates that the return order of the access data between the first memory access instructions is independent of or the same as the receiving order between the memory access instructions, and that the return order of the access data between the first memory access requests within the first memory access instruction is independent of or the same as the receiving order of the memory access requests.

18. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 16.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 16.

20. A computer program product, the computer program product comprising a non-transitory computer-readable storage medium storing a computer program, characterized in that, When the computer program is read and executed by a computer, it implements the steps of the method according to any one of claims 1 to 16.

Images