Memory merging device, data processing method, chip product and computer equipment

Abstract

Embodiments of the present disclosure provide a memory merging device, a data processing method, a chip product and a computer device. The memory merging device comprises: a request merging module configured to send a read data request corresponding to a read data instruction to a storage module, and add the read data request to a corresponding request queue; a plurality of data checking modules, each of which is configured to select a first request queue from a plurality of corresponding request queues in a current clock cycle, and check a data ready state of at least one read data request starting from a head of the first request queue in the storage module; an instruction submitting module configured to select, from each request queue, a target request queue in which a data ready state of all read data requests corresponding to a read data instruction of a read data request at a head of the target request queue is ready; and a data returning module configured to return target data of the read data instruction corresponding to the read data request at the head of the target request queue to a corresponding sender. In this way, the hardware area can be reduced.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of chip technology, and in particular to a memory merging device, a data processing method, a chip product, and a computer device. Background Technology

[0002] In modern parallel computing chip designs such as GPUs (Graphics Processing Units) and AI (Artificial Intelligence), memory access efficiency plays a crucial role in overall system performance. To reduce the number of accesses to cache modules or downstream memory modules, memory merging technology can be used in GPU / AI chips. This technology uses a memory merging device to merge multiple original requests at the cache line granularity to generate requests (also known as cache requests) that are adapted to the parallel processing capabilities of the cache modules.

[0003] In related technologies, in order to meet high performance requirements (such as out-of-order scheduling capabilities), memory merging devices usually require a lot of hardware resources, resulting in a large area cost. Summary of the Invention

[0004] In view of this, the present disclosure provides at least one memory merging apparatus, a data processing method, a chip product, and a computer device.

[0005] The technical solution of this disclosure embodiment is implemented as follows: This disclosure provides a memory merging apparatus, including: The request merging module is used to: send at least one read data request corresponding to the received read data instruction to the storage module, and add at least one read data request to the request queue corresponding to the read data instruction; Multiple data checking modules, any one of which is used to: select a first request queue from multiple request queues corresponding to the data checking module within the current clock cycle, and check the data readiness status of at least one read data request in the storage module starting from the head of the first request queue; and update the queue status of the first request queue in the current clock cycle according to the data readiness status. The instruction submission module is used to: select the target request queue for the current clock cycle from each request queue; the queue state of the target request queue in the current clock cycle is the first state, which indicates that the data ready state of all read data requests corresponding to the read data request at the head of the corresponding request queue and the same read data instruction is ready; The data return module is used to obtain the target data of each read data request corresponding to the target read data instruction from the storage module and return the target data to the sender of the target read data instruction. The target read data instruction corresponds to the read data request at the head of the target request queue.

[0006] This disclosure provides a data processing method applied to a memory merging apparatus. The memory merging apparatus includes a request merging module, multiple data checking modules, an instruction submission module, and a data return module. The method includes: Based on the received read data instruction, the request merging module sends at least one read data request corresponding to the read data instruction to the storage module, and adds the at least one read data request to the request queue corresponding to the read data instruction; Multiple data checking modules select a first request queue from their respective multiple request queues within the current clock cycle, check the data readiness status of at least one read data request starting from the head of the first request queue in the storage module, and update the queue status of the first request queue in the current clock cycle based on the data readiness status. The instruction submission module selects the target request queue for the current clock cycle from each request queue; the queue state of the target request queue in the current clock cycle is the first state, which indicates that all read data requests corresponding to the read data request at the head of the corresponding request queue and the same read data instruction are ready. The data return module retrieves the target data for each read data request corresponding to the target read data instruction from the storage module and returns the target data to the sender of the target read data instruction. The target read data instruction corresponds to the read data request at the head of the target request queue.

[0007] This disclosure provides a chip product that includes the memory merging device described in the above embodiments.

[0008] This disclosure provides a computer device that includes the memory merging apparatus described in the above embodiments.

[0009] In this embodiment of the disclosure, the memory merging device includes a request merging module, multiple data checking modules, an instruction submission module, and a data return module. Each data checking module corresponds to multiple request queues. The request merging module sends at least one read data request corresponding to a received read data instruction to the storage module and adds the at least one read data request to the request queue corresponding to the read data instruction. Each of the multiple data checking modules selects a first request queue from its respective multiple request queues within the current clock cycle and checks the data readiness status of at least one read data request starting from the head of the first request queue in the storage module. The system checks the readiness status of the data request queue and updates the queue status of the first request queue in the current clock cycle based on the data readiness status. The instruction submission module selects the target request queue for the current clock cycle from each request queue. The target request queue is in the first state in the current clock cycle, which indicates that all read data requests corresponding to the read data request at the head of the corresponding request queue are ready. The data return module retrieves the target data for each read data request corresponding to the target read data instruction from the storage module and returns the target data to the sender of the target read data instruction. The target read data instruction corresponds to the read data request at the head of the target request queue. In this way, within a single clock cycle, each data checking module can arbitrate multiple corresponding request queues and select one request queue to check the data readiness status of at least one read data request starting from the head of the queue. In this way, read data instructions within the same request queue can return data in the same order, while read data instructions between different request queues can return data out of order. Compared with the related technologies where each request queue corresponds to a data inspection module, this approach reduces the number of data inspection modules while maintaining high out-of-order scheduling capability, thus meeting overall performance requirements with a smaller area cost.

[0010] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0012] Figure 1 A schematic diagram of the composition structure of a memory merging device provided in this embodiment of the present disclosure. Figure 1 ; Figure 2 A schematic diagram of the composition structure of a memory merging device provided in this embodiment of the present disclosure. Figure 2 ; Figure 3A schematic diagram of the composition structure of a data inspection module provided in this embodiment of the disclosure. Figure 1 ; Figure 4 A schematic diagram of the composition structure of a data inspection module provided in this embodiment of the disclosure. Figure 2 ; Figure 5 A schematic diagram of the composition structure of a data inspection module provided in this embodiment of the disclosure. Figure 3 ; Figure 6 A schematic diagram illustrating the implementation flow of a data processing method provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of the composition structure of a chip product provided in an embodiment of the present disclosure; Figure 8 This is a schematic diagram of the composition structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0014] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0015] The terms “first / second / third” are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that “first / second / third” may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this disclosure.

[0017] The memory merging device is located between the upstream request module (such as the computing unit, graphics processing unit, controller, etc.) and the memory module. It is used to merge the original memory access requests corresponding to the memory access instructions sent by the upstream request module at the cache line granularity, resulting in at least one cache request. The memory access instructions can be, for example, read data instructions or write data instructions, and the cache requests can be, for example, read data requests or write data requests. The merged cache requests are then sent to the cache module or, through the cache module, to the downstream memory module for processing. Finally, the processing results are sent back to the upstream request module. For data loading requests, which require the return of data, the storage module returns the read data, and the memory merging module packages the read data according to certain rules before returning it to the upstream request module. It should be noted that the memory merging device can also be called a memory merging system, memory merging module, memory merging unit, request merging system, request merging device, request merging unit, etc.

[0018] In related technologies, each memory access instruction carries a thread bundle identifier (Warp_id) in its instruction information. The Warp_id is used to identify the thread bundle to which the thread sending the memory access instruction belongs. In parallel computing on chips (such as GPU chips, AI chips, etc.), the thread bundle (Warp) is the basic unit of scheduling and execution in a stream multi-processor (SM).

[0019] For read data instruction types that require data return, there are generally three options for the data return method: all instructions are returned in full order (regardless of Warp_id), all instructions are returned in full out-of-order (regardless of Warp_id), and read data instructions are returned in full order by Warp_id (that is, read data instructions with the same Warp_id can return the corresponding read data in full order, and read data instructions with different Warp_ids can return the corresponding read data in out-of-order).

[0020] In chip design, to optimize power-performance-area (PPA), the order of warp IDs is generally chosen. Assuming the system has multiple warp IDs, in the hardware implementation, each warp ID typically corresponds to a logical or physical first-in-first-out (FIFO) queue, referred to below as isched_fifo. The `isched_fifo` stores all read requests corresponding to read commands falling under this `Warp_id`. When all read requests corresponding to a single read command are ready, the conditions for requesting command commit arbitration are met, and the command commit phase can begin, packaging and returning all read requests corresponding to that read command. When all read requests corresponding to the first read command in multiple `isched_fifo` queues are ready, one `isched_fifo` can be selected from among them for data return. In other words, a command commit module (Commit arb) is needed to arbitrate among multiple `isched_fifo` queues. The process of checking whether the read requests corresponding to a single read command are ready is performed by a data checking module, which typically executes in parallel within a memory merging device. The command commit module implements arbitration logic to determine which read command's read requests can return all read requests to the upstream request module.

[0021] In related technologies, multiple isched_fifos each have a data checking module, which is used to check whether all the data to be read by the read data instruction corresponding to the read data request at the head of the queue in the corresponding isched_fifo is ready. The logic of the data checking module generally requires a lot of hardware resources. If high performance requirements (such as high out-of-order scheduling capability) are to be met, a lot of data checking modules are usually required, which consumes more hardware resources and increases the area cost.

[0022] In view of this, embodiments of this disclosure provide a memory merging apparatus, such as... Figure 1 As shown, the memory merging device 10 includes: a request merging module 11, multiple data checking modules 12, an instruction submission module 13, and a data return module 14, wherein: The request merging module 11 is used to: send at least one read data request corresponding to the received read data instruction to the storage module 20 based on the received read data instruction, and add the at least one read data request to the request queue corresponding to the read data instruction.

[0023] Multiple data checking modules 12, any one of which is used to: select a first request queue from multiple request queues isched_fifo corresponding to the multiple data checking modules 12 within the current clock cycle, and check the data readiness status of at least one read data request in the storage module 20 starting from the head of the first request queue; and update the queue status of the first request queue in the current clock cycle according to the data readiness status. The instruction submission module 13 is used to: select the target request queue for the current clock cycle from each request queue; the queue state of the target request queue in the current clock cycle is the first state, and the first state indicates that the data ready state of all read data requests corresponding to the read data request at the head of the corresponding request queue and the same read data instruction is ready; The data return module 14 is used to obtain the target data of each read data request corresponding to the target read data instruction from the storage module, and return the target data to the sender of the target read data instruction. The target read data instruction corresponds to the read data request at the head of the target request queue.

[0024] Here, the upstream of the memory merging device 10 may include one or more requesters 30, and the sender of the read data instruction may be any one of the one or more requesters.

[0025] Each read data instruction can correspond to at least one read data request, and the at least one read data request corresponding to a read data instruction can be used to read data from the storage module. The storage module can be, but is not limited to, at least one of a cache module, a memory module, etc. The storage module can be a module inside the memory merging device or a module outside the memory merging device; this disclosure does not limit this. For example, the storage module may include a cache module inside the memory merging device.

[0026] In some implementations, the read data instruction may carry the starting address and length of the data to be read. Based on the starting address and length, at least one read data request can be determined, and each read data request can be used to read data from a cache line in the cache module.

[0027] In some implementations, the request merging module 11 can merge at least one original request at the cache line granularity based on the data start address and data length corresponding to at least one original request in the read data instruction, to obtain at least one read data request. Each read data request can be used to read data from a cache line in the cache module. It is understood that the sender of the read data instruction can have a parallel execution unit. The M thread tasks executed by the parallel execution unit at one time can be called an instruction. If a thread task is used to read data from the cache module, the corresponding instruction is a read data instruction. This read data instruction is used to read data corresponding to at most M original requests from the cache module. Here, M is a positive integer. That is, the above M thread tasks correspond one-to-one with the M original requests. The value of M can be preset, depending on the number of threads executing tasks in parallel in the parallel execution unit of the instruction sender, for example, M=4, 6, or 8, etc., which is not limited in this embodiment. It should be noted that M here is the maximum capability of an instruction; an instruction may have all M threads effective, or only a few of the M threads may be effective. Each original request has a corresponding data start address and data length. The data start address indicates the storage address of the data requested by the original request in the cache module; the data length indicates the length or amount of data requested by the original request.

[0028] The memory merging device can maintain multiple logical or physical request queues, each request queue being used to store at least one read data request corresponding to at least one pending read data instruction.

[0029] In some implementations, the request queue is used to store each read data request of at least one read data instruction to be processed, and the request merging module can add each read data request of the read data instruction to the request queue corresponding to the read data instruction.

[0030] In some implementations, the request queue is a first-in, first-out (FIFO) queue.

[0031] In some implementations, the read data instruction carries a thread bundle identifier, which identifies the thread bundle to which the thread sending the read data instruction belongs. Each thread bundle corresponds to a request queue, and a single request queue stores the read data requests for at least one read data instruction submitted by threads within the same thread bundle. The request merging module can also be used to: determine the thread bundle to which the thread submitting the read data instruction belongs; and store the read data requests for the read data instruction in the request queue corresponding to that thread bundle. Thus, since read data instructions submitted by threads within the same thread bundle are stored in the same request queue, read data instructions corresponding to the same thread bundle can return the corresponding read data in order, while read data instructions from different thread bundles can return the corresponding read data out of order.

[0032] Each data inspection module 12 can correspond to multiple request queues. The data inspection module 12 is used to select a first request queue from the corresponding multiple request queues within a single clock cycle, and check the data readiness status of at least one read data request starting from the head of the first request queue in the storage module 20, thereby obtaining the data inspection result for each read data request. The data inspection module 12 can use any suitable arbitration method to select the first request queue from the corresponding multiple request queues; this embodiment of the present disclosure does not limit this. For example, the data inspection module 12 can use a polling method to select the first request queue from the corresponding multiple request queues. Furthermore, the multiple request queues corresponding to the data inspection module 12 can have different priorities, and the data inspection module 12 can select the first request queue according to the priority.

[0033] In some implementations, within a single clock cycle, the data checking module 12 can check the data readiness status of the read data request at the head of the first request queue selected in the current cycle in the storage module 20. The data readiness status indicates whether the data requested by the corresponding read data request is ready in the storage module 20.

[0034] In some implementations, within a single clock cycle, the data inspection module 12 can check the data readiness status of a first number of read data requests, starting from the head of the first request queue, in the storage module 20. Here, the first number is an integer greater than 1, and those skilled in the art can set an appropriate first number according to the actual application scenario; no limitation is made here. For example, the first number can be 2, 4, or 8, etc.

[0035] Within the current clock cycle, after the data checking module 12 checks the data readiness status of at least one read data request starting from the head of the first request queue in the storage module 20, it can update the queue status of the first request queue in the current clock cycle based on the data readiness status of the at least one read data request. The queue status of the request queue indicates whether the data readiness status of all read data requests corresponding to the same read data instruction as the read data request at the head of the request queue is ready. Understandably, after the data inspection module 12 checks the data readiness status of at least one read data request starting from the head of the first request queue in the storage module 20, it can determine the read data instruction corresponding to the read data request at the head of the first request queue, and determine whether the data readiness status of all read data requests corresponding to the read data instruction is ready in the current clock cycle. If yes, the queue status of the first request queue in the current clock cycle is updated to the first state; if no, the queue status of the first request queue in the current clock cycle is updated to the second state. The first state indicates that the data readiness status of all read data requests corresponding to the same read data instruction as the read data request at the head of the corresponding request queue is ready, that is, the read data instruction corresponding to the read data request at the head of the corresponding request queue has the condition to return data. The second state indicates that the data readiness status of at least one read data request corresponding to the same read data instruction as the read data request at the head of the corresponding request queue is not ready.

[0036] The instruction submission module 13 can select the request queue with the first state in the current clock cycle from multiple request queues as the target request queue. If the target read data instruction corresponding to the read data request at the head of the target request queue has the condition to return data in the current clock cycle, the data return module 14 can obtain the target data requested by each read data request corresponding to the target read data instruction from the storage module in the current clock cycle, and return the obtained target data to the sender of the target read data instruction.

[0037] In some implementations, when there are multiple candidate request queues with a first state among multiple request queues, the instruction submission module 13 can select one or more target request queues from the multiple candidate request queues.

[0038] In this embodiment of the disclosure, the memory merging device includes a request merging module, multiple data checking modules, an instruction submission module, and a data return module. Each data checking module corresponds to multiple request queues. The request merging module sends at least one read data request corresponding to a received read data instruction to the storage module and adds the at least one read data request to the request queue corresponding to the read data instruction. Each of the multiple data checking modules selects a first request queue from its respective multiple request queues within the current clock cycle and checks the data readiness status of at least one read data request starting from the head of the first request queue in the storage module. The system checks the readiness status of the data request queue and updates the queue status of the first request queue in the current clock cycle based on the data readiness status. The instruction submission module selects the target request queue for the current clock cycle from each request queue. The target request queue is in the first state in the current clock cycle, which indicates that all read data requests corresponding to the read data request at the head of the corresponding request queue are ready. The data return module retrieves the target data for each read data request corresponding to the target read data instruction from the storage module and returns the target data to the sender of the target read data instruction. The target read data instruction corresponds to the read data request at the head of the target request queue. In this way, within a single clock cycle, each data checking module can arbitrate multiple corresponding request queues and select one request queue to check the data readiness status of at least one read data request starting from the head of the queue. In this way, read data instructions within the same request queue can return data in the same order, while read data instructions between different request queues can return data out of order. Compared with the related technologies where each request queue corresponds to a data inspection module, this approach reduces the number of data inspection modules while maintaining high out-of-order scheduling capability, thus meeting overall performance requirements with a smaller area cost.

[0039] In some embodiments, the data inspection module 12 is further configured to: Based on the data check result of at least one target read data request in the previous clock cycle, the first request queue to be checked in the current clock cycle is selected from the corresponding multiple request queues; the target read data request is the read data request for which the data check module 12 performed a data readiness status check in the previous clock cycle.

[0040] Here, within the current clock cycle, the data inspection module 12 can select the first request queue to be inspected in the current clock cycle from the multiple request queues corresponding to the data inspection module 12, based on the data inspection result of the target read data request for which the data readiness status was checked in the previous clock cycle.

[0041] For example, the data inspection module 12 can determine whether to continue to check the data readiness status of other read data requests corresponding to the same read data instruction as the target read data request in the current clock cycle based on the data inspection result of the target read data request that the data inspection module 12 performed a data readiness status check in the previous clock cycle. That is, whether the current clock cycle should continue to use the first request queue selected in the previous clock cycle as the first request queue to be checked in the current clock cycle.

[0042] In the above embodiments, within the current clock cycle, the data checking module selects the first request queue to be checked in the current clock cycle from the multiple request queues corresponding to the data checking module, based on the data check result of the target read data request that performed the data readiness status check in the previous clock cycle. Since the timing of storage, update, and other processing of each read data request corresponding to the same read data instruction in the storage module is usually close (for example, the hit status of each read data request corresponding to the same read data instruction in the cache module is likely to be the same, or the processing timing in the cache module is usually close), the data requested by each read data request corresponding to the same read data instruction is usually ready within a few close clock cycles. That is, if the data requested by a certain read data request corresponding to a certain read data instruction is ready in the current clock cycle, then the data requested by other read data requests corresponding to the same read data instruction is likely to be ready or will be ready within a few close clock cycles. In this way, by considering the data check results of the target read data request that was checked for data readiness status in the previous clock cycle, the data checking module selects the first request queue to be checked in the current clock cycle from the multiple request queues corresponding to the data checking module. This allows for the selection of a more suitable request queue for checking, thereby increasing the probability that the data checking module will check for a ready read data request and reducing the waiting delay caused by the unready data of the target read data request being checked. This improves the resource utilization and data return efficiency of the data checking module, and ultimately enhances the overall performance of the memory merging device.

[0043] In some embodiments, the data inspection module 12 is further configured to: If at least one target read data request has a first check result in the data check of the previous clock cycle, the first request queue to be checked in the current clock cycle is selected from the multiple request queues corresponding to the data check module according to the set arbitration strategy; the first check result indicates that the data read status of the corresponding target read data request is not ready.

[0044] The arbitration strategy employed here can be pre-set by those skilled in the art based on the actual application scenario, and this disclosure does not limit it. For example, the set arbitration strategy may include, but is not limited to, polling or selection according to priority.

[0045] Understandably, if, among the target read data requests that are checked for data readiness status in the first request queue selected in the previous clock cycle, at least one target read data request has a data check result that is the first check result, meaning that the data read data request for at least one target read data request was not ready in the previous clock cycle, then in the current clock cycle, the first request queue selected in the previous clock cycle will not be used again. Instead, according to the set arbitration strategy, the first request queue to be checked in the current clock cycle will be re-arbitrated from the multiple request queues corresponding to the data check module.

[0046] In the above embodiments, since the data requested by each read data request corresponding to the same read data instruction is usually ready within a few clock cycles, if the data readiness status of a certain target read data request was not ready in the previous clock cycle, the data requested by other read data requests corresponding to the same read data instruction as the target read data request will likely have to wait for a long time before becoming ready. Therefore, in this case, according to the set arbitration strategy, the first request queue to be checked in the current clock cycle is re-arbitrated from the multiple request queues corresponding to the data checking module. This can quickly release the data checking module to check the read data requests in other request queues. In this way, the probability of the data checking module checking a read data request with a ready data status in the current clock cycle can be increased, and the waiting delay caused by the data not being ready of the target read data request being checked can be reduced.

[0047] In some embodiments, the data inspection module 12 is further configured to: If the data check results of each target read data request in the previous clock cycle are all the second check results, the first request queue selected by the data check module in the previous clock cycle is determined as the first request queue to be checked in the current clock cycle; the second check result indicates that the data read data request of the corresponding target read data request is ready.

[0048] Understandably, if the data check results for all target read data requests in the first request queue selected in the previous clock cycle are the second check results, meaning the data read status of each target read data request is ready, it indicates that the data requested by each read data request corresponding to the same read data instruction will likely be ready soon. In this case, continuing to use the first request queue selected by the data checking module in the previous clock cycle as the first request queue to be checked in the current clock cycle can increase the probability that the data checking module will check for ready read data requests in the current clock cycle, reducing the waiting delay caused by the unread data of the checked target read data requests.

[0049] In some embodiments, the data inspection module 12 is further configured to: If the data check results of each target read data request in the previous clock cycle are all the second check results, and each target read data request does not include the last read data request corresponding to any read data instruction, then the first request queue selected by the data check module in the previous clock cycle will be determined as the first request queue to be checked in the current clock cycle.

[0050] In some implementations, a counter can be set for each read data instruction to count read data requests whose data check result is the second check result, obtaining a first count value corresponding to the read data instruction. In the current clock cycle, if the first count value of the counter for the read data instruction corresponding to the target read data request at the head of the first request queue is less than the total number of read data requests corresponding to that read data instruction, then it is determined that the last read data request corresponding to any read data instruction is not included in the target read data requests. If the first count value of the counter for the read data instruction corresponding to the target read data request at the head of the first request queue is not less than the total number of read data requests corresponding to that read data instruction, then it is determined that the last read data request corresponding to that read data instruction is not included in the target read data requests. Specifically, in a certain clock cycle, if the data check result of the first number of target read data requests in the currently selected first request queue that have undergone data readiness status checks is the second check result, that is, the data readiness status of each target read data request is "ready," then the first count value of the counter for the read data instruction corresponding to the target read data request at the head of the first request queue increases by the first number in that clock cycle. In a certain clock cycle, if the data check result of at least one target read data request in the currently selected first request queue is the first check result, that is, the data read status of at least one target read data request is not ready, then the first count value of the counter of the read data instruction corresponding to the target read data request at the head of the first request queue remains unchanged in the clock cycle.

[0051] In some implementations, a counter can be set for each request queue to count the read data requests whose data check result is the second check result among the read data requests of the read data instruction corresponding to the target read data request at the head of the request queue, so as to obtain the first count value corresponding to the read data instruction.

[0052] Understandably, the first count value corresponding to a read data instruction can represent the number of read data requests whose data readiness status is "ready" according to data check results. Based on whether the first count value corresponding to the read data instruction is less than the total number of read data requests corresponding to that read data instruction, it can be determined whether at least one target read data request checked in the previous clock cycle includes the last read data request of the read data instruction corresponding to the target read data request at the head of the first request queue selected in the previous clock cycle; that is, it can be determined whether the data for all read data requests corresponding to that read data instruction is ready. If the first count value corresponding to the read data instruction is less than the total number of read data requests corresponding to that read data instruction, it indicates that the data for all read data requests corresponding to that read data instruction is not yet fully ready. If the first count value corresponding to the read data instruction is equal to the total number of read data requests corresponding to that read data instruction, it indicates that the data for all read data requests corresponding to that read data instruction is fully ready.

[0053] In this case, if the data check results of each target read data request in the previous clock cycle are all the second check results, and each target read data request does not include the last read data request corresponding to any read data instruction, it indicates that each target read data request checked in the previous clock cycle corresponds to the same read data instruction, and the data of each target read data request is ready. However, the data of each read data request corresponding to this read data instruction is not yet fully ready, and the data requested by the remaining unread data of the read data request corresponding to this read data instruction is likely to be ready soon. Therefore, the first request queue selected in the previous clock cycle can be determined as the first request queue to be checked in the current clock cycle, so as to continue to check the data readiness status of other read data requests corresponding to this read data instruction in the current clock cycle, thereby increasing the probability that the data check module will check the ready read data requests in the current clock cycle and reducing the waiting delay caused by the unread data of the checked target read data requests.

[0054] In some embodiments, the data inspection module 12 is further configured to: If the data check results of each target read data request in the previous clock cycle are all the second check results, and each target read data request includes the last read data request corresponding to any read data instruction, then according to the set arbitration strategy, the first request queue to be checked in the current clock cycle is selected by arbitration from the multiple request queues corresponding to the data check module.

[0055] Understandably, if the data check results of each target read data request in the previous clock cycle are all the second check results, and each target read data request includes the last read data request corresponding to any read data instruction, it indicates that the data of each read data request corresponding to that read data instruction (i.e., the read data instruction corresponding to the target read data request at the head of the first request queue) is all ready. Therefore, according to the set arbitration strategy, the first request queue to be checked in the current clock cycle can be re-arbitrated from the multiple request queues corresponding to the data check module. This allows the data check module to be quickly released for checking read data requests in other request queues, thereby increasing the probability that the data check module will check a ready read data request in the current clock cycle and reducing the waiting delay caused by the unread data of the target read data request being checked.

[0056] In some embodiments, the storage module 20 includes a cache module, and each read data request is used to read data in a cache line in the cache module.

[0057] Data inspection module 12 is also used for: Multiple target read data requests are selected starting from the head of the first request queue; within the current clock cycle, the data readiness status of the multiple target read data requests is checked in parallel based on the cache line identifiers corresponding to the multiple target read data requests.

[0058] Here, a read data instruction can correspond to at least one read data request, the starting address of which is at least one consecutive cache line identifier (i.e., line address). Thus, at least one read data request corresponding to the read data instruction can be used to read data from at least one consecutive cache line identifier.

[0059] For example, the read data instruction Instruction1 corresponds to read data requests cache_request1, cache_request2, and cache_request3. The starting addresses of the data in read data requests cache_request1, cache_request2, and cache_request3 are the line addresses cacheline_id1, cacheline_id2, and cacheline_id3, respectively. Among them, the line addresses cacheline_id1, cacheline_id2, and cacheline_id3 are contiguous.

[0060] Multiple target read data requests selected starting from the head of the first request queue may include unchecked read data requests and / or read data requests that have been checked in historical clock cycles and whose corresponding data check result is the first check result. The first check result indicates that the data readiness status of the corresponding read data request is not ready.

[0061] In implementation, the data inspection module 12 may select multiple consecutive target read data requests from the head of the first request queue in any suitable manner according to the actual situation. This embodiment of the present disclosure does not limit this.

[0062] The data inspection module 12 can employ any suitable parallel inspection logic to check the data readiness status of multiple target read data requests in parallel within the current clock cycle; this embodiment of the present disclosure does not limit this. For example, the data inspection module 12 can be configured with multiple parallel inspection channels, each channel being used to check the data readiness status of a target read data request based on the cache line identifier corresponding to that target read data request.

[0063] Understandably, the data checking module 12 can check the data readiness status of multiple consecutive target read data requests starting from the head of the first request queue within each clock cycle. Since the data to be read from the merged multiple read data requests corresponding to a single read data instruction is typically stored contiguously in the request queue, and these multiple read data requests are used to read data from at least one cache line with consecutive cache line identifiers, and the data requested by each read data request corresponding to the same read data instruction is usually ready within a few close clock cycles, the data readiness status of multiple target read data requests can be checked in parallel within a single clock cycle based on the cache line identifiers corresponding to each target read data request. This improves the resource utilization of the data checking module and the efficiency of checking the data readiness status of each read data request corresponding to the same read data instruction, thereby further improving the overall performance of the memory merging device.

[0064] In some implementations, the data inspection module 12 can check the data readiness status of a first number of read data requests in parallel within a single clock cycle. The data inspection module 12 can check the data readiness status of the first number of target read data requests in parallel within the current clock cycle, based on the cache line identifiers corresponding to each of the first number of target read data requests. For example, the data inspection module 12 can use the smallest of the cache line identifiers corresponding to each of the first number of target read data requests as the starting cache line identifier to be checked, and determine the first number of cache line identifiers starting from this starting cache line identifier as the cache line identifiers to be checked in the current clock cycle. It should be noted that the first number of cache line identifiers to be checked in the current clock cycle can all be cache line identifiers corresponding to each read data request in the same read data instruction, or it can include the cache line identifiers of at least one read data request corresponding to the first read data instruction corresponding to the target read data request at the head of the first request queue, and the cache line identifiers of at least one read data request corresponding to other read data instructions. If the first number of cache line identifiers to be checked in the current clock cycle includes the cache line identifier of at least one read data request corresponding to the first read data request corresponding to the target read data request at the head of the first request queue, and the cache line identifier of at least one read data request corresponding to other read data instructions, then only the data check results of at least one read data request corresponding to the first read data instruction can be retained.

[0065] In some embodiments, each cache line in the caching module has a status flag indicating whether the data in that cache line is ready.

[0066] Data inspection module 12 is also used for: For each target read data request in parallel, based on the cache line identifier corresponding to the target read data request, the status identifier of the target cache line corresponding to the target read data request is queried; based on the status identifier of the target cache line corresponding to each target read data request, the data readiness status of each target read data request is determined.

[0067] Here, the caching module can maintain a status flag for each cache line to indicate whether the data in that cache line is ready. In this way, the data inspection module 12 can determine the data readiness status of each target read data request by querying the status flag of the target cache line corresponding to each target read data request.

[0068] For example, see Figure 2After receiving a read data request sent by the request merging module 11, the cache module 21 can perform a cache hit check based on the cache line identifier corresponding to the read data request. If there is valid data in the cache line corresponding to the cache line identifier (i.e., a cache hit), the status identifier of the cache line is set to a first identifier indicating that the data in the cache line is ready. If there is no valid data in the cache line corresponding to the cache line identifier (i.e., a cache miss), the cache module 21 requests the data corresponding to the cache line identifier from the downstream memory module 40, and writes back the data returned by the downstream memory module 40 corresponding to the cache line identifier to the cache line corresponding to the cache line identifier. After the write-back is completed, the status identifier of the cache line is updated from a second identifier to a first identifier. The second identifier indicates that the data in the cache line is ready.

[0069] In the above embodiments, each cache line in the caching module has a status identifier indicating whether the data in that cache line is ready. The data checking module, in parallel, queries the status identifier of the target cache line corresponding to each target read data request based on the cache line identifier corresponding to that target read data request, and determines the data readiness status of each target read data request based on the status identifier of the target cache line corresponding to each target read data request. In this way, the data readiness status of multiple target read data requests can be checked in parallel simply and efficiently based on the cache line identifiers corresponding to multiple target read data requests within a single clock cycle.

[0070] In some embodiments, such as Figure 3 As shown, the data inspection module 12 includes A selectors (e.g., first selector MUX1, ..., Ath selector MUXA), the total number of cache lines to be inspected in the cache module is N, and the number of data input terminals of the selectors is... There are A cache lines to be checked in the cache module, where N and A are both integers greater than 1. The cache lines to be checked are grouped into groups of A. The cache line identifiers of the same group of cache lines are consecutive, and the status identifiers of the same group of cache lines are input to the data input terminals of different selectors.

[0071] A selectors are used, within a single clock cycle, to request the corresponding cache line identifier based on their respective target read data, and select from the corresponding... The status identifier for selecting the corresponding target cache line from each data input terminal.

[0072] Here, each selector can be a A selector that selects 1. Wherein, This refers to the ratio of N to A rounded up. For example, see [link to example]. Figure 3 The first selector MUX1 Each data input terminal can be used to input cached lines entry_0, entry_A, ..., entry_( -1) The status flag of A, the selector A, MUXA. Each data input terminal can be used to input the status flags of cache lines entry_(A-1), entry_(2A-1), ..., entry_(N-1), respectively; within a single clock cycle, the first selector MUX1 is used to select from the corresponding cache line flag B_index based on the corresponding target read data request. The selector A, MUXA, is used to select the status identifier of the corresponding target cache line from the data input terminals. The selector A is used based on the cache line identifier B_index+A-1 of the corresponding target read data request. The status identifier for selecting the corresponding target cache line from each data input terminal.

[0073] In some implementations, the number of selectors A is a power of 2, and N is an integer multiple of A. This allows each selector's data input to correspond to a cache line to be checked, receiving the status flag of the response cache line. This ensures that the data inputs of each selector are fully utilized, thereby improving hardware resource utilization. For example, assuming N is 512 and A is 2, then... The value is 256, meaning each selector can be a 256-choose-1 selector. For example, assuming N is 256 and A is 8, then... The value is 32, meaning each selector can be a 32-to-1 selector.

[0074] It should be noted that the cache lines to be inspected in the cache module are grouped into A groups. The cache line identifiers within the same group are consecutive, and the status identifiers of the same group are input to the data input terminals of different selectors. Thus, within a single clock cycle, by inputting the cache line identifier of a target read data request to the selection control input terminal of each selector, the A selectors can proceed in parallel based on their respective target read data request cache line identifiers, selecting from the corresponding... The status flag of the target cache line that matches the cache line flag of the target read data request is selected and output from each data input terminal. For example, see Figure 4Assuming A is 2 and N is 512, the data checking module 12 may include a first selector MUX1 and a second selector MUX2, both of which are 256-to-1 selectors. Cache lines entry_0 and entry_1 are grouped together, cache lines entry_2 and entry_3 are grouped together, ..., cache lines entry_510 and entry_511 are grouped together. The status flags of cache lines entry_0, entry_2, ..., entry_510 are sequentially input to the 256 data input terminals of the first selector MUX1, and the status flags of cache lines entry_1, entry_3, ..., entry_511 are sequentially input to the 256 data input terminals of the second selector MUX2. Within a single clock cycle, the first selector MUX1 is used to select the status identifier of the corresponding target cache line from the 256 corresponding data input terminals based on the cache line identifier B_index of the corresponding target read data request. The second selector MUX2 is used to select the status identifier of the corresponding target cache line from the 256 corresponding data input terminals based on the cache line identifier B_index+1 of the corresponding target read data request. Among them, the cache line identifiers B_index and B_index+1 are the cache line identifiers of the two target read data requests corresponding to the first read data instruction at the head of the first request queue to be checked in the multiple request queues corresponding to the data inspection module 12.

[0075] In the above embodiment, the data inspection module includes A selectors, the total number of cache lines to be inspected in the caching module is N, and the number of data input terminals of the selectors is [missing information]. There are A cache lines to be checked in the cache module, where N and A are both integers greater than 1. The cache lines in the same group are grouped into A groups, with consecutive cache line identifiers. The status identifiers of the same group are input to the data input terminals of different selectors. Each of the A selectors, within a single clock cycle, selects the cache line based on its corresponding target read data request cache line identifier from the corresponding selector. The corresponding target cache line status identifier is selected from each data input terminal. In this way, the data inspection module can utilize multiple selectors to simply and efficiently check the data readiness status of multiple target read data requests in parallel within a single clock cycle, based on the cache line identifiers corresponding to each target read data request. Furthermore, since A... The hardware resource consumption of a 1-to-1 selector is basically the same as that of a single N-to-1 selector. Therefore, compared with the related technologies that use a single selector to check the data readiness status of a single target read data request in a single clock cycle, the data checking module provided in this disclosure can check the data readiness status of multiple target read data requests in parallel in a single clock cycle without increasing hardware resource consumption. This better meets the out-of-order scheduling capability and parallel computing capability of the memory merging device, thereby further improving the overall chip performance without increasing hardware costs.

[0076] In some embodiments, the data inspection module 12 is further configured to: transmit the queue status of each request queue in the current clock cycle to the instruction submission module 13; the queue status of the request queue in the current clock cycle indicates whether the data ready status of all read data requests corresponding to the same read data instruction as the read data request at the head of the request queue is ready.

[0077] The instruction submission module 13 is also used to: select a target request queue from at least one candidate request queue whose queue status is in the first state in the current clock cycle.

[0078] In some implementations, the data checking module 12 can determine whether the data ready status of all read data requests corresponding to the same read data instruction as the read data request at the head of the selected first request queue in the current clock cycle is "ready" based on the data checking results of at least one read data request starting from the head of the queue. If the data ready status of all read data requests corresponding to the same read data instruction as the read data request at the head of the request queue is "ready", the data checking module 12 can update the queue status of the first request queue to the first state in the current clock cycle. In each clock cycle, the data checking module 12 can transmit the queue status of each request queue in the current clock cycle to the instruction submission module 13.

[0079] For example, see continue. Figure 1 and Figure 5The data checking module 12 may include at least one selector (e.g., first selector MUX1, ..., A-th selector MUXA) and a judgment module 121. During the current clock cycle, the data checking module 12 selects a first request queue from the corresponding multiple request queues (e.g., isched_fifo 0, isched_fifo 1, isched_fifo 2, ..., isched_fifo Q-1). The at least one selector is used to check the data readiness status of at least one target read data request starting from the head of the selected first request queue during the current clock cycle, and transmits the data readiness status of each target read data request to the judgment module 121. The judgment module 121 is used to determine, based on the data checking result of the at least one target read data request, whether the data readiness status of all read data requests corresponding to the same read data instruction as the target read data request at the head of the request queue is ready, and updates the queue status of the first request queue according to the judgment result. The judgment module 121 includes multiple signal output terminals (e.g., out0, out1, out2, ..., out(Q-1)) corresponding one-to-one with the corresponding multiple request queues. Each request queue's corresponding signal output terminal is used to transmit the corresponding queue status to the signal input terminal of the instruction submission module 13 corresponding to that request queue. The instruction submission module 13 can determine at least one candidate request queue from each request queue based on the queue status input from the signal input terminal of each request queue, and select the target request queue for the current clock cycle from the at least one candidate request queue. It then sends the queue identifier of the target request queue to the data return module 14. The data return module 14 is used to obtain the target data of each read data request in the target read data instruction corresponding to the target read data request at the head of the target request queue from the cache module according to the queue identifier sent by the instruction submission module 13, and return the obtained target data to the sender of the target read data instruction. In some embodiments, the judgment module 121 can first judge the data check result of the target read data request with the smallest cache line identifier. If the data check result of the target read data request with the smallest cache line identifier is a first check result indicating that the data is not ready, then the queue status of the first request queue is determined to be the second state, without needing to check the data check results of other target read data requests.

[0080] In the above embodiments, the data inspection module transmits the queue status of each request queue in the current clock cycle to the instruction submission module. The queue status of the request queue in the current clock cycle indicates whether the data ready status of all read data requests corresponding to the read data request at the head of the request queue and the same read data instruction is ready. The instruction submission module selects the target request queue from at least one candidate request queue whose queue status is in the first state in the current clock cycle. In this way, the candidate request queues that have met the data return conditions in the current clock cycle can be quickly and accurately determined by the queue status of each request queue, thereby facilitating the instruction submission module to more accurately arbitrate and select the target request queue that expects to return data in the current clock cycle from the candidate request queues that have met the data return conditions.

[0081] This disclosure provides a data processing method applied to a memory merging apparatus, which includes a request merging module, multiple data checking modules, an instruction submission module, and a data return module. Figure 6 As shown, the data processing method includes the following steps S501 to S504: Step S501: The request merging module sends at least one read data request corresponding to the received read data instruction to the storage module, and adds the at least one read data request to the request queue corresponding to the read data instruction. In step S502, multiple data checking modules select a first request queue from their respective multiple request queues within the current clock cycle, check the data readiness status of at least one read data request starting from the head of the first request queue in the storage module, and update the queue status of the first request queue in the current clock cycle based on the data readiness status. Step S503: The instruction submission module selects the target request queue for the current clock cycle from each request queue; the queue state of the target request queue in the current clock cycle is the first state, which indicates that the data ready state of all read data requests corresponding to the read data request at the head of the corresponding request queue and the same read data instruction is ready. In step S504, the data return module obtains the target data of each read data request corresponding to the target read data instruction from the storage module, and returns the target data to the sender of the target read data instruction. The target read data instruction corresponds to the read data request at the head of the target request queue.

[0082] In this embodiment of the disclosure, the memory merging device includes a request merging module, multiple data checking modules, an instruction submission module, and a data return module. Each data checking module corresponds to multiple request queues. The request merging module sends at least one read data request corresponding to a received read data instruction to the storage module and adds the at least one read data request to the request queue corresponding to the read data instruction. Each of the multiple data checking modules selects a first request queue from its respective multiple request queues within the current clock cycle and checks the data readiness status of at least one read data request starting from the head of the first request queue in the storage module. The system checks the readiness status of the data request queue and updates the queue status of the first request queue in the current clock cycle based on the data readiness status. The instruction submission module selects the target request queue for the current clock cycle from each request queue. The target request queue is in the first state in the current clock cycle, which indicates that all read data requests corresponding to the read data request at the head of the corresponding request queue are ready. The data return module retrieves the target data for each read data request corresponding to the target read data instruction from the storage module and returns the target data to the sender of the target read data instruction. The target read data instruction corresponds to the read data request at the head of the target request queue. In this way, within a single clock cycle, each data checking module can arbitrate multiple corresponding request queues and select one request queue to check the data readiness status of at least one read data request starting from the head of the queue. In this way, read data instructions within the same request queue can return data in the same order, while read data instructions between different request queues can return data out of order. Compared with the related technologies where each request queue corresponds to a data inspection module, this approach reduces the number of data inspection modules while maintaining high out-of-order scheduling capability, thus meeting overall performance requirements with a smaller area cost.

[0083] In some embodiments, a single request queue is used to store instruction request information for at least one read data instruction submitted by threads in the same thread bundle. Adding the at least one read data request to the request queue corresponding to the read data instruction in step S501 may include the following steps S511 to S513: Step S511: Determine the thread bundle to which the thread that submitted the read data instruction belongs; Step S512: Store at least one read data request of the read data instruction into the request queue corresponding to the thread bundle.

[0084] In some embodiments, the multiple data checking modules in step S502 may select a first request queue from their respective multiple request queues within the current clock cycle, which may include the following step S521: Step S521: Based on the data inspection result of at least one target read data request in the previous clock cycle, the data inspection module selects the first request queue to be inspected in the current clock cycle from the multiple request queues corresponding to the data inspection module; the target read data request is the read data request for which the data inspection module performed a data readiness status check in the previous clock cycle.

[0085] In some embodiments, step S521 may include the following steps S531 or S532: Step S531: If at least one target read data request has a first check result in the previous clock cycle, the data inspection module, according to the set arbitration strategy, arbitrates and selects the first request queue to be checked in the current clock cycle from the multiple request queues corresponding to the data inspection module; the first check result indicates that the data readiness status of the corresponding target read data request is not ready. In step S532, if the data check result of each target read data request in the previous clock cycle is the second check result, the data check module determines the first request queue selected by the data check module in the previous clock cycle as the first request queue to be checked in the current clock cycle; the second check result indicates that the data readiness status of the corresponding target read data request is ready.

[0086] In some embodiments, step S532 may include the following step S541: In step S541, if the data check results of each target read data request in the previous clock cycle are all the second check results, and the target read data requests do not include the last read data request corresponding to any read data instruction, the data check module determines the first request queue selected by the data check module in the previous clock cycle as the first request queue to be checked in the current clock cycle.

[0087] In some embodiments, step S532 may further include the following step S551: If the data inspection module finds that all target read data requests have the second inspection result in the previous clock cycle, and that each target read data request includes the last read data request corresponding to any read data instruction, then according to the set arbitration strategy, it arbitrates and selects the first request queue to be inspected in the current clock cycle from the multiple request queues corresponding to the data inspection module.

[0088] In some embodiments, the storage module includes a cache module, and each read data request is used to read data within a cache line in the cache module. Step S502, checking the data readiness status of at least one read data request starting from the head of the first request queue in the storage module, may include the following steps S561 and S562: Step S561: Select multiple target read data requests starting from the head of the first request queue; Step S562: Within the current clock cycle, based on the cache line identifiers corresponding to the multiple target read data requests, check the data readiness status of the multiple target read data requests in parallel.

[0089] In some embodiments, each cache line in the caching module has a status flag indicating whether the data in that cache line is ready. Step S562 may include the following steps S571 and S572: Step S571: For each target read data request, query the status identifier of the target cache line corresponding to the target read data request based on the cache line identifier corresponding to the target read data request. Step S572: Determine the data readiness status of each target read data request based on the status identifier of the target cache line corresponding to each target read data request.

[0090] In some embodiments, the data inspection module includes A selectors, the total number of cache lines to be inspected in the caching module is N, and the number of data input terminals of the selectors is N. There are A cache lines to be checked in the cache module, where N and A are both integers greater than 1. The cache lines to be checked are grouped into groups of A, with consecutive cache line identifiers within the same group. The status identifiers of the cache lines within the same group are input to the data input terminals of different selectors. Step S571 may include the following step S581: Step S581: Within a single clock cycle, A selectors respectively request the corresponding cache line identifier based on their respective target read data, and retrieve the corresponding cache line identifier from the cache. The status identifier for selecting the corresponding target cache line from each data input terminal.

[0091] In some embodiments, the above data processing method may further include the following steps S591 to S592: Step S591, the data inspection module transmits the queue status of each request queue in the current clock cycle to the instruction submission module; the queue status of the request queue in the current clock cycle indicates whether the data ready status of all read data requests corresponding to the same read data instruction as the read data request at the head of the request queue is ready. Step S592: The instruction submission module selects the target request queue from at least one candidate request queue whose queue state is in the first state in the current clock cycle.

[0092] The following describes the application of the memory merging apparatus and data processing method provided in the embodiments of this disclosure in real-world scenarios.

[0093] In related technologies, the signal input terminals of each data checking module and the instruction submission module in a memory merging device are often in one-to-one correspondence, meaning that the number of data checking modules is the same as the number of signal input terminals in the instruction submission module.

[0094] The logic of the data inspection module generally requires significant hardware resources, such as a large multiplexer (MUX), to select one of multiple cache lines to be inspected for data readiness. The number of signal inputs to the instruction submission module determines the out-of-order scheduling capability of the memory merging device. In memory merging devices that maintain order according to Warp_id, more isched_fifos are typically needed to meet the performance requirements of out-of-order scheduling between different Warp_ids.

[0095] In related technologies, meeting higher performance requirements necessitates more data checking modules, leading to increased area. To address this, this disclosure proposes a cache design scheme where the data checking modules in the memory merging device do not correspond one-to-one with the signal inputs of the instruction submission module. This allows for meeting overall performance requirements with less area overhead (i.e., fewer data checking modules). For example, it achieves the same out-of-order scheduling capability as related technologies; that is, the number of signal inputs for the instruction submission module remains unchanged compared to related technologies.

[0096] In some implementations, it is assumed that the number of signal input terminals required by the instruction submission module is M, that is, the performance requirement is that out-of-order scheduling is possible between read data instructions among M Warp_ids, where M is a positive integer. It is assumed that the memory merging device has N data checking modules, where N <= M, and M is a multiple of N. The value of N can be set from the perspective of hardware resources and the parallel capability of the data checking modules, and this disclosure does not impose any restrictions. Assuming M / N = Q, each data checking module can correspond to Q request queues isched_fifo_0 to isched_fifo_Q-1. Within one data checking module, arbitration can be performed once among the Q request queues. Each clock cycle, the data checking module selects one first request queue from the corresponding Q request queues for checking; the data checking module can update the queue status of the first request queue based on the checking result. The arbitration strategy among the Q request queues is not limited in this disclosure.

[0097] For example, within the data inspection module, the grant switching strategy for arbitration among Q request queues can include: 1. If a request queue is granted authorization (i.e., the request queue is the first request queue selected in the current clock cycle), and the data readiness status of each target read data request starting from the head of the request queue is ready in the current clock cycle, and the target read data requests do not include the last read data request of any read data instruction, then the authorization of the request queue is maintained, and the next clock cycle continues to check the data readiness status of at least one target read data request in the next batch after each target read data request in the current batch in the request queue.

[0098] 2. If a request queue has been granted authorization, and the read read status of all target read data requests in that queue (starting from the head of the queue) is "ready" according to the clock check, and each target read data request includes the last read data request of any read data instruction, then the authorization for that request queue is released. In the next clock cycle, a new request queue is selected from the Q request queues using the set arbitration strategy to allocate authorization.

[0099] 3. If a request queue has been granted authorization, and the read data read status of at least one target data request starting from the head of that request queue is not ready, then the authorization for that request queue is released. In the next clock cycle, a new request queue is selected from the Q request queues using the set arbitration strategy to allocate authorization.

[0100] The arbitration strategy set may include, but is not limited to, round-robin.

[0101] The above authorization switching strategy allows for the following: If, in the currently selected request queue, all read data requests corresponding to the read data request at the head of the queue are in a ready state, the data can quickly pass through the data check module, meeting the conditions for requesting the instruction submission module (i.e., the read data instruction corresponding to the read data request at the head of the queue has been confirmed by the data check module to have all read data requests in a ready state). If the read data request corresponding to a read data instruction is not in a ready state, the data check module can be quickly released for use by other request queues. Understandably, in the relevant technical solutions, one data checking module corresponds to only one request queue, meaning that one data checking module is only used by one request queue. If the data readiness status of a certain read data request in the request queue is not ready, the data readiness status of this read data request will still be checked in the next clock cycle. However, if the data of a read data request is not ready (e.g., cache miss), it will generally take a long time to reach data readiness. Therefore, if the data readiness status of this read data request is still checked in the next clock cycle, it will cause a waste of resources for the data checking module.

[0102] Furthermore, in related technical solutions, the data checking module only checks the data readiness status of one read request per clock cycle (i.e., only checks whether the data of one cache line is ready). In some scenarios, the data checking module can become a performance bottleneck. For example, if all read requests for a certain read instruction have cache hits, the data readiness status of all read requests for that read instruction is "ready," then the data checking module will become the performance bottleneck for Q request queues. Based on this, the embodiments of this disclosure can optimize the checking logic of the data checking module so that it can check the data readiness status of more than one read request per clock cycle.

[0103] Assuming the total number of cache lines for each read data request to be checked is total_entry, in related technical solutions, the data checking module needs to use a multiplexer MUX to select one cache line from total_entry to check whether its data is ready. For example, the multiplexer is used to query the status flag corresponding to the cache line, which represents whether the data in the cache line is ready. In the corresponding hardware resources, MUX is a selector that selects 1 from total_entry.

[0104] In this embodiment of the disclosure, the data inspection module can check the data readiness status of multiple consecutive read data requests starting from the head of the first request queue each clock cycle. For example, the data inspection module can check the data readiness status of A consecutive read data requests starting from the head of the first request queue each clock cycle, where A>=1 and A is a power of 2. Since the multiple read data requests corresponding to a read data instruction after request merging are stored consecutively in the corresponding request queue, and the data of these multiple read data requests are also stored in multiple consecutive cache lines (i.e., cache request entries), the multiple read data requests checked in one clock cycle belong to the same read data instruction, or at least one read data request at the tail of a read data instruction plus at least one read data request at the head of the next read data instruction. In this embodiment, the MUX in the data inspection module that selects 1 for each total_entry is improved to a selector with A total_entry / A selectors. The cached lines of the total_entry read data requests to be inspected are first grouped into A groups, with each group having consecutive cache line identifiers (corresponding to entry IDs). Cache lines within the same group are assigned to different selectors. In this way, the data inspection module can check the status identifiers of A consecutive cache lines in a single clock cycle, meaning the data inspection module can check the data readiness status of A read data requests each clock cycle. If the A read data requests checked in the current clock cycle belong to the same read data instruction, the performance of the data checking module is directly A times that of the related technology solution (i.e., using a selector with one total_entry option). If the A read data requests checked in the current clock cycle include at least one read data request at the tail of the currently checked read data instruction and at least one read data request at the head of the next read data instruction, then only the data ready state of at least one read data request at the tail of the read data instruction corresponding to the read data request at the head of the current request queue is retained, and the arbitration of the request queue in the next clock cycle is performed according to the authorization switching strategy described above. In this case, the performance of the data checking module does not reach A times that of the related technology solution, but it is not worse than the performance of the related technology solution. Furthermore, in terms of hardware resource consumption, the optimized selector with A total_entry options / A option is basically the same as the selector with one total_entry option. In other words, the data checking module provided by this embodiment can have better performance without increasing hardware resource consumption.

[0105] The descriptions of the above method embodiments are similar to those of the above device embodiments, and have similar beneficial effects. In some embodiments, the functions or modules included in the device provided in this disclosure can be used to perform the methods described in the above method embodiments. For technical details not disclosed in the method embodiments of this disclosure, please refer to the descriptions of the device embodiments of this disclosure for understanding.

[0106] It should be noted that the implementation of the embodiments disclosed herein is not limited to any specific hardware, software or firmware, or any combination of hardware, software and firmware.

[0107] This disclosure provides a chip product, such as... Figure 7 As shown, the chip product 60 includes the memory merging device 10 in the above embodiments. For example, the chip product may include, but is not limited to, at least one of a GPU chip, an AI chip, a TPU (Tensor Processing Unit) chip, an NPU (Neural Processing Unit) chip, etc.

[0108] This disclosure provides a computer device, such as... Figure 8 As shown, the computer device 70 includes the memory merging device 10 in the above embodiments. The computer device 70 may include, but is not limited to, personal computers, workstations, game consoles, or some mobile devices (such as tablets, smartphones, etc.), and may also be in-vehicle terminal devices, smart home devices, smart TVs, or smart robots, etc. It may also be servers, server clusters, artificial intelligence computing clusters, or cloud computing clusters, etc., wherein the artificial intelligence computing cluster may also be simply referred to as an intelligent computing cluster or smart computing cluster, and this disclosure does not limit it.

[0109] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above method, chip product, and device embodiments are similar to the descriptions of the above apparatus embodiments and have similar beneficial effects. For technical details not disclosed in the method, chip product, and device embodiments of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0110] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above embodiments of this disclosure are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0111] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0112] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may all be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the integrated unit may be implemented in hardware or in a combination of hardware and software functional units.

[0114] The above are merely exemplary embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A memory merging device, characterized in that, include: The request merging module is configured to: send at least one read data request corresponding to the received read data instruction to the storage module based on the received read data instruction, and add the at least one read data request to the request queue corresponding to the read data instruction; Multiple data checking modules are provided, and any one of the multiple data checking modules is used to: select a first request queue from multiple request queues corresponding to the data checking module within the current clock cycle, and check the data readiness status of at least one read data request in the storage module starting from the head of the first request queue; and update the queue status of the first request queue in the current clock cycle according to the data readiness status. The instruction submission module is used to: select the target request queue for the current clock cycle from each of the request queues; the queue state of the target request queue in the current clock cycle is a first state, the first state indicates that the data ready state of all read data requests corresponding to the read data request at the head of the corresponding request queue and the same read data instruction is ready; The data return module is used to obtain the target data of each read data request corresponding to the target read data instruction from the storage module, and return the target data to the sender of the target read data instruction. The target read data instruction corresponds to the read data request at the head of the target request queue.

2. The memory merging apparatus according to claim 1, characterized in that, The data inspection module is also used for: Based on the data check results of at least one target read data request in the previous clock cycle, select the first request queue to be checked in the current clock cycle from the corresponding multiple request queues; The target read data request is the read data request made by the data checking module during the previous clock cycle when checking the data readiness status.

3. The memory merging apparatus according to claim 2, characterized in that, The data inspection module is also used for one of the following: If at least one of the target read data requests has a first check result in the previous clock cycle, the first request queue to be checked in the current clock cycle is selected from the multiple request queues corresponding to the data check module according to the set arbitration strategy. The first check result indicates that the data read data request for the corresponding target is not ready. If the data check result of each of the target read data requests in the previous clock cycle is the second check result, the first request queue selected by the data check module in the previous clock cycle will be determined as the first request queue to be checked in the current clock cycle. The second check result indicates that the data read request for the corresponding target is ready.

4. The memory merging apparatus according to claim 3, characterized in that, The data inspection module is also used for: If the data check results of all the target read data requests in the previous clock cycle are the second check results, and none of the target read data requests include the last read data request corresponding to any read data instruction, then the first request queue selected by the data check module in the previous clock cycle is determined as the first request queue to be checked in the current clock cycle.

5. The memory merging apparatus according to claim 3, characterized in that, The data inspection module is also used for: If the data check results of all the target read data requests in the previous clock cycle are the second check results, and each target read data request includes the last read data request corresponding to any read data instruction, then according to the set arbitration strategy, the first request queue to be checked in the current clock cycle is selected by arbitration from the multiple request queues corresponding to the data check module.

6. The memory merging apparatus according to claim 1, characterized in that, The storage module includes a caching module, and each read data request is used to read data from a cache line in the caching module; the data checking module is further used for: Select multiple target read data requests starting from the head of the first request queue; Within the current clock cycle, the data readiness status of the multiple target read data requests is checked in parallel based on the cache line identifiers corresponding to the multiple target read data requests.

7. The memory merging apparatus according to claim 6, characterized in that, Each cache line in the caching module has a status flag indicating whether the data in the cache line is ready; the data checking module is further used for: For each target read data request, the status identifier of the target cache line corresponding to the target read data request is queried in parallel based on the cache line identifier corresponding to the target read data request. Based on the status identifier of the target cache line corresponding to each target read data request, the data readiness status of each target read data request is determined.

8. The memory merging apparatus according to claim 7, characterized in that, The data inspection module includes A selectors, the caching module has N cache lines to be inspected, and the number of data input terminals of the selectors is [missing information]. There are A cache lines to be checked in the cache module, where N and A are both integers greater than 1. The cache lines to be checked in the cache module are grouped into groups of A. The cache line identifiers of the same group of cache lines are consecutive, and the status identifiers of the same group of cache lines are respectively input to the data input terminals of different selectors. The A selectors are each used within a single clock cycle to request the corresponding cache line identifier based on their respective target read data, and select from the corresponding cache line identifier. The status identifier for selecting the corresponding target cache line from each data input terminal.

9. The memory merging apparatus according to any one of claims 1 to 8, characterized in that, The data inspection module is further configured to: transmit the queue status of each request queue in the current clock cycle to the instruction submission module; the queue status of the request queue in the current clock cycle indicates whether the data ready status of all read data requests corresponding to the same read data instruction as the read data request at the head of the request queue is ready. The instruction submission module is further configured to: select the target request queue from at least one candidate request queue whose queue state is the first state in the current clock cycle.

10. A data processing method, characterized in that, An application is made to a memory merging apparatus, the apparatus comprising a request merging module, multiple data checking modules, an instruction submission module, and a data return module; the method includes: The request merging module sends at least one read data request corresponding to the received read data instruction to the storage module, and adds the at least one read data request to the request queue corresponding to the read data instruction. Each of the data checking modules selects a first request queue from its respective multiple request queues within the current clock cycle, checks the data readiness status of at least one read data request in the storage module starting from the head of the first request queue, and updates the queue status of the first request queue in the current clock cycle based on the data readiness status. The instruction submission module selects the target request queue for the current clock cycle from each of the request queues; the queue state of the target request queue in the current clock cycle is the first state, which indicates that the data ready state of all read data requests corresponding to the read data request at the head of the corresponding request queue and the same read data instruction is ready. The data return module obtains the target data of each read data request corresponding to the target read data instruction from the storage module, and returns the target data to the sender of the target read data instruction. The target read data instruction corresponds to the read data request at the head of the target request queue.

11. A chip product, characterized in that, The chip product includes the memory merging device as described in any one of claims 1 to 9.

12. A computer device, characterized in that, The computer device includes a memory merging apparatus as described in any one of claims 1 to 9.

Images