Cache request fusion method and artificial intelligence chip

Abstract

The invention provides a cache request fusion method and an artificial intelligence chip, and relates to the technical field of artificial intelligence chips, and the method comprises the steps: querying an uncompleted buffer area based on a first memory address carried by a first data request; when at least one second data request of the same memory address is queried, that is, when the first data request and the at least one second data request need to operate the same cache line, fusing the first data request and the at least one second data request into a target data request; respective response contents of the first data request and the at least one second data request can be obtained by executing the target data request, so that the number of executed data requests is reduced; and correspondingly, the consumption frequency of resources necessary for executing the data request is also reduced, so that the resource waste is reduced, and meanwhile, the internal bandwidth is improved. The multiple data requests are fused into one data request to be executed, the memory access data volume of a single request is increased, and therefore the cache performance and the memory access efficiency are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence chip, and particularly relate to a cache request fusion method and an artificial intelligence chip. BACKGROUND

[0002] In an artificial intelligence chip, a large size is generally used for a memory access path of a last level cache (LLC), that is, a data amount of a single cache line in the LLC is large. In order to reduce implementation complexity, there is generally only one hit-miss check pipeline. Therefore, increasing the memory data amount of a single request is a key to improving the speed of the LLC.

[0003] In related technologies, an artificial intelligence chip generally adopts a single instruction, multiple threads (SIMT) architecture. In this architecture, there may be multiple small-size requests (that is, the memory data amount of the request is much smaller than the data amount of a single cache line in the LLC), which leads to resource waste and thus reduces cache performance. SUMMARY

[0004] Embodiments of the present application provide a cache request fusion method and an artificial intelligence chip, which are used to fuse multiple data requests of the same address into one data request for memory access, so as to reduce resource consumption and improve cache performance.

[0005] In one aspect, the present application provides a cache request fusion method, which includes the following steps. receiving a first data request of a first processing core, the first data request including a first memory address; when there is at least one second data request corresponding to the first memory address in an unfinished buffer, fusing the first data request and the at least one second data request into one target data request, the unfinished buffer being used to store a data request received by the cache and not completed; executing the target data request to obtain a target request response, the target request response including a response content of each of the first data request and the at least one second data request.

[0006] Optionally, when there is at least one second data request corresponding to the first memory address in the unfinished buffer, fusing the first data request and the at least one second data request into one target data request includes the following steps. when there is at least one second data request corresponding to the first memory address in the uncompleted buffer, issuing one second data request in the at least one second data request as the target data request, and executing the target data request to obtain a target request response. setting the first data request to a waiting state.

[0007] Optionally, the first data request further comprises a first request identifier. adding a first information record of the first data request in the uncompleted buffer, wherein the first information record comprises the first request identifier and the first memory address.

[0008] Optionally, after executing the target data request to obtain a target request response, the method further comprises: returning a response content of the first data request to the first processing core; returning a response content of each second data request to a corresponding second processing core.

[0009] Optionally, the uncompleted buffer is a read uncompleted list, and the first data request and the second data request are read requests. the executing the target data request to obtain a target request response comprises: when the target data request hits a cache line, reading data of the cache line from a cache memory based on tag information of the cache line; adding the data of the cache line as the target request response to a first buffer in a read data cache.

[0010] Optionally, the read uncompleted list comprises a target information record of the target data request. after the adding the data of the cache line as the target request response to a first buffer in a read data cache, the method further comprises: adding a target pointer pointing to the first buffer in the target information record; reading a response content of the target data request from the first buffer according to the target pointer and a memory access offset carried by the target data request.

[0011] Optionally, the read uncompleted list further comprises a first information record of the first data request. after the adding the data of the cache line as the target request response to a first buffer in a read data cache, the method further comprises: adding a target pointer pointing to the first buffer in the first information record; The response content of the first data request is read from the first buffer according to the target pointer and the memory access offset carried by the first data request.

[0012] Optionally, the incomplete buffer is a list of incomplete writes; the first data request and the second data request are write requests; the first data request further includes: an input buffer address and a first set of data to be written; After receiving the first data request from the first processing core, the process further includes: Write the first data to be written to the second buffer corresponding to the input buffer address.

[0013] Optionally, executing the target data request and obtaining the target request response includes: When the target data request hits a cache line, read the full data already stored in the second buffer; and write the full data into the storage area corresponding to the cache line in the cache memory; After a successful write operation, a write success message will be sent as the response to the target request.

[0014] Optionally, after successfully writing the data and sending a write success message as a response to the target request, the method further includes: The write success message is used as the response content for the first data request and the at least one second data request, respectively.

[0015] On one hand, embodiments of this application provide an artificial intelligence chip, including: Processes the core, cache, and main memory; The cache is used to execute the aforementioned cache request fusion method.

[0016] On one hand, embodiments of this application provide a computer device, including a memory, an artificial intelligence chip, and a computer program stored on the memory and executable on the artificial intelligence chip. When the artificial intelligence chip executes the computer program, it implements the steps of the above-mentioned cache request fusion method.

[0017] In the embodiment of the present application, the cache receives the first data request sent by the first processing core, and then queries the uncompleted buffer based on the first memory address carried by the first data request. The uncompleted buffer is used to store the data requests received by the cache and not completed. When at least one second data request of the same memory address is queried, that is, the first data request and the at least one second data request need to operate the same cache line, the first data request and the at least one second data request are fused into one target data request. Then, the response content of the first data request and the at least one second data request is obtained by executing the target data request, which reduces the number of executed data requests. Correspondingly, the number of consumptions of resources necessary for executing the data requests (such as hit check resources, bandwidth resources, and various resources) is also reduced, thereby reducing resource waste and improving internal bandwidth.

[0018] Secondly, the plurality of data requests are fused into one data request for execution, which increases the memory data amount of a single request, thereby improving cache performance and memory efficiency. In addition, the request fusion method of the cache in the present application only needs to increase relevant logic when the data request enters the cache, the implementation logic is simple, and the design difficulty of the cache is not increased, which is easy to implement. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0020] Figure 1 A structure schematic diagram of an artificial intelligence chip provided by the embodiment of the present application is shown in the figure. Figure 2 A structure schematic diagram of an artificial intelligence chip provided by the embodiment of the present application is shown in the figure. Figure 3 A structure schematic diagram of a merging engine provided by the embodiment of the present application is shown in the figure. Figure 4 A flowchart of a request fusion method of a cache provided by the embodiment of the present application is shown in the figure. Figure 5 A structure schematic diagram of a computer device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and beneficial effects of the present application more clear, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0022] The terms "first", "second", "third", etc. in this application are only for descriptive purposes and cannot be understood as explicitly or implicitly indicating relative importance or implicitly indicating the number of the technical features indicated. In addition, in the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise specified.

[0023] Under the related art, in the access process of the LLC by the processing core under the SIMT architecture, there may be multiple small-size requests (i.e., the access data amount of the request is much smaller than the data amount of a single cache line in the LLC). Since the cache line is the smallest operation unit in the cache, even if the access data amount of the current request is much smaller than the data amount of a single cache line in the LLC, the cache will process it at the granularity of the cache line.

[0024] For example, the data amount of the cache line is 512 bytes, and the access data amount of the read request is 128 bytes. At this time, the cache will read 512 bytes of data and return it to the processing core, and the processing core selects the required 128 bytes from the returned data. The other data returned is actually not needed.

[0025] In this entire access process, the hit check resources, bandwidth resources and various resources of the cache need to be consumed, thereby causing resource waste; at the same time, because the small-size access needs to consume a large amount of resources, it may cause back pressure on the upstream, thereby causing the cache performance to decline.

[0026] In view of this, the present application provides a cache request fusion method, which fuses multiple data requests of the same address into one data request for access, thereby reducing resource consumption and improving cache performance.

[0027] Before introducing the cache request fusion method in the present application, a structure diagram of an artificial intelligence chip to which the embodiments of the present application are applicable is introduced.

[0028] Reference Figure 1 It is a structure diagram of an artificial intelligence chip 100 to which the embodiments of the present application are applicable, which at least includes a processing core 101, a cache 102 and a main memory 103; wherein the processing core 101 has the ability of artificial intelligence related calculation, such as matrix multiplication, convolution operation, etc. These operations are common calculation operations in artificial intelligence models.

[0029] The cache 102 is a small and fast storage area located between the processing core 101 and the main memory 103, which is used to temporarily store data or instructions that the processing core 101 or other hardware devices may frequently access in the near future, with the purpose of speeding up the access and reducing the access delay. The main memory 103 is the next level of the cache 102, and compared with the cache 102, the main memory 103 generally has larger capacity but slower speed.

[0030] The cache 102 comprises a plurality of levels of cache, and the request fusion method of the cache in the present application is applicable to LLC in the plurality of levels of cache and is also applicable to other levels of cache. The present application does not make a specific limitation in this regard.

[0031] In some embodiments, referring to Figure 2 , the cache 102 comprises a merge engine 201, an input buffer 202, a hit-miss check 203, a cache RAM 204, and a schedule buffer 205.

[0032] In a specific implementation, the tag memory is used to store tag information of each cache line in a plurality of cache sets, wherein a cache set is a logical grouping composed of a plurality of cache lines. A cache line is the smallest unit of storing data in the cache 102 and is also the smallest operation unit of the cache 102.

[0033] The cache line comprises data and tag information, wherein the tag information comprises a tag, a valid bit, and a dirty bit. The tag is used to identify the address range of the data in the main memory. The valid bit indicates whether the data is valid. The dirty bit indicates whether the data is modified, i.e., whether it is consistent with the main memory.

[0034] The cache RAM 204 stores the data in each cache line. The tag information stored in the tag memory has a one-to-one correspondence with the data stored in the cache RAM 204.

[0035] In the embodiments of the present application, the merge engine 201 can receive a data request sent by a plurality of processing cores 101 or can receive a plurality of data requests sent by one processing core 101. Based on this, the first processing core and the second processing core described in the present application are an example of the processing core 101. The first processing core and the second processing core can refer to the same processing core or can refer to two independent processing cores.

[0036] Specifically, the merge engine 201 receives a first data request sent by the first processing core, and the first data request comprises a first memory address. The merge engine 201 queries an uncompleted buffer based on the first memory address, and the uncompleted buffer is used to store data requests received by the cache 102 and not completed. When there is at least one second data request corresponding to the first memory address in the uncompleted buffer, the first data request and the at least one second data request are fused into one target data request.

[0037] The target data request can be one of the at least one second data request received earliest. When the target data request is received, there is no data request corresponding to the first memory address in the uncompleted buffer of the merging engine 201, and thus the target data request is sent to the hit checking unit 203. The second data requests and the first data requests received after the target data request are not sent any more, but are put into a waiting state.

[0038] The hit checking unit 203 reads the tag information of each cache line from the tag memory, and then performs hit checking on the first memory address of the target data request based on the read tag information, and sends the checking result to the scheduling buffer 205.

[0039] When the first memory address matches the tag information of one cache line, the checking result is that the target data request hits one cache line; when there is no tag information matching the first memory address, the checking result is that the target data request does not hit a cache line, and thus a cache line is allocated for the target data request, and the memory address of the cache line is written into the tag memory.

[0040] In some embodiments, if the first data request and the second data request are read requests corresponding to the same memory address, when the target data request hits one cache line, the scheduling buffer 205 schedules reading the data of the cache line from the cache memory 204, and returns the data of the cache line as the target request response to the merging engine 201; the data of the cache line can be the data read from the main memory 103 before.

[0041] The merging engine 201 obtains the response content of the first data request and the response content of each second data request from the target request response; then returns the response content of the first data request to the first processing core; and returns the response content of each second data request to the corresponding second processing core.

[0042] After the response content is returned to the corresponding processing core, the merging engine 201 releases the storage space used to save the target request response, and sends a prompt message that the storage space has been released to the scheduling buffer 205, so that the scheduling buffer 205 can use the released storage space to save new target request responses subsequently.

[0043] In some embodiments, if the first data request and the second data request are write requests corresponding to the same memory address, the first data request further includes an input buffer address and first to-be-written data. After the merging engine 201 receives the first data request of the first processing core, the first to-be-written data is written into the second buffer of the input buffer 202 according to the input buffer address.

[0044] When the target data request hits a cache line, the dispatch buffer 205 dispatches reading the full data stored in the second buffer; the full data refers to all the data stored in the second buffer; in practical applications, the full data not only contains the first to-be-written data, but also can contain the second to-be-written data carried by each of the at least one second data request. The full data is written into the storage area corresponding to the cache line in the cache memory 204.

[0045] In some cases, before the full data is written into the cache memory 204, an address query can be performed on the merge engine 201, that is, the input buffer address currently requiring to write data is viewed. After the query, the merge engine 201 no longer allows a newly received write request corresponding to the first memory, and the first data request and the at least one second data request are fused.

[0046] After the writing is successful, the dispatch buffer 205 returns a write success message to the merge engine 201 as a target request response. The merge engine 201 returns the write success message to the first processing core as a response content of the first data request, and returns the write success message to the responding second processing core as a response content of each second data request.

[0047] In some embodiments, referring to Figure 3 , the merge engine 201 includes a read allocation engine 301, a read outstanding list 302, a read data cache 303, a write allocation engine 304, a write outstanding list 305, a first selector 306, and a second selector 307.

[0048] Each information record in the read outstanding list 302 includes the following elements: a request identifier (req_id), a memory address (addr), and a pointer, wherein the pointer is used to point to a specific buffer in the read data cache 303.

[0049] Each information record in the write outstanding list 305 includes the following elements: a request identifier (req_id), a memory address (addr), and an input buffer address (corresponding to a specific buffer in the input buffer 202).

[0050] In some cases, the information records in the read outstanding list 302 and the write outstanding list 305 further include other elements such as sideband information.

[0051] In the case where the first data request is a read request, the read allocation engine 301 receives the first data request, and then adds a first information record of the first data request in the read outstanding list 302, the first information record including a first request identifier and a first memory address; the pointer in the first information record is a default pointer.

[0052] Scanning the read outstanding list 302 based on the first memory address; if at least one second data request corresponding to the first memory address and valid is scanned, the execution of the first data request is no longer issued, that is, the first data request is put into a waiting state.

[0053] The first selector 306 receives the data of the cache line returned by the cache memory 204 (that is, the data read by executing the target data request), and then saves the data of the cache line as the target request response in the first buffer of the read data cache 303. The target information record of the target data request in the read outstanding list 302 is queried, and the default pointer in the target information record is adjusted to the target pointer pointing to the first buffer.

[0054] According to the target pointer and the memory access offset carried by the target data request, the response content of the target data request is read from the first buffer and returned to the corresponding second processing core; then in the read outstanding list 302, the target information record is set to be invalid.

[0055] Scanning the data requests in the read outstanding list 302 corresponding to the same memory address (that is, the first memory address) as the target data request, which at least includes the first data request; in some cases, it can also contain one or more second data requests.

[0056] For the scanned first data request, the default pointer in the corresponding first information record is adjusted to the target pointer pointing to the first buffer. According to the target pointer and the memory access offset carried by the first data request, the response content of the first data request is read from the first buffer and returned to the first processing core; then in the read outstanding list 302, the first information record is set to be invalid.

[0057] For the scanned second data request, the default pointer in the corresponding second information record is adjusted to the target pointer pointing to the first buffer. According to the target pointer and the memory access offset carried by the second data request, the response content of the second data request is read from the first buffer and returned to the second processing core; then in the read outstanding list 302, the second information record is set to be invalid.

[0058] After each response content is returned to the corresponding processing core, the read data cache 303 releases the first buffer used to save the target request response, and sends a release message of the first buffer to the dispatch buffer 205, so that the dispatch buffer 205 can use the released first buffer to save new target request responses in the future.

[0059] In the case that the first data request is a write request, the write allocation engine 304 receives the first data request, and then adds a first information record of the first data request in the write outstanding list 305, the first information record including: a first request identifier, a first memory address, and an input buffer address of the second buffer in the input buffer 202. The first to-be-written data is written into the second buffer in the input buffer 202 according to the input buffer address.

[0060] The write outstanding list 305 is scanned based on the first memory address; if at least one second data request corresponding to the first memory address and valid is scanned, the first data request is no longer issued for execution, that is, the first data request is placed in a waiting state.

[0061] It should be noted that the write outstanding list 305 further includes a second information record of each second data request; the first information record and the second information record both include the first memory address and the input buffer address, and the request identifiers of the first information record and the second information record are different. In the at least one second data request, one second data request that has been issued for execution is a fused target data request (corresponding to a target information record).

[0062] The merge engine 201 receives the write success message (a write success message obtained by executing the target data request) sent by the dispatch buffer 205, returns the write success message to the second processing core sending the target data request, and then invalidates the target information record in the write outstanding list 305.

[0063] The write success message is returned to the first processing core sending the first data request, and then the first information record in the write outstanding list 305 is invalidated.

[0064] The write success message is returned to the second processing core sending the other second data request, and then the second information record in the write outstanding list 305 is invalidated.

[0065] When the first data request is received, if no data request corresponding to the first memory address is scanned, the first data request is issued to the second selector 307 for judgment. When the second selector 307 selects to execute the first data request, the first data request is issued to the hit check unit 203 for hit check. At this time, the first data request is the fused target data request, and the data request of the same address received after the first data request does not need to be issued, but is in a waiting state. In this case, the processing process of the first data request is the same as the processing process of the target data request described above, which will not be described here.

[0066] The artificial intelligence chip 100 in the present application can further include other structures in addition to the above structure, and the present application does not make specific limitations thereto.

[0067] Artificial intelligence chips 100 can be: Graphics Processing Unit (GPU), General-Purpose Graphics Processing Units (GPGPU), Domain Specific Architecture (DSA), etc.

[0068] The following is based on Figure 1 The diagram shown illustrates the architecture of an AI chip and details the process of a cached request fusion method. (See also...) Figure 4 This method is executed by a cache in an artificial intelligence chip and includes the following steps: Step 401: Receive a first data request from the first processing core. The first data request includes a first memory address.

[0069] Specifically, the first data request can be any data request sent to the cache by the first processing core. The first memory address is used to perform a hit check on the first data request; the first data request can be a read request or a write request. The first data request also includes: a first request identifier, which is used to uniquely identify the first data request.

[0070] Step 402: When there is at least one second data request corresponding to the first memory address in the uncompleted buffer, merge the first data request and at least one second data request into a single target data request.

[0071] Specifically, the incomplete buffer is used to store cached but incomplete data requests. After receiving the first data request from the first processing core, a first information record of the first data request is added to the incomplete buffer. The first information record includes: a first request identifier and a first memory address.

[0072] Since the first data request and at least one second data request received by the cache correspond to the same memory address (i.e., the first memory address), the first data request and at least one second data request operate on the same cache line. If the first data request and at least one second data request are executed separately, the same cache line needs to be operated on repeatedly, which results in large resource overhead and low memory access efficiency.

[0073] Based on this, in this application, at least one of the second data requests that has been issued and executed is taken as the target data request; and the first data request is set to a waiting state.

[0074] When the second data requests corresponding to the first memory address are multiple, the target data request is the earliest received second data request among the multiple second data requests; when the target data request is received, no data request corresponding to the first memory address and being valid is scanned in the uncompleted buffer.

[0075] Among the multiple second data requests, except for the second data request (i.e. the target data request) that has been issued for execution, other second data requests do not need to be issued, but are placed in a waiting state.

[0076] Step 403: executing the target data request to obtain a target request response.

[0077] Specifically, the first data request and the at least one second data request operate on the same cache line, therefore, the target request response obtained by executing the target data request (i.e. one second data request) includes the response content of the first data request and the response content of the at least one second data request.

[0078] In this way, the response content of the first data request is obtained from the target request response, and then the response content of the first data request is returned to the first processing core. The response content of each second data request is obtained from the target request response, and then the response content of each second data request is returned to the corresponding second processing core.

[0079] In the embodiment of the present application, the cache receives the first data request sent by the first processing core, and then queries the uncompleted buffer based on the first memory address carried by the first data request; when at least one second data request of the same memory address is queried, i.e. the first data request and the at least one second data request need to operate on the same cache line, the first data request and the at least one second data request are fused into one target data request; then the target data request is executed, and the response content of the first data request and the response content of the at least one second data request are obtained, which reduces the number of executed data requests; correspondingly, the number of consumptions of resources (such as hit check resources, bandwidth resources and various resources) necessary for executing the data requests is reduced, thereby reducing resource waste and improving internal bandwidth.

[0080] Secondly, the multiple data requests are fused into one data request for execution, which increases the memory data amount of a single request, thereby improving cache performance and memory efficiency.

[0081] In addition, the request fusion method of the cache in the present application only needs to increase relevant logic when the data request enters the cache, the implementation logic is simple, and the design difficulty of the cache is not increased, which is easy to implement.

[0082] In some embodiments, the unfinished buffer is a read unfinished list, the first data request and the second data request are read requests. After receiving the first data request of the first processing core, a first information record of the first data request is added to the read unfinished list, and the first information record includes: a first request identifier, a first memory address, and a default pointer.

[0083] The read unfinished list further includes a second information record of each second data request; the first information record and the second information record both include the first memory address and the default pointer; the first information record and the second information record have different request identifiers. In at least one second data request, one second data request that has been issued for execution is a target data request (corresponding to a target information record).

[0084] For the target data request that has been issued for execution, a hit check is first performed on the target data request. When the target data request hits a cache line, data of the cache line is read from the cache memory based on tag information of the cache line; and then the data of the cache line is added to the first buffer in the read data cache as a target request response.

[0085] In some embodiments, the first buffer has a size same as that of a cache line, and the first buffer stores data of one cache line. In the target information record, a target pointer pointing to the first buffer is added; and then the target data request response content is read from the first buffer according to the target pointer and a memory access offset carried by the target data request.

[0086] Specifically, in the target information record, the default pointer is adjusted to the target pointer pointing to the first buffer. The target pointer is used to locate the first buffer in the read data cache; and the memory access offset of the target data request is used to locate a specific position of the target data request response content in the first buffer, i.e., a specific position of the target data request response content in the target request response. The target data request response content can be the target request response itself (i.e., data of the entire cache line read), or can be part of the target request response (i.e., part of the data of the cache line).

[0087] In some embodiments, since the first data request and the target data request correspond to the same memory address (i.e., the first data request and the target data request have a binding relationship), after the target pointer pointing to the first buffer is added in the target information record; accordingly, the target pointer pointing to the first buffer is added in the first information record, i.e., the default pointer is adjusted to the target pointer pointing to the first buffer in the first information record. Then the first data request response content is read from the first buffer according to the target pointer and a memory access offset carried by the first data request.

[0088] Specifically, the target pointer is used to locate the first buffer from the read data cache; the access offset carried by the first data request is used to locate the specific position of the response content of the first data request in the first buffer, i.e., the specific position of the response content of the first data request in the target request response. The response content of the first data request can be the entire cache line data read, or part of the data in the cache line.

[0089] The response content of the first data request and the response content of the target data request can be completely different data, or can be data with overlap, which is not limited in the present application.

[0090] In addition, when there are multiple second data requests corresponding to the first memory address, in addition to the second data request (i.e., the target data request) that has been issued for execution, for each of the other second data requests, a target pointer pointing to the first buffer is added in the second information record of the second data request; and the response content of the second data request is read from the first buffer according to the target pointer and the access offset carried by the second data request. The specific process is not repeated here.

[0091] The response content of the first data request and the response content of the target data request can be completely different data, or can be data with overlap, which is not limited in the present application.

[0092] For example, the cache receiving processing core 1 sends a read request 1 (carrying a memory address 1), and adds an information record 1 of the read request 1 in the read incomplete list. Since no other information record containing the memory address 1 is found in the read incomplete list, the read request 1 is issued for execution.

[0093] If the processing core 2 sends a read request 2 (carrying the memory address 1) during the execution of the read request 1, an information record 2 of the read request 2 is added in the read incomplete list. Since the information record 1 in the read incomplete list contains the memory address 1, the read request 2 is not issued for execution, but enters a waiting state.

[0094] During the execution of the read request 1, the read request 1 is checked for a hit. When the read request 1 hits a cache line, the data of the cache line is read from the cache memory, wherein the size of the cache line is 512 bytes. The data of the cache line is added to the idle buffer 1 in the read data cache.

[0095] Then, the pointer 1 pointing to the buffer 1 is added in the information record 1, and the pointer 1 pointing to the buffer 1 is added in the information record 2.

[0096] The access offset indicated by the set read request 1 is 256 bytes from address 0. Based on this, the buffer 1 in the read data cache is located according to the pointer 1, and then the first 256 bytes of data in the buffer 1 are read as the response content of the read request 1. The response content of the read request 1 is returned to the processing core 1, and the information record 1 in the read incomplete list is invalidated.

[0097] The access offset indicated by the set read request 2 is 256 bytes from address 255. Based on this, the buffer 1 in the read data cache is located according to the pointer 1, and then the last 256 bytes of data in the buffer 1 are read as the response content of the read request 2. The response content of the read request 2 is returned to the processing core 2, and the information record 2 in the read incomplete list is invalidated.

[0098] After the information record 1 and the information record 2 are invalidated, the buffer 1 is released. In this way, the buffer 1 can be used to store data of a new cache line in the future.

[0099] In the embodiment of the application, a plurality of read requests corresponding to the same memory address are fused into one read request. The response content of each of the plurality of read requests can be obtained by executing the one read request, which reduces the number of consumptions of resources such as hit check resources and bandwidth resources, thereby reducing overall resource consumption. In addition, the plurality of read requests are fused into one read request for execution, which effectively improves the efficiency of data reading and improves cache performance.

[0100] In some embodiments, the incomplete buffer is a write incomplete list; the first data request and the second data request are write requests; and the first data request further includes an input buffer address and first to-be-written data.

[0101] After receiving the first data request of the first processing core, a first information record of the first data request is added to the write incomplete list, and the first information record includes a first request identifier, a first memory address, and an input buffer address.

[0102] The write incomplete list further includes a second information record of each second data request; the first information record and the second information record both include the first memory address and the input buffer address, and the first information record and the second information record have different request identifiers. In at least one second data request, one second data request that has been issued for execution is a target data request after fusion (corresponding to a target information record).

[0103] After receiving the first data request of the first processing core, the first to-be-written data is written to a second buffer corresponding to the input buffer address.

[0104] Specifically, the size of the second buffer is the same as the size of a cache line. The first data to be written is less than or equal to the size of the second buffer. At this time, the first data to be written is written into the second buffer according to the input buffer address and the access offset carried by the first data request.

[0105] The input buffer address is used to locate the specific position of the second buffer in the input buffer, and the access offset carried by the first data request is used to locate the specific writing position of the first data to be written in the second buffer.

[0106] In addition, the second buffer not only writes the first data to be written, but also writes the second data to be written carried by each second data request. The writing method of the second data to be written is the same as that of the first data to be written, which will not be described here.

[0107] It should be noted that if multiple data requests write data to the second buffer in sequence and the writing positions overlap, the data written later will cover the data written earlier.

[0108] For the target data request that has been issued for execution, when the target data request hits a cache line, the full amount of data stored in the second buffer is read. The full amount of data refers to all the data stored in the second buffer, and the full amount of data is written into the storage area corresponding to the cache line in the cache memory. After the writing is successful, a write success message is returned as the response of the target request.

[0109] Specifically, the full amount of data includes the first data to be written; in some cases, the full amount of data may also include all or part of the second data to be written. Through one read-write operation, the full amount of data in the second buffer is written into the storage area corresponding to the cache line in the cache memory.

[0110] Then the write success message is returned as the response content of the first data request to the first processing core, and the write success message is returned as the response content of each second data request to the corresponding second processing core.

[0111] For example, the cache receives the write request 1 sent by the processing core 1, and the write request 1 includes: the memory address 1, the data to be written 1, the input buffer address 1, and the access offset 1. The size of the data to be written 1 is 256 bytes; the access offset 1 indicates that the address 0 is offset by 256 bytes; the input buffer address 1 is the address of the buffer 2 in the input buffer; the size of the buffer 2 is 512 bytes.

[0112] The information record of the write request 1 is added to the write-incomplete list, and then the data to be written 1 is written into the buffer 2 according to the input buffer address 1 and the access offset 1. Since no other information record containing the memory address 1 is found in the write-incomplete list, the write request 1 is issued for execution.

[0113] If the write request 2 sent by the processing core 2 is received in the process of executing the write request 1, the write request 2 includes: the memory address 1, the to-be-written data 2, the input buffer address 1, and the memory access offset 2. The size of the to-be-written data 2 is 256 bytes; the memory access offset 2 indicates that the address 255 is taken as a starting point and offset by 256 bytes.

[0114] An information record of the write request 2 is added to the write-incomplete list; since the information record 1 in the write-incomplete list contains the memory address 1, the write request 2 is no longer issued for execution, but enters a waiting state. The to-be-written data 2 is written into the buffer 2 according to the input buffer address 1 and the memory access offset 2. The to-be-written data 1 and the to-be-written data 2 exactly fill the buffer 2.

[0115] In the process of executing the write request 1, a hit check is performed on the write request 1. When the write request 1 hits a cache line, the full amount of data (i.e., the to-be-written data 1 and the to-be-written data 2) stored in the buffer 2 is read; and then the full amount of data is written into a storage area corresponding to the cache line in the cache memory.

[0116] After the writing is successful, a write success message is returned to the processing core 1, and at the same time, the information record 1 in the write-incomplete list is invalidated. The write success message is returned to the processing core 2, and at the same time, the information record 2 in the write-incomplete list is invalidated.

[0117] In the embodiments of the present application, the to-be-written data of multiple write requests corresponding to the same memory address are saved in the same buffer, and after the multiple write requests are fused into one write request, the to-be-written data of the multiple write requests can be written into the cache memory at one time by executing the fused write request, without performing a hit check and writing the cache memory for each write request, thereby effectively reducing the number of consumptions of resources such as a hit check resource and a bandwidth resource, reducing overall resource consumption, and improving internal bandwidth. Secondly, the multiple write requests are fused into one write request for execution, which effectively improves the efficiency of writing data and further improves cache performance.

[0118] Based on the same technical concept, the embodiments of the present application provide a computer device, as shown in Figure 5 The computer device includes at least one artificial intelligence chip 100 and a memory 501 connected with the at least one artificial intelligence chip 100. In the embodiments of the present application, the specific connection medium between the artificial intelligence chip 100 and the memory 501 is not limited, Figure 5 For example, the artificial intelligence chip 100 and the memory 501 are connected through a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0119] In the embodiments of the present application, the memory 501 stores instructions executable by the at least one artificial intelligence chip 100, and the at least one artificial intelligence chip 100 can execute the steps of the cached request fusion method by executing the instructions stored in the memory 501. The artificial intelligence chip 100 is the control center of the computer device, can connect various parts of the computer device through various interfaces and lines, and realize the cached request fusion by running or executing the instructions stored in the memory 501 and calling the data stored in the memory 501. Optionally, the artificial intelligence chip 100 can include one or more processing units, and the artificial intelligence chip 100 can integrate an application processor and a modem processor. The application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 501 can be implemented on the same chip, and in some embodiments, they can also be implemented on separate chips respectively.

[0120] The artificial intelligence chip 100 can be a general processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware processor execution or executed by a combination of hardware and software modules in the processor.

[0121] The memory 501 can be used to store non-volatile software programs, non-volatile computer executable programs and modules as a non-volatile computer readable storage medium. The memory 501 can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. The memory 501 is any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer device, but is not limited thereto. The memory 501 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used for storing program instructions and / or data.

[0122] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium storing a computer program executable by a computer device, which, when running on the computer device, causes the computer device to perform the steps of the above-mentioned method for fusing cached requests.

[0123] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0124] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer device or other programmable data processing devices produce a machine that implements the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 apparatus for performing the functions specified in the block or blocks.

[0125] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more processes and / or blocks Figure 1 the functions specified in the block or blocks.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the processes specified in the flowchart Figure 1 one or more processes and / or blocks Figure 1 the steps specified in the block or blocks.

[0127] Although preferred embodiments of the application have been described herein, changes and modifications can be suggested to one skilled in the art and are intended to be encompassed within the scope of the appended claims. It is the intent, therefore, to be limited only as indicated by the scope of the claims appended hereto.

[0128] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the application can be practiced otherwise than as specifically described herein.

Claims

1. A method of fusing cached requests, the method comprising: The method comprises: receiving a first data request of a first processing core, the first data request comprising a first memory address; when there is at least one second data request corresponding to the first memory address in an uncompleted buffer, merging the first data request and the at least one second data request into a target data request, the uncompleted buffer being used to store the received and uncompleted data requests; executing the target data request to obtain a target request response; the target request response comprising the response content of the first data request and the at least one second data request respectively.

2. The method of claim 1, wherein, The step of merging the first data request and the at least one second data request into a target data request when there is at least one second data request corresponding to the first memory address in an uncompleted buffer comprises: when there is at least one second data request corresponding to the first memory address in the uncompleted buffer, taking one second data request that has been issued for execution in the at least one second data request as the target data request; placing the first data request in a waiting state.

3. The method of claim 1, wherein, The first data request further comprises a first request identifier; and after receiving the first data request of the first processing core, the method further comprises: adding a first information record of the first data request in the uncompleted buffer, the first information record comprising the first request identifier and the first memory address.

4. The method of claim 1, wherein, After executing the target data request to obtain a target request response, the method further comprises: returning the response content of the first data request to the first processing core; returning the response content of each second data request to the corresponding second processing core.

5. The method according to any one of claims 1 to 4, characterized in that, The uncompleted buffer is a read uncompleted list, and the first data request and the second data request are read requests; The step of executing the target data request to obtain a target request response comprises: when the target data request hits a cache line, reading the data of the cache line from a cache memory based on the tag information of the cache line; adding the data of the cache line as the target request response to a first buffer in a read data cache.

6. The method of claim 5, wherein, The read uncompleted list comprises a target information record of the target data request; After adding the data of the cache line as the target request response to a first buffer in a read data cache, the method further comprises: adding a target pointer pointing to the first buffer in the target information record; reading the response content of the target data request from the first buffer according to the target pointer and the memory access offset carried by the target data request.

7. The method of claim 5, wherein, The read uncompleted list further comprises a first information record of the first data request; After adding the data of the cache line as the target request response to a first buffer in a read data cache, the method further comprises: adding a target pointer pointing to the first buffer in the first information record; reading the response content of the first data request from the first buffer according to the target pointer and the memory access offset carried by the first data request.

8. The method of any one of claims 1 to 4, wherein, The unfinished buffer is a write unfinished list; the first data request and the second data request are write requests; The first data request further comprises an input buffer address and first data to be written; After receiving the first data request of the first processing core, the method further comprises: writing the first data to be written into a second buffer corresponding to the input buffer address.

9. The method of claim 8, wherein, The execution of the target data request obtains a target request response, which comprises: when the target data request hits a cache line, reading full data stored in the second buffer; and writing the full data into a storage area corresponding to the cache line in the cache memory; After the writing is successful, a write success message is taken as the target request response.

10. The method of claim 9, wherein, After the write success message is taken as the target request response, the method further comprises: taking the write success message as the response content of the first data request and the at least one second data request respectively.

11. An artificial intelligence chip, characterized by, The method comprises: a processing core, a cache and a main memory; The cache is used to execute the method according to any one of claims 1 to 10.

Images