Method for reordering buffering and products related thereto

Abstract

The present disclosure relates to a method for reordering buffering and related products thereof, including a reordering buffering device, an artificial intelligence processor, an apparatus, a board card and a computer readable storage medium. The apparatus can be included in a computing processing device of a combined processing device, which can include one or more data processing devices. The aforementioned combined processing device can also include an interface device and other processing devices. The computing processing device and the other processing devices interact with each other to complete a user-specified computing operation. The combined processing device can also include a storage device connected to the apparatus and the other processing devices, respectively, for storing data of the apparatus and the other processing devices. Through the scheme of the present disclosure, the memory access efficiency of data can be significantly improved.

Description

Technical Field

[0001] This disclosure generally relates to the field of computers. More specifically, this disclosure relates to a method for reordering a buffer, a reordering buffer apparatus for performing the aforementioned method, an artificial intelligence processor, a board, an apparatus, and a computer-readable storage medium. Background Technology

[0002] In traditional processor design, instructions that are independent of each other can be out of order. To achieve this, traditional processors typically use a reorder buffer (ROB) to record the actual order of all instructions and then release them in order after execution. However, for high-performance artificial intelligence (AI) processors, the instructions in the instruction set are more geared towards the needs of AI algorithms. Therefore, the amount of data computed, the functionality implemented, and / or the computation time required by a single instruction may be far greater than that of a traditional processor instruction. Thus, how to efficiently utilize the reorder buffer in AI computing scenarios to accelerate processor processing speed has become a key research focus. Summary of the Invention

[0003] In view of the technical problems mentioned in the background section above, this disclosure proposes an efficient reordering buffer scheme. Using the scheme of this disclosure, the data access speed in a cluster architecture of an artificial intelligence processor can be significantly improved, thereby accelerating the execution of instructions from the cluster and the overall operational performance of the processor. To this end, this disclosure provides a scheme for reordering buffers in several aspects, including...

[0004] In a first aspect, this disclosure provides a reordering buffer device disposed at the interface between dies of an artificial intelligence processor, the artificial intelligence processor including multiple dies and a storage device, each die including one or more clusters for performing operations, each cluster including multiple processor cores, the reordering buffer device including: a reordering buffer circuit configured to reside data read from a predetermined physical address range of the storage device; a receiving circuit configured to receive a read request from the cluster for the storage device, wherein the read request includes a target physical space address for reading data from the storage device; a determining circuit configured to determine not to send the read request when the target physical space address is within the predetermined physical address range; and a sending circuit configured to read the resided data corresponding to the target physical space address from the reordering buffer circuit, and send it to the cluster as the data read by the read request.

[0005] In a second aspect, this disclosure provides an artificial intelligence processor, comprising: a plurality of bare cores, wherein the bare cores communicate with each other via a bare core-to-bare core interface, and each bare core includes one or more clusters, each cluster including a plurality of processor cores; and the reordering buffer device described in the first aspect above, which is arranged at the interface between the bare dies for returning corresponding resident data in response to read requests in the clusters.

[0006] In a third aspect, this disclosure provides a method for a reordering buffer, applied to a reordering buffer device disposed at an interface between dies of an artificial intelligence processor, the artificial intelligence processor including a plurality of dies and a storage device, each die including one or more clusters for performing operations, the method comprising: residing data read from a predetermined physical address range of the storage device; receiving a read request from a cluster for the storage device, wherein the current read request includes a target physical space address for reading data from the storage device; determining not to send the current read request when the target physical space address is within the predetermined physical address range; and reading the residing data corresponding to the target physical space address to send to the cluster as data read by the current read request.

[0007] In a fourth aspect, this disclosure provides an apparatus for a reordering buffer, comprising: a processor; and a memory storing program instructions for reordering the buffer, which, when executed by the processor, cause the method described in the third aspect above to be performed.

[0008] In a fifth aspect, this disclosure provides a computer-readable storage medium storing program instructions for reordering a buffer, which, when executed by a processor, cause the method described in the third aspect above to be implemented.

[0009] In a sixth aspect, this disclosure provides a board that includes the artificial intelligence processor described in the second aspect or the device described in the fourth aspect.

[0010] Through the solutions provided in the above aspects, the reordering buffer of this disclosure can reduce the data return operations of the storage device for read requests from multiple clusters, thereby saving the bandwidth of the memory access unit. Furthermore, in some embodiments, by increasing the residence of data in the reordering buffer, the return of data to the cluster can be accelerated. In other embodiments, by merging the current read request with the previous read request based on the storage address—that is, selecting the corresponding data from the previous read request for return without considering the current read request—the processing speed of read requests can be improved, fully utilizing the data caching function of the ROB device. Attached Figure Description

[0011] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the present disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0012] Figure 1 This is a simplified block diagram schematically illustrating an artificial intelligence processor according to an embodiment of the present disclosure;

[0013] Figure 2 This is a schematic diagram illustrating a detailed structural block diagram of a reordering buffer device according to an embodiment of the present disclosure;

[0014] Figure 3 This is a simplified flowchart illustrating a method for reordering buffers according to the present disclosure;

[0015] Figure 4 This is a schematic structural block diagram illustrating one possible implementation of an artificial intelligence processor according to an embodiment of the present disclosure;

[0016] Figure 5 This is a flowchart schematically illustrating details of a method for reordering a buffer according to an embodiment of the present disclosure;

[0017] Figure 6 This is a structural diagram of a board according to an embodiment of the present disclosure;

[0018] Figure 7 This is a structural diagram illustrating a combined processing apparatus according to an embodiment of the present disclosure;

[0019] Figure 8 This is a schematic diagram showing the internal structure of a computing device according to an embodiment of the present disclosure;

[0020] Figure 9 This is a schematic diagram illustrating the internal structure of a processor core according to an embodiment of the present disclosure; and

[0021] Figure 10 This is a schematic diagram illustrating the data writing process between processor cores of different clusters according to embodiments of the present disclosure. Detailed Implementation

[0022] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0023] It should be understood that the terms "first," "second," "third," and "fourth," etc., in the claims, specification, and drawings of this disclosure are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0024] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0025] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0026] To improve data access efficiency in artificial intelligence computing platforms, this disclosure innovatively proposes setting a reordering buffer ("ROB") at the die-to-die (D2D) interface in the artificial intelligence processor. By setting such an ROB, memory access requests (including read and / or write requests) from the cluster can be processed uniformly. In the context of this disclosure, the artificial intelligence processor can be a multi-core processor with a hierarchical architecture. This multi-core processor can be implemented on one or more of the aforementioned bare dies, each bare die being a system-on-a-chip (SoC), and can include multiple of the aforementioned clusters, with each cluster including multiple cores. It can be seen that the multi-core processor is constructed in a system-on-a-chip-cluster-core hierarchy.

[0027] In the scheme disclosed herein, the cluster in the bare chip can be connected to an off-chip memory device via a ROB device to read or write corresponding data to the off-chip memory device. The off-chip memory device may, for example, be a dynamic random access memory (“DRAM”).

[0028] As mentioned earlier, to achieve efficient data access in the cluster, this disclosure proposes a "merging" operation on read requests from the cluster at the ROB device to avoid frequent reads from off-chip storage devices. Specifically, for multiple read requests targeting the same storage address, the scheme of this disclosure will retain the corresponding data obtained from the storage device by the initial read request at the ROB device. Based on this, subsequent read requests targeting the same storage address can be "merged" with the previous read request; that is, for subsequent read requests, the scheme of this disclosure will no longer read the corresponding data from the storage device. Instead, the ROB device directly returns the corresponding data from the retained data to the cluster that issued the read request. In some implementation scenarios, such a "merging" operation can be initiated or triggered in different ways, such as through hardware or software instructions. Similarly, a similar mechanism can be used to cancel such a "merging" operation, thereby making the data residency at the ROB device more flexible and controllable.

[0029] The specific embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0030] Figure 1 This is a simplified block diagram schematically illustrating an artificial intelligence (“AI”) processor 100 according to an embodiment of the present disclosure. It will be understood that the AI ​​processor described herein can be any of the following... Figure 6 The AI ​​processor 601 to be described has one or more processor cores so that it can execute multiple tasks in parallel or collaboratively process the same task.

[0031] like Figure 1 As shown, the artificial intelligence processor 100 may include a bare die 102 and clusters 106 arranged within the bare die, wherein each cluster exemplarily includes four processor cores 108. The bare dies communicate with each other via a D2D interface. As previously described, the present disclosure proposes to provide a reordering buffer (“ROB”) device 112 at the D2D interface, which is connected to the storage device 110 to enable corresponding resident data in the storage device.

[0032] In this disclosure, the processor core shown in the figures can be an arithmetic logic unit (ALU) or computational unit in an artificial intelligence processor, which can be used to perform various computational tasks, including computational tasks involving tensor or vector type data. In some scenarios, a cluster including multiple processor cores has a corresponding cluster identifier (“ID”) and can initiate data access requests to storage device 110 individually via the ROB device of this disclosure, such as read and write requests for data in the storage device. Storage device 110 can be various types of off-chip memory, such as dynamic random access memory (“DRAM”), which stores various data or intermediate results related to the cluster performing computational tasks.

[0033] To achieve efficient data access, the ROB device 112 of this disclosure is positioned between the cluster and the storage device to uniformly handle data interaction between the cluster and the storage device. To achieve a "merging" operation of read requests from different clusters, the ROB device 112 can initially record multiple different read requests to form a historical read request record, where each read request is associated with a specific storage address on the storage device for data retrieval and storage. In one implementation scenario, a specific storage space can be set up in the ROB device 112 to store the access addresses of these read requests (i.e., the storage addresses of the storage devices), and these access addresses can be used to distinguish and record these previous read requests.

[0034] Furthermore, the ROB device 112 can reside the data corresponding to these read requests in, for example, a reordering buffer circuit mentioned below. Based on this, when a read request with the same storage address as a read request in a historical read request record is received, the ROB device 112 can "merge" the received read request with the previously recorded read request. In other words, the ROB device 112 will not forward the received read request to the storage device; instead, it directly uses the resident data of the historical read request corresponding to the received read request to provide feedback to the cluster. In some application scenarios, to flexibly control the ROB device, the present disclosure also proposes using various instructions to start or stop the "merge" operation and data "residence" operation of the ROB device as described above. Therefore, the ROB device of the present disclosure will operate in a more user-friendly and efficient manner, thereby improving the data access efficiency of the cluster.

[0035] It should be noted that although the diagram only shows two bare dies and each dies includes four clusters, it is for illustrative purposes only. In some application scenarios, the AI ​​processor may include more bare dies, and each bare die may include fewer or more than four clusters, and each cluster may also include fewer or more than four processor cores. Therefore, the scheme disclosed herein does not limit the number of bare dies, clusters, and processor cores. In addition, the ROB devices of this disclosure can also be flexibly configured. For example, one ROB device can be configured for each cluster, and the ROB devices can be connected in a fully interconnected manner, such as via interconnect or crossbar. When data in a specific segment of physical address space in the storage device (i.e., the predetermined physical address segment of this disclosure) is allocated for data residency of multiple ROB devices, each ROB device can reside the corresponding data from the storage device to the ROB device according to the equally divided physical address segment.

[0036] Figure 2This is a schematic diagram illustrating a detailed structural block diagram of the reordering buffer device 112 according to an embodiment of the present disclosure. It will be understood that, due to... Figure 2 The ROB device 112 shown is also Figure 1 The ROB device shown is therefore the same as the description of the ROB device above.

[0037] like Figure 2 As shown, the ROB device 112 of this disclosure may include a reordering buffer circuit 201, a receiving circuit 202, a determination circuit 203, and a transmitting circuit 204.

[0038] In terms of functionality, the reordering buffer circuit 201 can be configured to retain data read from a predetermined physical address range of the storage device. The receiving circuit 202 can be configured to receive read requests from the cluster for the aforementioned storage device, wherein the read request includes the target physical space address from which data is to be read from the storage device. Next, the determining circuit 203 can be configured to determine not to send the read request if the target physical space address is within the predetermined physical address range. "Not sending" here means merging the current read request with previously received read requests for the same target physical space address, thus disregarding the current read request and not accessing the storage device for it. Finally, the sending circuit 204 is configured to read the retained data corresponding to the target physical space address from the reordering buffer circuit and send it to the cluster as the data read by the read request.

[0039] The main functions of each circuit in the ROB device 112 have been described above. However, it is understood that, depending on different embodiments and implementation scenarios, each of the above circuits may perform more specific or additional operations. The specific operations performed by each circuit will be described in detail below.

[0040] In one embodiment, the reordering buffer circuit 201 can be configured to reside data read from a predetermined physical address range of the storage device. The resided data may be corresponding data initially retrieved from the storage device in response to read requests from clusters, and these multiple read requests may target data at different storage addresses of the storage device. In one implementation scenario, the read request of this disclosure may include a task flag bit used to indicate that the task is jointly executed by multiple clusters. Further, the flag may be a flag assigned to a specific cluster, such as cluster 1, cluster 2, and cluster 3, which perform the same task. In this case, the sending circuit can be configured to return corresponding resided data to read requests from multiple clusters targeting the same target physical address.

[0041] To achieve effective control over the data residing in the reordering buffer circuit, the reordering buffer circuit 201 can also be configured to maintain the resident data during the execution of a first instruction containing one or more read requests for the same memory address. Subsequently, after the execution of the first instruction, the corresponding data residing in the reordering buffer circuit is released. In other words, the lifetime of the resident data here is the execution period of the first instruction. It is understood that in this application scenario, the ROB device of this disclosure can implement data resident and release on an instruction-by-instruction basis. Based on this, data obtained from multiple read requests of the same instruction can be shared by multiple clusters. When the instruction finishes execution, the ROB device can release the data corresponding to the multiple read requests.

[0042] In some application scenarios, during the execution of the first instruction, if a second instruction is detected reading corresponding data from the reordering buffer circuit, the reordering buffer circuit will retain the corresponding data instead of releasing it after the first instruction is executed, as described above. This extended residency method improves the utilization rate of resident data and avoids frequent data retrieval from the storage device.

[0043] In some application scenarios, to more effectively control the residence time and release time of resident data, this disclosure proposes to include time window information of the corresponding data residing in the reordering buffer circuit in the read request from the cluster. The reordering buffer circuit is configured to reside the corresponding data within the time window indicated by the time window information and release the corresponding data outside the time window. With this time window set, the read request can include a start read request and an end read request. The start read request can include a start time indication in the time window information indicating the start of resident data, and the end read request can include an end time indication in the time window information indicating the end of resident data. As an example application, for a read request instruction for a storage device (e.g., DDR) issued by cluster 0, a start signal (i.e., the aforementioned start time indication) is set in its first burst, and for the last burst, an end signal (i.e., the aforementioned end time indication) is set. From the moment the ROB device receives the start request from the start signal from cluster 0 until the moment it receives the end request from the end signal from cluster 0, all read data requested by cluster 0 will be returned from the ROB device. The time from the start request to the end request is also known as the time window mentioned above.

[0044] For cluster 1, which has the same task (“job”) as cluster 0, its read storage device behaves similarly to cluster 0, meaning it also has a similar time window. Assume time window A for cluster 0 and time window B for cluster 1. When time windows A and B overlap in time dimension, the end time of the earlier-ending time window will be automatically extended to the end time of the later-ending time window. In other words, the reordering buffer circuitry is configured to extend the residence time of corresponding data into a relatively later time window. When the two time windows do not overlap, their respective residence times are valid within their respective time windows, and once the requested residence data is fed back to the cluster, the ROB device will automatically release the residence data or declare the residence data invalid.

[0045] As the receiving end of the reordering buffer device 112, the receiving circuit 202 can be configured to receive current read requests, wherein the current read requests are used to read corresponding data from the storage address of the storage device. Depending on the scenario, the current read request can be a read request from the cluster or a read request from the control device. Here, the control device is used to perform control operations on the cluster, and in some scenarios, it forwards the cluster's read requests to the receiving circuit 202. In some scenarios, the control device can also send read requests to the receiving circuit 202 independently, and the corresponding data as feedback for the read request can also be sent directly from the reordering buffer device to the cluster.

[0046] To facilitate efficient control of the ROB device, this disclosure also proposes using hardware instructions to initiate data residency and release operations on the ROB device. Accordingly, the receiving circuit 202 can be configured to receive a third instruction to begin residing corresponding data within the reordering buffer circuit. In response to the receiving circuit receiving the third instruction, the reordering buffer circuit can be configured to initiate the operation of residing the corresponding data at the storage address of the storage device, so that the cluster can directly read the corresponding resided data from the reordering buffer circuit via a read request. In the scenario where corresponding data is resided in the ROB device via the third instruction, the receiving circuit can also be configured to receive a write request for the storage address from the cluster, and the reordering buffer circuit is further configured to automatically release the corresponding resided data in response to receiving the write request for the storage address.

[0047] In other words, in the above application scenario, a third instruction is used to initiate data residency on the ROB device and release the corresponding data upon detecting a corresponding write request. This release operation is performed because a write request to the storage address will modify or update the data at that storage address. In this case, the resided data on the ROB device will differ from the data on the storage device, thus requiring its release from the ROB device so that the updated data can be resided in the reordering buffer circuit in response to a new read request. For the third instruction, in one embodiment, it can be a synchronization instruction, and one of its fields can be used to indicate its use as an instruction to initiate data residency on the ROB device. Furthermore, in the context of this disclosure, the release operation can be implemented in different ways, such as directly deleting, removing, or invalidating the corresponding data from the reordering buffer circuit.

[0048] Depending on the application scenario, the write and read requests mentioned above can originate from the same cluster. Alternatively, the write and read requests can also originate from different clusters performing the same task. Therefore, when multiple read requests from different clusters for the same storage address occur, the solution disclosed herein only needs to consider one of the read requests, i.e., these read requests are "merged," thereby reducing the number of read requests processed. Similarly, write requests for the same storage address from one cluster can release resident data previously shared by multiple clusters from the ROB device.

[0049] To implement the "merging" operation of read requests, the determination circuit 203 can be configured to determine not to send the current read request to the storage device when the storage address read by the current read request is the same as the storage address read by the previous read request. Determining not to send means merging the current read request with the corresponding previous read request, so that the corresponding previous read request replaces the current read request, and no further read request forwarding operation is performed to the storage device.

[0050] In response to a determination not to send a current read request to the storage device, the sending circuit 204 can be configured to read resident data corresponding to the storage address from the reordering buffer circuit, and send it to the cluster as the data read by the current read request. It can be seen that since the data corresponding to the same storage address has previously resided in the reordering buffer circuit, the operation of reading data corresponding to the current read request from the storage device will not be performed again. Instead, previously resident data for that storage address is directly read from the reordering buffer circuit. Since both are for the same storage address, the data obtained from the storage device based on the current read request and the previous read request are also the same. Based on this, the solution of this disclosure directly uses previously stored resident data in the ROB device as feedback for the current read request.

[0051] In some application scenarios, to achieve flexible manipulation of data in the ROB device, the receiving circuit is also configured to receive a fourth instruction for storing corresponding data in the reordering buffer, and the reordering buffer circuit is further configured to store the corresponding data in response to receiving the fourth instruction. Correspondingly, the receiving circuit is also configured to receive a fifth instruction for releasing the corresponding data from the reordering buffer. In this case, the reordering buffer circuit is further configured to release the corresponding data in response to receiving the fifth instruction for releasing the corresponding data from the reordering buffer. It is understood that the use of the fourth instruction can enable prefetching of stored data; for example, the cluster can pre-store corresponding data in the ROB device without immediately returning it to the cluster. Therefore, the fourth instruction can be regarded as a prefetch instruction for corresponding data, and it can be included in the I / O instructions of the storage device, thereby reducing the latency of instruction processing. In addition, the fifth instruction can be a data release instruction sent by the cluster to the ROB device, which can utilize the order-preserving characteristics between the cluster and the ROB device to ensure that the ROB device releases the stored data in the normal order.

[0052] The above combination Figure 2 The reordering buffer device 112 of this disclosure has been described in detail. It is understood that the above description is merely exemplary and not limiting, and those skilled in the art can modify the implementation of the reordering buffer device based on the teachings of this disclosure. For example, although the reordering buffer device is depicted in the figures as a circuit arrangement, it can also be implemented by software modules or units. For example, the receiving circuit and the transmitting circuit can be replaced by transceiver units implemented in software code to realize the reception of read requests and / or write requests and the transmission of resident data.

[0053] Figure 3 This is a simplified flowchart schematically illustrating a method 300 for reordering buffers according to this disclosure. It will be understood that method 300 herein can be derived from... Figure 1 and Figure 2 The reordering buffer shown is used to perform the operation, therefore the above description of the operation of the reordering buffer also applies to the description of method 300 below.

[0054] like Figure 3As shown, in step S302, data read from a predetermined physical address segment of the storage device is stored. As mentioned earlier, this predetermined physical address segment can be a physical storage address space of the storage device, which can be specified in advance by the programmer through program instructions. Furthermore, this storage can be an action taken in response to a read request from the cluster, for example, when the data for the read request has not previously been read from the storage device and is not stored on the reordering buffer circuit. Next, in step S304, a read request for the storage device is received from the cluster.

[0055] In step S306, if the target physical space address is within a predetermined physical address range, it is determined that the current read request will not be sent. By determining whether the storage addresses are the same, it can be determined whether a previous read request can replace the current read request; that is, whether the resident data for the previous read request can be used for the current read request. Finally, in step S308, since the storage addresses are the same, the resident data corresponding to the target physical space address is read and sent to the cluster as the data read by the current read request. Clearly, through this "merging" operation, the scheme of this disclosure does not require frequent access to the storage device for the same read request (for the same storage address), thereby accelerating the memory access efficiency of data (especially tensor or vector data).

[0056] Figure 4 This is a schematic structural block diagram illustrating one possible implementation of an artificial intelligence processor 400 according to an embodiment of the present disclosure. It is understood that the artificial intelligence processor 400 here can be considered as... Figure 1 This is one embodiment of the artificial intelligence processor 100, therefore the description of the artificial intelligence processor 100 also applies to the following embodiments. Figure 4 Description of the artificial intelligence processor 400.

[0057] As shown in the figure, the AI ​​processor 400 exemplarily includes four raw dies (die1 to die4), each of which includes four clusters and an associated ROB device 112. The ROB devices 112 can be fully connected via a crossbar switch 402 and are connected to the storage device 110. In operation, a predetermined physical address range in the storage device 110 is allocated for data residency on the four ROB devices 112; that is, each ROB device 112 will reside data corresponding to one-quarter of the predetermined physical address range. When a read request from one cluster of a raw die arrives at its corresponding ROB device 112, it will determine whether there is corresponding residency data in order to "merge" the read request. If a previous read request exists targeting the same physical address as the read request, the read request will not be sent to the storage device. Instead, the residency data corresponding to the previous read request targeting the same physical address will be fed back to the cluster. Conversely, when a ROB device associated with the cluster does not have a corresponding previous read request, it can interact with the other three ROB devices via a crossbar switch so that the other three ROB devices can determine whether there is corresponding resident data. When corresponding resident data exists, the aforementioned "merge" operation is performed, and the resident data is fed back to the corresponding cluster.

[0058] Understandable, Figure 4 The architecture of the AI ​​processor 400 shown is merely exemplary and not limiting. Those skilled in the art will also conceive of modifications to this structure based on the teachings of this disclosure. For example, in addition to each bare die being associated with one ROB device, ROB devices can also be associated with each cluster. Thus, the ROB device configuration of this disclosure can also be a hierarchical or layered structure, i.e., a first-level ROB device architecture consisting of ROB devices associated with bare dies and a second-level ROB device architecture consisting of ROB devices associated with clusters, thereby accelerating data access between the cluster and storage devices and reducing the frequency of data reads.

[0059] Figure 5 This is a schematic diagram illustrating a detailed flowchart of a method 500 for reordering a buffer according to an embodiment of the present disclosure. It will be understood that method 500 can be... Figure 1 , Figure 2 and Figure 4 The reordering buffer shown is used for execution, and it can be considered as a possible implementation of method 300. Therefore, the preceding text combines... Figures 1-4 The descriptions made also apply to the descriptions below.

[0060] like Figure 5As shown, in step S502, a start read request for residing data is received. Then, in step S504, in response to receiving the aforementioned start read request, the corresponding data is residing, thereby accumulating historical data (including historical read requests and corresponding residing data). For example, this residing involves receiving a read request from the cluster or control device for the storage device, residing the data read from the storage device in the ROB device, and returning it to the cluster. Since this is still in the initial stage of data residing accumulation, the comparison operation of the storage address of the read request may not be performed or may be performed less frequently. In some scenarios, when multiple read requests for the same storage address arrive at the ROB device, the ROB device may directly select one read request to perform the operation of reading the corresponding data from the storage device, and ignore the remaining read requests without performing any further read operations.

[0061] In step S506, a current read request for reading corresponding data from the storage address of the storage device is received. In step S508, in response to receiving the current read request, a previous read request for the same storage address is searched from the historical read request record. In step S510, in response to finding a previous read request for the same storage address, it is determined not to send the current read request to the storage device. In other words, since the previous read request and the current read request are for the same storage address, and the corresponding data for that storage address already resides at the ROB device, there is no need to forward the current read request to the storage address again. Therefore, in step S512, the corresponding data residing in the previous read request is directly read and sent to the cluster as the corresponding data for the current read request.

[0062] At step S514, a read termination request for releasing data is received. Finally, at step S516, in response to receiving the read termination request, the corresponding resident data is automatically released. As previously stated, data resident remains valid during the time window between the read start request and the read termination request. Conversely, when the time window has expired, the resident data will be released or invalidated.

[0063] The above combination Figure 5 The method for reordering buffers disclosed herein is described in detail. Figure 5 The method described herein, based on a "merging" strategy for read requests to storage addresses, reduces the number of read requests to storage addresses via the ROB device, thereby saving memory access bandwidth. Furthermore, through data residency and release, the method of this disclosure also supports flexible control over data residing on the ROB device.

[0064] The device side in this embodiment may be an artificial intelligence chip or board, etc. Figure 6 A schematic diagram of the structure of a board 600 according to an embodiment of this disclosure is shown. Figure 6 As shown, board 600 includes a chip (or "processing chip") 601, which is a system-on-chip (SoC) integrating one or more combined processing devices. These combined processing devices are artificial intelligence computing units used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in cloud intelligence. A significant characteristic of cloud intelligence applications is the large volume of input data, placing high demands on the platform's storage and computing capabilities. Board 600 in this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and substantial computing power.

[0065] Chip 601 is connected to external device 603 via external interface device 602. External device 603 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 603 to chip 601 via external interface device 602. The calculation results from chip 601 can be transmitted back to external device 603 via external interface device 602. Depending on the application scenario, external interface device 602 may have different interface forms, such as a PCIe interface.

[0066] The board 600 also includes a storage device 604 for storing data, which includes one or more memory cells 605. The storage device 604 is connected to and transmits data with the controller 606 and the chip 601 via a bus. The controller 606 in the board 600 is configured to regulate the state of the chip 601. Therefore, in one application scenario, the controller 606 may include a microcontroller (MCU).

[0067] Figure 7 This is a structural diagram illustrating the combined processing device 700 in chip 601 of this embodiment. (As shown...) Figure 7 As shown, the combined processing device 700 includes a computing device 701, an interface device 702, a processing device 703, and a DRAM 704.

[0068] The computing device 701 is configured to perform user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 703 through the interface device 702 to jointly complete the user-specified operations.

[0069] Interface device 702 is used to transmit data and control commands between computing device 701 and processing device 703. For example, computing device 701 can obtain input data from processing device 703 via interface device 702 and write it to on-chip storage device of computing device 701. Further, computing device 701 can obtain control commands from processing device 703 via interface device 702 and write them to on-chip control cache of computing device 701. Alternatively or optionally, interface device 702 can also read data from storage device of computing device 701 and transmit it to processing device 703.

[0070] The processing device 703, as a general-purpose processing device, performs basic controls including but not limited to data transfer and starting / stopping the computing device 701. Depending on the implementation, the processing device 703 may be one or more types of processors, including but not limited to digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, the computing device 701 disclosed herein can be considered as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 701 and the processing device 703 are considered together, they are considered to form a heterogeneous multi-core structure.

[0071] DRAM 704 is used to store data to be processed. It is DDR memory, typically 16GB or larger, and is used to store data in computing device 701 and / or processing device 703. Although in Figure 7 As not shown, the ROB device of this disclosure can be arranged between the cluster and DRAM 704 within the computing device 701 for performing according to Figures 1-5 The described operating steps.

[0072] Figure 8A schematic diagram of the internal structure of computing device 701 is shown. Computing device 701 is used to process input data such as computer vision, speech, natural language processing, and data mining. The computing device 701 in the diagram adopts a multi-core hierarchical architecture design. As a system-on-a-chip, computing device 701 includes multiple clusters, each cluster containing multiple processor cores, which can be used to execute the tasks disclosed herein. In other words, computing device 701 is constructed in a hierarchical structure of system-on-a-chip, clusters, and processor cores.

[0073] From the perspective of system-on-a-chip hierarchy, such as Figure 8 As shown, the computing device 701 includes an external storage controller 801, a peripheral communication module 802, an on-chip interconnect module 803, a synchronization module 804, and multiple clusters 805.

[0074] There can be multiple external storage controllers 801; two are shown as an example in the figure. These controllers are used to respond to access requests from the processor core to access external storage devices, such as… Figure 9 The DRAM 904 (which can be equivalent to the storage device of this disclosure) is used to read data from or write data to external devices. The peripheral communication module 802 receives control signals from the processing device 703 via the interface device 702 to initiate the computing device 701 to perform tasks. The on-chip interconnect module 803 connects the external storage controller 801, the peripheral communication module 802, and multiple clusters 805 to transmit data and control signals between the modules. The synchronization module 804 is a global barrier controller (GBC) used to coordinate the working progress of each cluster and ensure information synchronization. The multiple clusters 805 are the computing core of the computing device 701. Four are shown exemplary in the figure; however, with hardware development, the computing device 701 disclosed herein may also include 8, 16, 64, or even more clusters 805.

[0075] From the perspective of cluster hierarchy, such as Figure 8 As shown, each cluster 805 includes multiple processor cores (IPU cores) 806 and one memory core (MEM core) 807. Four processor cores 806 are shown exemplarily in the diagram; this disclosure does not limit the number of processor cores 806. Its internal architecture is as follows... Figure 8 As shown. Each processor core 806 includes three main modules: a control module 91, an arithmetic module 92, and a storage module 93.

[0076] The control module 91 coordinates and controls the operation of the computation module 92 and the storage module 93 to complete the deep learning task. It includes an instruction fetch unit (IFU) 911 and an instruction decode unit (IDU) 912. The instruction fetch unit 911 fetches instructions from the processing device 703, while the instruction decode unit 912 decodes the fetched instructions and sends the decoding result as control information to the computation module 92 and the storage module 93.

[0077] The computation module 92 includes a vector operation unit 921 and a matrix operation unit 922. The vector operation unit 921 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit 922 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution.

[0078] Storage module 93 is used to store or move related data, including neuron RAM (NRAM) 931, weight RAM (WRAM) 932, input / output direct memory access (IODMA) 933, and move direct memory access (MVDMA) 934. NRAM 931 is used to store input, output data, and intermediate results for the processor core 806 to calculate; WRAM 932 is used to store the weights of the deep learning network; IODMA 933 controls the memory access of NRAM 931 / WRAM 932 and DRAM 704 through broadcast bus 1009; MVDMA 934 controls the memory access of NRAM 931 / WRAM 932 and SRAM 808.

[0079] Back Figure 8 The storage core 807 is primarily used for storage and communication, namely storing shared data or intermediate results among processor cores 806, and performing communication between cluster 805 and DRAM 704, communication between clusters 805, and communication between processor cores 806. In other embodiments, the storage core 807 has scalar operation capabilities and is used to perform scalar operations.

[0080] Storage core 807 includes a shared memory unit (SRAM) 808, a broadcast bus 809, a cluster direct memory access (CDMA) module 810, and a global direct memory access (GDMA) module 811. SRAM 808 acts as a high-performance data relay station. Data multiplexed between different processor cores 806 within the same cluster 805 does not need to be obtained from DRAM 704 by each processor core 806 individually. Instead, it is relayed between processor cores 806 via SRAM 808. Storage core 807 only needs to quickly distribute the multiplexed data from SRAM 808 to multiple processor cores 806, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip I / O access. It is understood that although the diagram shows the cluster accessing data from DRAM via the storage core, this configuration is merely exemplary and not restrictive. As previously stated, in order to improve access to DRAM, this disclosure proposes to set up a distributed ROB device between the cluster and DRAM to perform the read request "merging" and resident data feedback operations as described above.

[0081] Broadcast bus 809, CDMA 810, and GDMA 811 are used to perform communication between processor cores 806, communication between clusters 805, and data transfer between cluster 805 and DRAM 704, respectively. These will be explained separately below.

[0082] The broadcast bus 809 is used to complete high-speed communication between the processor cores 806 within the cluster 805. In this embodiment, the broadcast bus 809 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point (i.e., data transmission from one processor core to another) data transmission. Multicast is a communication method that transmits a piece of data from SRAM 808 to several specific processor cores 806. Broadcast is a communication method that transmits a piece of data from SRAM 808 to all processor cores 806, and is a special case of multicast.

[0083] CDMA 810 is used to control SRAM 808 access between different clusters 805 within the same computing device 701. Figure 10 This diagram illustrates the operation of CDMA810 when one processor core attempts to write data to another processor core in a different cluster. In this application scenario, the same computing device comprises multiple clusters. For simplicity, only clusters 0 and 1 are shown in the diagram. Both clusters 0 and 1 contain multiple processor cores; similarly, for ease of explanation, only processor core 0 is shown in cluster 0, and only processor core 1 is shown in cluster 1. Processor core 0 intends to write data to processor core 1.

[0084] First, processor core 0 sends a unicast write request to write data into its local SRAM 0. CDMA 0 acts as the master and CDMA 1 acts as the slave. The master pushes the write request to the slave, that is, the master sends the write address AW and the write data W to transmit the data to SRAM 1 of cluster 1. Then, the slave sends a write response B as a response. Finally, processor core 1 of cluster 1 sends a unicast read request to read the data from SRAM 1.

[0085] Back Figure 8 The GDMA 811, in conjunction with the external memory controller 801, controls memory access from the SRAM 808 of the cluster 805 to the DRAM 704, or reads data from the DRAM 704 into the SRAM 808. As previously described, communication between the DRAM 704 and the NRAM 931 or WRAM 932 can be achieved through two channels. The first channel is a direct connection between the DRAM 704 and the NRAM 931 or WRAM 932 via the IODAM 933; the second channel involves first transmitting data between the DRAM 704 and SRAM 808 via the GDMA 811, and then transmitting data between the SRAM 808 and the NRAM 931 or WRAM 932 via the MVDMA 934. Although the second channel appears to require more components and has a longer data flow, in some embodiments, the bandwidth of the second channel is actually much greater than that of the first channel. Therefore, communication between the DRAM 704 and the NRAM 931 or WRAM 932 may be more efficient via the second channel. The embodiments of this disclosure can select the data transmission channel according to their own hardware conditions.

[0086] In other embodiments, the functions of GDMA 811 and IODMA 933 can be integrated into the same component. For ease of description, this disclosure treats GDMA 811 and IODMA 933 as different components. For those skilled in the art, any component whose implemented functions and achieved technical effects are similar to this disclosure is within the scope of protection of this disclosure. Furthermore, the functions of GDMA 811, IODMA 933, CDMA 810, and MVDMA 934 can also be implemented by the same component. Similarly, any component whose implemented functions and achieved technical effects are similar to this disclosure is within the scope of protection of this disclosure.

[0087] The above combination Figures 6-10The hardware and software architecture and internal structure of this disclosure are described in detail. It is understood that the above description is merely exemplary and not restrictive. Depending on different application scenarios and hardware specifications, those skilled in the art may also make changes to the board (or artificial intelligence device) and its internal structure disclosed herein, and such changes shall still fall within the protection scope of this disclosure.

[0088] Based on the foregoing description, those skilled in the art will understand that this application also discloses a device including a processor and a memory. Specifically, the memory may store program instructions for reordering a buffer, which, when executed by the processor, implement the functionality of this application. Figures 1-5 The described operational steps. Furthermore, since the solution of this application can be implemented by computational program instructions, this application also discloses a computer-readable storage medium or computer program product storing a computer program / instructions for reordering a buffer, thereby achieving the combination... Figures 1-5 The described operating steps.

[0089] The solutions disclosed herein have been described in detail above with reference to the accompanying drawings. Depending on the application scenario, the devices or apparatus disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The devices or apparatus disclosed herein can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare.

[0090] Furthermore, the devices or apparatuses disclosed herein can also be used in application scenarios related to artificial intelligence, big data, and / or cloud computing, such as cloud computing, edge computing, and terminals. In one or more embodiments, the high-power devices or apparatuses according to the disclosed scheme can be applied to cloud devices (e.g., cloud servers), while the low-power devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling, and collaborative work of end-to-cloud or cloud-edge-end integration.

[0091] It should be noted that, for the sake of brevity, this disclosure describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solutions disclosed herein are not limited by the order of the described actions. Therefore, based on the disclosure or teachings of this document, those skilled in the art will understand that some steps can be performed in a different order or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this disclosure can be considered optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions disclosed herein. In addition, depending on the solution, the description of some embodiments in this disclosure may have different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this disclosure can also be referred to the relevant descriptions of other embodiments.

[0092] In terms of specific implementation, based on the disclosure and teachings of this document, those skilled in the art will understand that several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various units in the device or apparatus embodiments described above, this document divides them based on logical functions, but in actual implementation, there may be other division methods. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationships between different units or components, the connections discussed above in conjunction with the accompanying drawings can be direct or indirect couplings between units or components. In some scenarios, the aforementioned direct or indirect couplings involve communication connections utilizing interfaces, where the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

[0093] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this disclosure. Additionally, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically independently.

[0094] In some implementation scenarios, the integrated unit described above can be implemented as a software program module. If implemented as a software program module and sold or used as an independent product, the integrated unit can be stored in a computer-readable storage device (CMSDD). Therefore, when the disclosed solution is embodied in a software product (e.g., a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to cause a computer device (e.g., a personal computer, server, or network device) to execute some or all of the steps of the method described in the embodiments of this disclosure. The aforementioned memory may include, but is not limited to, various media capable of storing program code, such as USB flash drives, flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0095] In other implementation scenarios, the integrated units described above can also be implemented in hardware, i.e., as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the hardware structure of the circuit may include, but is not limited to, physical devices, which may include, but are not limited to, transistors or memristors. Therefore, the various devices described herein (e.g., computing devices or other processing devices) can be implemented using appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage unit or storage device can be any suitable storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), such as resistive random access memory ("RRAM"), dynamic random access memory ("DRAM"), static random access memory ("SRAM"), enhanced dynamic random access memory ("EDRAM"), high bandwidth memory ("HBM"), hybrid memory cube ("HMC"), ROM, and RAM, etc.

[0096] The foregoing can be better understood in accordance with the following terms:

[0097] Clause A1. A reordering buffer is disposed at the interface between dies of an artificial intelligence processor, the artificial intelligence processor including multiple dies and storage devices, each die including one or more clusters for performing operations, each cluster including multiple processor cores, the reordering buffer comprising:

[0098] A reordering buffer circuit configured to reside data read from a predetermined physical address segment of the storage device;

[0099] A receiving circuit configured to receive a read request from the cluster for the storage device, wherein the read request includes a target physical space address from which data is to be read from the storage device;

[0100] A determination circuit, configured to determine not to send the read request when the target physical space address is within the predetermined physical address range; and

[0101] A sending circuit configured to read resident data corresponding to a target physical space address from the reordering buffer circuit, and send it to the cluster as the data read by the read request.

[0102] Clause A2, the reordering buffer apparatus according to Clause A1, wherein the read request includes a task flag bit for marking that the task is performed jointly by multiple clusters, and the sending circuit is configured to return corresponding resident data to read requests from the multiple clusters for the same target physical space address.

[0103] Clause A3, the reordering buffer device according to Clause A2, wherein the cluster has an associated cluster identifier, and the task flag bit indicates multiple clusters performing a task together by marking multiple cluster identifiers.

[0104] Clause A4. The reordering buffer device according to Clause A1, wherein the read request includes time window information of the corresponding data residing in the reordering buffer circuit, wherein the reordering buffer circuit is configured to reside the corresponding data within the time window indicated by the time window information and to release the corresponding data outside the time window.

[0105] Clause A5. The reordering buffer device according to Clause A4, wherein the read request includes a start read request and an end read request, wherein the start read request includes a start time indication in time window information for indicating the start of residing in the corresponding data, and the end read request includes an end time indication in the time window information for indicating the end of residing in the corresponding data.

[0106] Clause A6. The reordering buffering device according to Clause A4, wherein some or all of the plurality of clusters send their respective read requests and the time windows for responding to their respective read requests overlap for a period of time, the reordering buffering circuit is configured to extend the residence time of the corresponding data into a relatively later time window.

[0107] Clause A7. The reordering buffer device according to Clause A1, wherein the reordering buffer circuit is configured to:

[0108] During the execution of a first instruction containing one or more read requests for the same storage address, the corresponding data remains resident; and

[0109] After the first instruction is executed, the corresponding data residing in the reordering buffer is released.

[0110] Clause A8. The reordering buffer device according to Clause A7, wherein the reordering buffer circuit is configured to:

[0111] During the execution of the first instruction by the cluster, in response to the detection of a second instruction to read the corresponding data on the reordering buffer circuit, the corresponding data is kept resident until it is released after the second instruction is executed.

[0112] Clause A9. The reordering buffer device according to Clause A1, wherein the receiving circuit is further configured to receive a third instruction indicating that the corresponding data should begin to reside in the reordering buffer circuit, and in response to receiving the third instruction, the reordering buffer circuit is configured to initiate a resident operation of the corresponding data at the storage address of the storage device so that the cluster can directly read the corresponding data from the reordering buffer circuit via the read request.

[0113] Clause A10, the reordering buffer device according to Clause A5, wherein the receiving circuit is further configured to receive a write request for the storage address, and the reordering buffer circuit is further configured to automatically release the resident corresponding data in response to receiving a write request for the storage address.

[0114] Clause A11, the reordering buffer as described in Clause A10, wherein the write request and the read request are received from the same cluster.

[0115] Clause A12, the reordering buffer as described in Clause A10, wherein the write request and the read request are received from different clusters performing the same task.

[0116] Clause A13, the reordering buffer device according to Clause A9, wherein the third instruction is a synchronization instruction, and a field of the synchronization instruction is used to indicate its use as an instruction to initiate residency.

[0117] Clause A14. The reordering buffer apparatus according to Clause A1, wherein the receiving circuitry is further configured to receive a fourth instruction for residing the corresponding data in the reordering buffer, and the reordering buffer circuitry is further configured to residing the corresponding data in response to receiving the fourth instruction; and

[0118] The receiving circuit is further configured to receive a fifth instruction to release the corresponding data from the reordering buffer, and the reordering buffer circuit is further configured to release the corresponding data in response to receiving the fifth instruction to release the corresponding data from the reordering buffer.

[0119] Clause A15, the reordering buffer device as described in Clause A14, wherein the fourth instruction is included in an I / O instruction for the storage device.

[0120] Clause A16, an artificial intelligence processor, comprising:

[0121] Multiple bare cores, wherein the bare cores communicate with each other via bare-core interfaces, and each bare core comprises one or more clusters, each cluster comprising multiple processor cores; and

[0122] The reordering buffer, as described in any one of clauses A1-A15, is located at the interface between dies to return corresponding resident data in response to read requests in the cluster.

[0123] Clause A17, the artificial intelligence processor as described in Clause A16, wherein each naked core is provided with an associated reordering buffer, or the plurality of naked cores share a single reordering buffer.

[0124] Clause A18, the artificial intelligence processor described in Clause A16 or A17, wherein each cluster has an associated reordering buffer, or the multiple clusters share a single reordering buffer.

[0125] Clause A19, the artificial intelligence processor as described in Clause A18, wherein each cluster is connected to a corresponding reordering buffer, and the reordering buffers are connected in a fully connected manner.

[0126] Clause A20, the artificial intelligence processor according to any one of clauses A16-A19, wherein each reordering buffer is allocated for residing data in a portion of the predetermined physical address range stored in the storage device.

[0127] Clause A21. A method for a reordering buffer, applied to a reordering buffer device disposed at an interface between dies of an artificial intelligence processor, the artificial intelligence processor including a plurality of dies and storage devices, each die including one or more clusters for performing computations, the method comprising:

[0128] Data read from a predetermined physical address range of the storage device is stored thereon;

[0129] Receive a read request from the cluster for the storage device, wherein the current read request includes the target physical space address from which data is to be read from the storage device;

[0130] If the target physical space address is within the predetermined physical address range, it is determined that the current read request will not be sent; and

[0131] Read the resident data corresponding to the target physical space address, and send it to the cluster as the data read by the current read request.

[0132] Clause A22, the method according to Clause A21, wherein the read request includes a task flag bit for marking that the task is executed jointly by multiple clusters, and the method includes:

[0133] For read requests from the multiple clusters targeting the same physical address, the corresponding resident data is returned.

[0134] Clause A23, the method according to Clause A22, wherein the cluster has an associated cluster identifier, and the task tag bit indicates multiple clusters performing a task together by tagging multiple cluster identifiers.

[0135] Clause A24, the method according to Clause A21, wherein the read request includes time window information of the corresponding data residing in the reordering buffer, wherein the method includes residing the corresponding data within the time window indicated by the time window information and releasing the corresponding data outside the time window.

[0136] Clause A25. The method according to Clause A24, wherein the read request includes a start read request and an end read request, wherein the start read request includes a start time indication in time window information for indicating the start of residing in the corresponding data, and the end read request includes an end time indication in time window information for indicating the end of residing in the corresponding data.

[0137] Clause A26, the method according to Clause A24, wherein some or all of the plurality of clusters send their respective read requests, and the method further comprises:

[0138] If the time windows of the respective read requests overlap, the residence time of the corresponding data will be extended to a later time window.

[0139] Clause A27. The method described pursuant to Clause A21 further includes:

[0140] During the execution of a first instruction containing one or more read requests for the same storage address, the corresponding data remains resident; and

[0141] After the first instruction is executed, the corresponding data residing in the reordering buffer is released.

[0142] Clause A28, the methods described pursuant to Clause A27, further include:

[0143] During the execution of the first instruction by the cluster, in response to the detection of a second instruction to read the corresponding data on the reordering buffer, the corresponding data is kept resident until it is released after the second instruction is executed.

[0144] Clause A29, the method according to Clause A21, further includes receiving a third instruction to indicate the commencement of residing the corresponding data, and in response to receiving the third instruction, initiating an operation to residing the corresponding data at the storage address of the storage device, so that the cluster can directly read the corresponding data from the residing data via the read request.

[0145] Clause A30, the method according to Clause A25, further includes receiving a write request for the storage address, and automatically releasing the corresponding resident data in response to receiving the write request for the storage address.

[0146] Clause A31, the method according to Clause A30, wherein the write request and the read request are received from the same cluster.

[0147] Clause A32, the method according to Clause A30, wherein the write request and the read request are received from different clusters performing the same task.

[0148] Clause A33, the method according to Clause A29, wherein the third instruction is a synchronization instruction, and a field of the synchronization instruction is used to indicate its use as an instruction to initiate residency.

[0149] Clause A34, the method according to Clause A31, wherein the method further comprises: receiving a fourth instruction for residing the corresponding data in the reordering buffer, and residing the corresponding data in response to receiving the fourth instruction; and

[0150] The method further includes receiving a fifth instruction to release the corresponding data from the reordering buffer, and releasing the corresponding data in response to receiving the fifth instruction to release the corresponding data from the reordering buffer.

[0151] Clause A35, the method according to Clause A34, wherein the fourth instruction is included in an I / O instruction for the storage device.

[0152] Clause A36, a device for reordering buffers, comprising:

[0153] Processor; and

[0154] A memory storing program instructions for reordering a buffer, which, when executed by a processor, cause the method described under any one of clauses A21-A35 to be implemented.

[0155] Clause A37. A computer-readable storage medium storing program instructions for reordering a buffer, which, when executed by a processor, cause the implementation of the method according to any one of clauses A21-A35.

[0156] Clause A38, a board comprising an artificial intelligence processor as described in any one of Clauses A16-A20 or a device as described in Clause A36.

[0157] While the embodiments of this disclosure are described above, the content is merely an example for the purpose of facilitating understanding of this disclosure and is not intended to limit the scope or application scenarios of this disclosure. Any person skilled in the art can make any modifications and changes in form and detail of the implementation without departing from the spirit and scope disclosed herein; however, the patent protection scope of this disclosure shall still be determined by the scope defined in the appended claims.

Claims

1. A reordering buffer device disposed at the interface between dies of an artificial intelligence processor, the artificial intelligence processor comprising multiple dies and storage devices, each die comprising one or more clusters for performing operations, each cluster comprising multiple processor cores, the reordering buffer device comprising: A reordering buffer circuit configured to reside in data read from a predetermined physical address segment of the storage device; A receiving circuit configured to receive a read request from the cluster for the storage device, wherein the read request includes a target physical space address from which data is to be read from the storage device; A determination circuit is configured to determine not to send the read request when the target physical space address is within the predetermined physical address range; as well as A sending circuit configured to read resident data corresponding to a target physical space address from the reordering buffer circuit, and send it to the cluster as the data read by the read request.

2. The reordering buffer apparatus of claim 1, wherein the read request includes a task flag bit for marking that the task is jointly executed by multiple clusters, and the sending circuit is configured to return corresponding resident data to read requests from the multiple clusters for the same target physical space address.

3. The reordering buffer apparatus of claim 2, wherein the cluster has an associated cluster identifier, and the task flag indicates multiple clusters performing the task together by marking multiple cluster identifiers.

4. The reordering buffer apparatus according to claim 1, wherein the read request includes time window information of the corresponding data residing in the reordering buffer circuit, wherein the reordering buffer circuit is configured to reside the corresponding data within the time window indicated by the time window information and release the corresponding data outside the time window.

5. The reordering buffer apparatus according to claim 4, wherein the read request includes a start read request and an end read request, wherein the start read request includes a start time indication in the time window information for indicating the start of residing in the corresponding data, and the end read request includes an end time indication in the time window information for indicating the end of residing in the corresponding data.

6. The reordering buffer apparatus of claim 2, wherein some or all of the plurality of clusters send their respective read requests and the time windows of their respective read requests overlap for a period of time, the reordering buffer circuit is configured to extend the residence time of the corresponding data to a relatively later time window.

7. The reordering buffer device according to claim 4, wherein the reordering buffer circuit is configured to: During the execution of a first instruction containing one or more read requests for the same storage address, the corresponding data remains resident; and After the first instruction is executed, the corresponding data residing in the reordering buffer is released.

8. The reordering buffer device according to claim 7, wherein the reordering buffer circuit is configured to: During the execution of the first instruction by the cluster, in response to the detection of a second instruction to read the corresponding data on the reordering buffer circuit, the corresponding data is kept resident until it is released after the second instruction is executed.

9. The reordering buffer apparatus of claim 4, wherein the receiving circuit is further configured to receive a third instruction indicating that the corresponding data should begin to reside in the reordering buffer circuit, and in response to receiving the third instruction, the reordering buffer circuit is configured to initiate a resident operation for the corresponding data at the storage address of the storage device, so that the cluster can directly read the corresponding data from the reordering buffer circuit via the read request.

10. The reordering buffer apparatus of claim 7, wherein the receiving circuit is further configured to receive a write request for the storage address, and the reordering buffer circuit is further configured to automatically release the corresponding resident data in response to receiving a write request for the storage address.

11. The reordering buffer apparatus of claim 10, wherein the write request and the read request are received from the same cluster.

12. The reordering buffer apparatus of claim 10, wherein the write request and the read request are received from different clusters performing the same task.

13. The reordering buffer apparatus of claim 9, wherein the third instruction is a synchronization instruction, and a field of the synchronization instruction is used to indicate its use as an instruction to initiate resident.

14. The reordering buffer apparatus of claim 4, wherein the receiving circuit is further configured to receive a fourth instruction for residing the corresponding data in the reordering buffer, and the reordering buffer circuit is further configured to residing the corresponding data in response to receiving the fourth instruction; and The receiving circuit is further configured to receive a fifth instruction to release the corresponding data from the reordering buffer, and the reordering buffer circuit is further configured to release the corresponding data in response to receiving the fifth instruction to release the corresponding data from the reordering buffer.

15. The reordering buffer device of claim 14, wherein the fourth instruction is included in an I / O instruction for the storage device.

16. An artificial intelligence processor, comprising: Multiple bare cores, wherein the bare cores communicate with each other through a bare core-to-bare core interface, and each bare core includes one or more clusters, each cluster including multiple processor cores; as well as The reordering buffer according to any one of claims 1-4 is arranged at the interface between dies for returning corresponding resident data in response to read requests in the cluster.

17. The artificial intelligence processor of claim 16, wherein each naked core is provided with an associated reordering buffer, or the plurality of naked cores share a single reordering buffer.

18. The artificial intelligence processor of claim 16, wherein each cluster is provided with an associated reordering buffer, or the multiple clusters share a single reordering buffer.

19. The artificial intelligence processor of claim 18, wherein each cluster is connected to a corresponding reordering buffer, and the reordering buffers are connected in a fully connected manner.

20. The artificial intelligence processor according to any one of claims 16-19, wherein each reordering buffer is allocated for residing data stored in a portion of the predetermined physical address segment of the storage device.

21. A method for a reordering buffer, applied to a reordering buffer device disposed at an interface between dies of an artificial intelligence processor, the artificial intelligence processor including a plurality of dies and storage devices, each die including one or more clusters for performing computations, the method comprising: Data read from a predetermined physical address range of the storage device is stored thereon; Receive a read request from the cluster for the storage device, wherein the current read request includes the target physical space address from which data is to be read from the storage device; If the target physical space address is within the predetermined physical address range, it is determined that the current read request will not be sent; as well as Read the resident data corresponding to the target physical space address, and send it to the cluster as the data read by the current read request.

22. The method of claim 21, wherein the read request includes a task flag bit for marking that the task is executed jointly by multiple clusters, and the method comprises: For read requests from the multiple clusters targeting the same physical address, the corresponding resident data is returned.

23. The method of claim 22, wherein the cluster has an associated cluster identifier, and the task tag bit indicates multiple clusters performing the task together by tagging multiple cluster identifiers.

24. The method of claim 21, wherein the read request includes time window information of the corresponding data residing in the reordering buffer, wherein the method includes residing the corresponding data within the time window indicated by the time window information and releasing the corresponding data outside the time window.

25. The method of claim 24, wherein the read request includes a start read request and an end read request, wherein the start read request includes a start time indication in time window information for indicating the start of residing in the corresponding data, and the end read request includes an end time indication in the time window information for indicating the end of residing in the corresponding data.

26. The method of claim 22, wherein some or all of the plurality of clusters send their respective read requests, and the method further comprises: If the time windows of the respective read requests overlap, the residence time of the corresponding data will be extended to a later time window.

27. The method of claim 24, further comprising: During the execution of a first instruction containing one or more read requests for the same storage address, the corresponding data remains resident; as well as After the first instruction is executed, the corresponding data residing in the reordering buffer is released.

28. The method of claim 27, further comprising: During the execution of the first instruction by the cluster, in response to the detection of a second instruction to read the corresponding data on the reordering buffer, the corresponding data is kept resident until it is released after the second instruction is executed.

29. The method of claim 24, further comprising receiving a third instruction to indicate the commencement of residing the corresponding data, and in response to receiving the third instruction, initiating an operation to residing the corresponding data at the storage address of the storage device, so that the cluster can directly read the corresponding data from the residing data via the read request.

30. The method of claim 27, further comprising receiving a write request for the storage address, and automatically releasing the resident corresponding data in response to receiving the write request for the storage address.

31. The method of claim 30, wherein the write request and the read request are received from the same cluster.

32. The method of claim 30, wherein the write request and the read request are received from different clusters performing the same task.

33. The method of claim 29, wherein the third instruction is a synchronization instruction, and a field of the synchronization instruction is used to indicate its use as an instruction to initiate residency.

34. The method of claim 31, wherein the method further comprises: Receive a fourth instruction for residing the corresponding data in the reordering buffer, and in response to receiving the fourth instruction, reside the corresponding data. as well as The method further includes receiving a fifth instruction to release the corresponding data from the reordering buffer, and releasing the corresponding data in response to receiving the fifth instruction to release the corresponding data from the reordering buffer.

35. The method of claim 34, wherein the fourth instruction is included in an I / O instruction for the storage device.

36. A device for reordering a buffer, comprising: processor; as well as A memory storing program instructions for reordering a buffer, which, when executed by a processor, cause the method according to any one of claims 21-35 to be implemented.

37. A computer-readable storage medium storing program instructions for reordering a buffer, which, when executed by a processor, cause the method according to any one of claims 21-35 to be implemented.

38. A board comprising an artificial intelligence processor according to any one of claims 16-20 or a device according to claim 36.

Images