Data Processing Circuit, Artificial Intelligence Chip, Data Processing Method and Device

Abstract

Embodiments of the present disclosure provide a data processing circuit, an artificial intelligence chip, a data processing method, and a device. Instructions are processed by multiple processing units. Each processing unit independently determines whether there is a data hazard between the instructions processed by this processing unit based on the information of the first instruction received by this processing unit and the information of the second instruction already sent by this processing unit. In the case where there is a data hazard between the instructions processed by some of the processing units, instructions can still be sent by other processing units, thereby reducing instruction congestion.

Description

Technical Field

[0001] The present disclosure relates to the field of integrated circuit design technologies, and in particular, to a data processing circuit, an artificial intelligence chip, a data processing method, and a device. Background Art

[0002] With the continuous development of artificial intelligence and high-performance computing, the amount of data that a processing system needs to process is becoming increasingly large. During the processing, a large amount of data needs to be transferred between internal storage space and external storage space. During the transfer, a read-write data hazard may occur, that is, at least two instructions read and write data to the same storage address respectively. When a read-write data hazard occurs, it is necessary to control the read instruction and the write instruction to prevent data read-write errors. Related technologies generally only determine whether there is a data hazard between two consecutive instructions. Once a data hazard occurs, subsequent instructions cannot be issued, which easily leads to instruction congestion. Summary of the Invention

[0003] The present disclosure provides a data processing circuit, an artificial intelligence chip, a data processing method, and a device.

[0004] According to a first aspect of an embodiment of the present disclosure, a data processing circuit is provided. The circuit includes a plurality of processing units, and each processing unit in the plurality of processing units includes: an instruction queue for caching a first instruction received; a sent information queue for caching information of each of at least one second instruction that has been sent; a detection unit for detecting, based on the information of the first instruction and the information of each second instruction, whether the first instruction meets an instruction sending condition, and when the instruction sending condition is met, taking out the first instruction from the instruction queue for sending; the information includes at least one of read-write type information and address information.

[0005] Optionally, the detection unit is configured to: when the sent information queue is not full, detect whether the first instruction meets the instruction sending condition based on the read-write type information of the first instruction, the read-write type information of the second instruction, the address information of the first instruction, and the address information of the second instruction; when the sent information queue is full, detect whether the first instruction meets the instruction sending condition based on the address information of the first instruction and the address information of the second instruction.

[0006] Optionally, when the sent information queue is not full, the detection unit is configured to: determine that the first instruction does not meet the instruction sending condition when the read / write type information of any second instruction in the sent information queue is different from the read / write type information of the first instruction and the address information is the same as the address information of the first instruction; determine that the first instruction meets the instruction sending condition when the address information of each second instruction in the sent information queue is different from the address information of the first instruction, or when the target address information and the target read / write type information of the same second instruction in the sent information queue do not simultaneously meet the condition that the target address information is the same as the address information of the first instruction and the target read / write type information is different from the read / write type information of the first instruction.

[0007] Optionally, when the sent information queue is full, the detection unit is configured to: determine that the first instruction does not meet the instruction sending condition when there is at least one instruction in the sent information queue whose read / write type information is different from the read / write type information of the first instruction; determine that the first instruction meets the instruction sending condition when the read / write type information of each second instruction in the sent information queue is the same as the read / write type information of the first instruction.

[0008] Optionally, different processing units are used to process instructions sent by different thread groups.

[0009] Optionally, the circuit further includes a first instruction distribution unit configured to: receive the first instructions sent by each thread group, where the first instructions sent by each thread group carry the identification information of the corresponding thread group; and distribute the first instructions sent by each thread group to the corresponding processing unit respectively based on the identification information carried in the first instructions sent by each thread group.

[0010] Optionally, instructions sent by different processing units have different priorities; the circuit further includes: an instruction arbitration unit configured to receive the first instructions sent by each processing unit and send the first instructions sent by each processing unit in sequence based on the priorities of the first instructions sent by each processing unit.

[0011] Optionally, the first instruction includes the storage address of the bypass information, and the bypass information corresponding to the first instruction is stored under the storage address of the bypass information; the circuit further includes: a second instruction distribution unit, configured to decouple the original instruction carrying the bypass information to obtain the decoupled original instruction and the bypass information, store the bypass information under the storage address of the bypass information, generate the first instruction based on the decoupled original instruction and the storage address of the bypass information, and send the first instruction to the instruction queue; a bus control unit, configured to generate a target instruction based on the first instruction and the storage address of the bypass information, and send the target instruction.

[0012] Optionally, the bus control unit is further configured to: clear the information of the second instruction that has been processed and completed from the sent queue.

[0013] Optionally, the first instruction includes a write instruction, and the bypass information includes the first bypass information corresponding to the write instruction; the circuit further includes: a first storage unit, configured to store the first bypass information.

[0014] Optionally, the bus control unit is configured to: extract the storage address of the first bypass information from the write instruction, obtain the first bypass information from the storage address of the first bypass information, generate the target instruction based on the first bypass information and the write instruction, and send the target instruction.

[0015] Optionally, the first instruction includes a read instruction, and the bypass information includes the second bypass information corresponding to the read instruction; the circuit further includes: a second storage unit, configured to store the second bypass information.

[0016] Optionally, the bus control unit is further configured to: use the read instruction as the target instruction.

[0017] Optionally, the bus control unit is further configured to: receive the target data read by the read instruction, where the target data carries the storage address of the second bypass information; write the target data into the storage address of the second bypass information in the second storage unit.

[0018] Optionally, the bus control unit is further configured to: clear the bypass information corresponding to the first instruction from the storage address of the bypass information when the processing of the first instruction is completed.

[0019] Optionally, when the total number of the second instructions exceeds the length of the sent information queue, all the information in the sent information queue is set to invalid.

[0020] Optionally, the circuit further includes a statistical unit for statistically calculating the following information: the total number of the second instructions; the number of instructions that have been processed and completed among the respective second instructions; and the read / write type information of the respective second instructions; the detection unit is configured to: when all the information in the sent information queue is invalid and there is a second instruction with a read / write type information different from that of the first instruction, detect whether the first instruction meets the instruction sending condition based on the information statistically calculated by the statistical unit.

[0021] Optionally, when all the information in the sent information queue is invalid and there is a second instruction with a read / write type information different from that of the first instruction, if all the second instructions have been processed, it is determined that the first instruction meets the instruction sending condition.

[0022] Optionally, the information of the first instruction is written into the sent information queue when the first instruction is successfully sent.

[0023] According to a second aspect of the embodiments of the present disclosure, there is provided an artificial intelligence chip, including: the data processing circuit according to any one of the embodiments of the present disclosure; and a control unit configured to send the first instruction to an instruction queue in the data processing circuit.

[0024] According to a third aspect of the embodiments of the present disclosure, there is provided a data processing method, which is applied to a detection unit in the data processing circuit according to any one of the embodiments of the present disclosure, and the method includes: detecting whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction; when the instruction sending condition is met, taking out the first instruction from the instruction queue for sending; the information includes at least one of read / write type information and address information.

[0025] According to a fourth aspect of the embodiments of the present disclosure, there is provided a data processing device, which is applied to a detection unit in the data processing circuit according to any one of the embodiments of the present disclosure, and the device includes: a detection module configured to detect whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction; a sending module configured to take out the first instruction from the instruction queue for sending when the instruction sending condition is met; the information includes at least one of read / write type information and address information.

[0026] Embodiments of the present disclosure process instructions through multiple processing units. Each processing unit independently determines whether there is a data hazard between the instructions processed by this processing unit based on the information of the first instruction received by this processing unit and the information of the second instruction already sent by this processing unit. In the case where there is a data hazard between the instructions processed by some of the processing units, instructions can still be sent by other processing units, thereby reducing instruction congestion.

[0027] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure.

[0029] Figure 1 is a schematic diagram of a data processing circuit according to an embodiment of the present disclosure.

[0030] Figure 2 is a schematic diagram of the detection principle of a detection unit according to an embodiment of the present disclosure.

[0031] Figure 3 is a schematic diagram of a data processing circuit according to another embodiment of the present disclosure.

[0032] Figure 4 is a schematic diagram of instruction out-of-order issue according to an embodiment of the present disclosure.

[0033] Figure 5A and Figure 5B are respectively schematic diagrams of instruction decoupling and merging according to an embodiment of the present disclosure.

[0034] Figure 6 is a schematic diagram of an instruction sending process according to an embodiment of the present disclosure.

[0035] Figure 7 is an overall flowchart according to an embodiment of the present disclosure.

[0036] Figure 8 is a block diagram of an artificial intelligence chip according to an embodiment of the present disclosure.

[0037] Figure 9 is a flowchart of a data processing method according to an embodiment of the present disclosure.

[0038] Figure 10 is a block diagram of a data processing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0040] The terms used in the present disclosure are for the purpose of describing particular embodiments only and are not intended to limit the present disclosure. The singular forms "a", "the", and "said" used in the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. Additionally, the term "at least one" as used herein means any one of a plurality or any combination of at least two of a plurality.

[0041] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0042] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure and to make the above objects, features, and advantages of the embodiments of the present disclosure more apparent and understandable, the technical solutions in the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0043] When data is transferred between the internal storage space and the external storage space, read / write data hazards may occur, that is, at least two instructions read and write data to the same storage address respectively. When a read / write data hazard occurs, it is necessary to control the read instruction and the write instruction to prevent data read / write errors. For example, the data d1 is stored at the address 1 in the external storage space, and the control unit generates two instructions. One instruction is to read the data from the address 1 and write it into a register, and the other instruction is to write the data d2 to the address 1. The external storage space may include, but is not limited to, Double Data Rate Synchronous Dynamic Random Access Memory (DDR) or High Bandwidth Memory (HBM), etc. The control unit may include, but is not limited to, Central Processing Unit (CPU) or Graphics Processing Unit (GPU), etc. These two instructions are executed in different orders, and different execution results will be produced. If the read instruction is executed first and then the write instruction, the data written to the register is the original data d1 at the address 1. If the write instruction is executed first and then the read instruction, the data written to the register is the data d2 modified by the write instruction at the address 1. This data read / write error that may be caused by executing instructions of different read / write types on the same address is called a data hazard. In order to reduce data hazards, it is necessary to control the sending process of the read instruction and the write instruction. However, the related technology generally only judges whether there is a data hazard between the previous and the subsequent instructions. Once a data hazard occurs, subsequent instructions cannot be issued, which easily leads to instruction congestion.

[0044] Based on this, an embodiment of the present disclosure provides a data processing circuit 100, see Figure 1 and Figure 3 , the circuit 100 includes a plurality of processing units, and each processing unit 101 in the plurality of processing units includes:

[0045] An instruction queue 1011 for caching the received first instruction;

[0046] A sent information queue 1012 for caching the information of each of at least one second instruction that has been sent;

[0047] A detection unit 1013 for detecting whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction, and taking out the first instruction from the instruction queue for sending when the instruction sending condition is met;

[0048] The information includes at least one of read / write type information and address information.

[0049] In the prior art, generally, when there is a data hazard, it is necessary to stop sending instructions and wait for the processed instructions that have been sent to be completed before resending instructions. In the embodiments of the present disclosure, multiple independent data processing units 101 are adopted. Even if there is a data hazard in the instructions processed by one or some of the processing units, instructions can still be sent through other processing units. Only when there are data hazards in the instructions processed by all the processing units is it necessary to stop sending instructions. Therefore, the embodiments of the present disclosure can effectively improve the instruction sending efficiency and reduce instruction congestion.

[0050] Since each processing unit independently determines whether there is a data hazard between the instructions processed by the processing unit based on the information of the first instruction received by the processing unit and the information of the second instruction already sent by the processing unit. When there is a data hazard between the instructions processed by some of the processing units, instructions can still be sent through other processing units, thereby reducing instruction congestion.

[0051] The above instruction queue 1011 can be a First In First Out (FIFO) queue. Each time the instruction queue 1011 receives a first instruction sent by the upper control unit, it can cache the first instruction. The caching order of the first instruction in the instruction queue 1011 is the same as the order in which the instruction queue 1011 receives the first instruction. The first instruction can include a read instruction for reading data from a certain address in an external storage space (such as a hard disk). The first instruction can also include a write instruction for writing data to a certain address in the external storage space. The first instruction can include information corresponding to the first instruction, and the information includes address information and read / write type information. Among them, the address information is used to indicate the address requested to be accessed by the first instruction. For a read instruction, the address information represents the address where the data to be read is located (i.e., the data source); for a write instruction, the address information represents the address where the data needs to be written (i.e., the data destination). The read / write type information is used to indicate whether the first instruction is a read instruction or a write instruction.

[0052] In the above-mentioned sent information queue 1012, the information of each second instruction can be cached. The second instruction refers to an instruction that has been successfully sent. Among them, the second instruction can include both an instruction that has been successfully sent but not yet processed completely and an instruction that has been successfully sent and has been processed completely. It can be determined that the instruction has been processed completely when a notification message indicating that the instruction has been processed completely is received for the second instruction. The return condition of this notification message can be set based on the actual situation. For example, for a read instruction, when the data requested by the read instruction is returned to the data requester, the data requester returns the notification message. Another example is that for a write instruction, when the data carried in the write instruction is written to the specified data recipient, the data recipient returns the notification message. Of course, the actual situation is not limited to the above-listed methods.

[0053] The information of the second instruction can also include address information and read / write type information. For the meanings of the address information and read / write type information included in the second instruction, reference can be made to the meanings of the address information and read / write type information included in the first instruction, which will not be elaborated here. The address information and read / write type information in the sent information queue 1012 can be cached correspondingly, that is, the address information and read / write type information of the same second instruction are cached as one piece of information.

[0054] The detection unit 1013 can compare the information of the first instruction with each piece of information cached in the sent information queue 1012 respectively, so as to determine whether there is a data hazard between the first instruction and the sent second instruction. When there is a data hazard between the information of the first instruction and any second instruction, the first instruction is not sent. Only when there is no data hazard between the first instruction and any second instruction, the first instruction is sent. When there is a data hazard, the detection unit can detect again whether there is a data hazard after a certain time interval until the first instruction is taken out from the instruction queue 1011. Among them, the time interval can be one clock cycle or other durations.

[0055] The following will be combined with Figure 2 Describe the method for determining whether there is a data hazard. The detection unit 1013 can determine whether there is a data hazard based on at least one of the read / write type information and the address information. Specifically, it can first determine whether the sent information queue is full (step 201). When the sent information queue is not full, it can detect whether the first instruction meets the instruction sending condition based on the read / write type information of the first instruction, the read / write type information of the second instruction, the address information of the first instruction, and the address information of the second instruction.

[0056] Specifically, in the case where the address information of each second instruction in the sent information queue 1012 is different from the address information of the first instruction, it can be determined that the first instruction meets the instruction sending condition (step 202), and the first instruction is sent (step 207). For example, assume that the information included in the sent information queue 1012 is as shown in Table 1:

[0057] Table 1 Sent Information Queue

[0058] Instruction to which the information belongs Address information Read / write type information Instruction 1 A1 Read Instruction 2 A2 Write Instruction 3 A2 Read

[0059] Among them, Instruction 1, Instruction 2, and Instruction 3 are all second instructions. The information included in the sent information queue 1012, {A1, read}, is the information of Instruction 1, {A2, write} is the information of Instruction 2, and {A3, read} is the information of Instruction 3. Assume further that the address information in the first instruction is A3, A3 is different from A1, and A3 is different from A2. Since the address information in the first instruction is different from the address information of each of the sent second instructions, therefore, regardless of the read / write data types of the first instruction and each second instruction, because data read / write between different addresses is independent of each other, there is no data hazard between the first instruction and any second instruction, that is, the first instruction meets the sending condition, and thus the first instruction can be sent.

[0060] Still assume that the information included in the sent information queue 1012 is as shown in Table 1, and assume that the address information in the first instruction is A1. Since the first instruction and Instruction 1 target the same address, if the first instruction is sent at this time, in the case where the read / write type information of the first instruction and Instruction 1 is different, data read / write errors may occur, that is, there is a data hazard. Therefore, in this case, it is necessary to jointly judge whether the first instruction meets the instruction sending condition by combining the read / write type information and the address information.

[0061] In the case where there is target address information in the sent information queue 1012 that is the same as the address information of the first instruction, if there is target read / write type information in the sent information queue 1012 that is different from the data read / write type information of the first instruction, and the target read / write type information and the target address information belong to the same second instruction, it can be determined that the first instruction does not meet the instruction sending condition (step 203), and thus the first instruction is not sent (step 206).

[0062] When there is target address information in the sent information queue 1012 that is the same as the address information of the first instruction, if there is no target read / write type information in the sent information queue 1012 that is different from the data read / write type information of the first instruction, where the target read / write type information and the target address information belong to the same second instruction, it can be determined that the first instruction meets the instruction sending condition (step 202), and the first instruction is sent (step 207).

[0063] For example, in the above embodiment, assume that the read / write type information of the first instruction is write. Since the address information of the first instruction is the same as the address information of instruction 1, and the read / write type information of the first instruction is different from the read / write type information of instruction 1, there is a data hazard, and the first instruction does not meet the instruction sending condition. Assume that the read / write type information of the first instruction is read. Then, there is no second instruction that satisfies both the condition that the address information of the second instruction is the same as the address information of the first instruction and the condition that the read / write type information of the second instruction is different from the read / write type information of the first instruction. Therefore, there is no data hazard, and the first instruction meets the instruction sending condition.

[0064] When the sent information queue is full, it is possible to detect whether the first instruction meets the instruction sending condition based on the address information of the first instruction and the address information of the second instruction. Specifically, when the read / write type information of each second instruction in the sent information queue 1012 is the same as the read / write type information of the first instruction, it is determined that the first instruction meets the instruction sending condition (step 204), and the first instruction is sent (step 207).

[0065] For example, referring to Figure 4 , assume that the read / write type information of each item in the sent information queue 1012 is write, and the read / write type information of the first instruction is also write. Regardless of whether the address information (address 1, address 2,..., address n) in each write instruction is the same, there is no data hazard between the write instructions. Therefore, in this case, the first instruction can be directly sent. Similarly, assume that the read / write type information of each item in the sent information queue 1012 is read, and the read / write type information of the first instruction is also read. The first instruction can also be directly sent. The above situation is called instruction over-issuance because the number of instructions sent has exceeded the length of the sent information queue 1012. Through instruction over-issuance, the sending efficiency of multiple consecutive instructions with the same read / write type can be improved, and instruction congestion can be further reduced.

[0066] When there is at least one instruction in the sent information queue 1012 whose read / write type information is different from that of the first instruction, it is determined that the first instruction does not meet the instruction sending condition (step 205), and thus the first instruction is not sent (step 206). For example, the read / write type information of each item in the sent information queue 1012 includes both read and write; or, the read / write type information of each item in the sent information queue 1012 is all read, but the read / write type information of the first instruction is write; or, the read / write type information of each item in the sent information queue 1012 is all write, but the read / write type information of the first instruction is read. In the above three cases, regardless of the address information in the first instruction and the second instruction, the first instruction is not sent.

[0067] It should be noted that when performing instruction over-issuance, since the number of sent instructions has exceeded the length of the sent information queue 1012. For example, the number of already sent instructions is 5, and the sent information queue 1012 can cache a total of 4 instruction information (that is, the length of the sent information queue 1012 is 4). In this case, it is no longer possible to determine whether there is a data hazard based on the information in the sent information queue 1012. For example, assume that the over-issued instruction S0 is a read instruction for address A0, and the information {A0, read} of instruction S0 is not cached in the sent information queue 1012, and assume that the information of each item in the sent information queue 1012 is as shown in Table 2:

[0068] Address information Read / write type information A1 Read A2 Read A3 Read

[0069] When the second instruction corresponding to the information {A1, read} is processed and completed, the information {A1, read} is cleared from the sent information queue 1012, and at this time the sent information queue 1012 is not full. If the instruction queue 1011 receives an instruction S with the information {A0, write} k ,then according to the aforementioned method of judging data hazards, since there is no address information in the sent information queue 1012 that is the same as the address information of instruction S k ,therefore, if judged based on the information in the sent information queue 1012, instruction S k will meet the instruction sending condition. But in fact, since there is an instruction S0 in the already sent instructions whose read / write type information is different from that of instruction S k and the address information is the same, there is a data hazard between instruction S0 and instruction S k ,so in fact instruction S k does not meet the instruction sending condition. That is to say, in the case of instruction over-issuance, if the information in the sent information queue 1012 is still used to judge whether the instruction sending condition is met, a wrong judgment result may be obtained.

[0070] To improve the judgment accuracy in the case of instruction out-of-order issue, when the total number of the second instructions exceeds the length of the sent information queue, each piece of information in the sent information queue can be set to invalid. When each piece of information in the sent information queue is invalid and there is a second instruction with a read / write type information different from that of the first instruction, it is possible to detect whether the first instruction meets the instruction sending condition based on the statistical information of the sent instructions. In this case, the first instruction is sent only after all the sent second instructions are processed; as long as there is an unprocessed second instruction, the first instruction is not sent.

[0071] In some embodiments, the circuit further includes a statistical unit 1014 for statistically calculating the following information: the total number of the second instructions; the number of the processed instructions among each of the second instructions; and the read / write type information of each of the second instructions. The detection unit 1013 can detect whether the first instruction meets the instruction sending condition based on the information statistically calculated by the statistical unit 1014 when each piece of information in the sent information queue 1012 is invalid and there is a second instruction with a read / write type information different from that of the first instruction.

[0072] The detection unit 1013 can first determine whether the read / write type information of the first instruction is the same as that of each of the sent second instructions. When the read / write type information is different, the total number of the sent second instructions and the number of the processed instructions are obtained from the statistical unit 1014. The first instruction is determined to meet the instruction sending condition only when the total number of the sent second instructions is equal to the number of the processed instructions; otherwise, the first instruction is determined not to meet the instruction sending condition.

[0073] In some embodiments, different processing units 101 are used to process the instructions sent by different thread groups. Each processing unit 101 can be used to process the instructions sent by one or more thread groups, and the thread groups responsible for different processing units can be the same or different. For example, the processing unit 0 is used to process the instructions sent by the thread group 0, the processing unit 1 is used to process the instructions sent by the thread groups 1 and 2, and the processing unit 2 is used to process the instructions sent by the thread groups 3 and 4. To distribute the instructions sent by different thread groups to the corresponding processing units, the circuit further includes an instruction distribution unit 102 for receiving the first instructions sent by each thread group, where the first instructions sent by each thread group carry the identification information of the corresponding thread group; and distributing the first instructions sent by each thread group to the corresponding processing unit respectively based on the identification information carried in the first instructions sent by each thread group.

[0074] In some embodiments, the instructions sent by different processing units have different priorities; the circuit further includes: an instruction arbitration unit 103, configured to receive the first instructions sent by each processing unit 101, and successively send the first instructions sent by each processing unit 101 based on the priorities of the first instructions sent by each processing unit 101.

[0075] In some embodiments, the first instruction includes the storage address of the bypass information, and the bypass information corresponding to the first instruction is stored under the storage address of the bypass information; the circuit further includes: an instruction distribution unit 102, configured to decouple the original instruction carrying the bypass information to obtain the decoupled original instruction and the bypass information, store the bypass information under the storage address of the bypass information, generate the first instruction based on the decoupled original instruction and the storage address of the bypass information, and send the first instruction to the instruction queue; a bus control unit, configured to generate a target instruction based on the first instruction and the storage address of the bypass information, and send the target instruction.

[0076] It should be noted that, in the above embodiments, the instruction distribution unit for distributing instructions to each processing unit and the instruction distribution unit for decoupling the original instruction are the same instruction distribution unit 102. However, in practical applications, instruction distribution and instruction decoupling may also be performed by different instruction distribution units respectively.

[0077] The original instruction may carry some bypass information, which is irrelevant to the process of determining whether the first instruction meets the instruction sending condition. For example, for a write instruction, the bypass information may include, but is not limited to, the data to be written, the identification information for identifying the valid bit of the data to be written, etc. For a read instruction, the bypass information may include, but is not limited to, the target address for data reading, the register address storing the target address, etc. If this bypass information is always carried in the instruction, each processing unit needs additional storage space to store this bypass information, which will increase the area and power consumption of the data processing circuit. Therefore, in this embodiment, the bypass information is decoupled from the instruction, the bypass information is stored separately, and data hazard detection is performed based on the part of the instruction other than the bypass information, thereby reducing the area and power consumption of the circuit and reducing the crossbar complexity at the same time. Only when it is determined that the first instruction meets the instruction sending condition, the bypass information is merged with the first instruction again to obtain the target instruction, and the target instruction is sent.

[0078] See Figure 5A and Figure 5B, first, after receiving the original instruction including bypass information and instruction information, the distribution unit 102 can extract the bypass information from the original instruction and send the bypass information to the bypass information storage unit. Here, the bypass information corresponding to the read instruction and the bypass information corresponding to the write instruction can be stored separately. Among them, the bypass information corresponding to the write instruction can be stored in the first storage unit 105, and the bypass information corresponding to the read instruction can be stored in the second storage unit 106. The bypass information storage unit can return the storage address of the bypass information to the distribution unit 102. For the write instruction, the storage address of the bypass information is the storage address of the bypass information corresponding to the write instruction in the first storage unit 105; for the read instruction, the storage address of the bypass information is the storage address of the bypass information corresponding to the read instruction in the second storage unit 106. The distribution unit 102 can merge the instruction information and the storage address of the bypass information to generate a first instruction, and send the first instruction to the bus control unit 104 through the instruction arbitration unit.

[0079] The bus control unit 104 can generate and send the final target instruction. Among them, for the write instruction and the read instruction, the bus control unit 104 can generate the target instruction in different ways. Specifically, for the write instruction, since bypass information such as the data to be written needs to be sent to the target address along with the instruction in order to write the data to be written to the target address, therefore, the bus control unit 104 can extract the storage address of the first bypass information from the write instruction, obtain the first bypass information from the storage address of the first bypass information, generate the target instruction based on the first bypass information and the write instruction, and send the target instruction. For the read instruction, the data storage unit storing the data to be read does not need to know where the data to be read will be read, so the bus control unit 104 can directly send the read instruction carrying the storage address of the bypass information and the instruction information as the target instruction.

[0080] Furthermore, for the read instruction, the bus control unit 104 can also receive the target data read by the read instruction, and the target data carries the storage address of the bypass information corresponding to the read instruction. The bus control unit 104 can write the target data to the storage address of the bypass information corresponding to the read instruction, so that the data requester (for example, the register) can read the target data from the storage address of the bypass information.

[0081] In some embodiments, the bus control unit 104 is further configured to, when the first instruction is processed, clear the bypass information corresponding to the first instruction from the storage address of the bypass information, so that the storage unit for storing the bypass information can free up storage space to store the bypass information of other instructions. Among them, when the read instruction is processed, the bypass information corresponding to the read instruction can be cleared from the storage address of the bypass information corresponding to the read instruction; when the write instruction is processed, the bypass information corresponding to the write instruction can be cleared from the storage address of the bypass information corresponding to the write instruction.

[0082] In some embodiments, when the first instruction is successfully sent, the information of the first instruction is written into the sent information queue. Refer to Figure 6 , during the instruction sending process, assume that in clock cycle T1, the sent information queue includes the information of instruction 1, the information of instruction 2, and the information of instruction 3; the instruction queue includes instruction 4, instruction 5, and instruction 6. Then, the information of the instruction at the forefront of the instruction queue (i.e., instruction 4) can be extracted, and based on the information of instruction 4 and the information of instruction 1, it is detected whether instruction 4 meets the instruction sending condition. If it meets, instruction 4 is sent. When instruction 4 is successfully sent, in clock cycle T2, the information of instruction 4 is stored in the sent information queue. In clock cycle T3, the information of the processed instruction (assumed to be the information of instruction 1) can also be cleared from the sent information queue. It should be noted that clock cycle T2 can be before clock cycle T3 or after clock cycle T3, and the present disclosure does not limit this. In addition, the time when the previously sent instruction is processed can be earlier or later than the time when the later sent instruction is processed, that is, the order in which the information of each instruction is stored in the sent information queue is not necessarily the same as the order in which the information of each instruction is cleared from the sent information queue.

[0083] In some embodiments, the bus control unit 104 is further configured to empty the information of the processed second instruction from the sent information queue. The sent information queue 1012 can send the cached information and the cache address of the information to the bus control unit 104. When the second instruction to which the information belongs is processed, the bus control unit 104 can send an enable signal to the sent information queue 1012, and the enable signal carries the cache address of the information of the processed second instruction in the sent information queue 1012. In this way, the sent information queue 1012 can clear the information at the corresponding cache address in response to the enable signal.

[0084] Refer to Figure 7 , which is the overall flowchart of the embodiments of the present disclosure. This process can be implemented through Figure 3The circuit implementation shown below. First, the instruction distribution unit 102 can distribute instructions to the instruction queues 1011 in each processing unit 101 according to the thread group number (S1). The instruction distribution unit 102 can also decouple the instruction information and the bypass information in the instruction, and store the decoupled bypass information in the storage address of the bypass information (S2). This storage address is merged with the instruction information to generate a first instruction. The detection unit 1013 can determine whether there is a data hazard (i.e., whether the first instruction meets the instruction sending condition) based on the information of the first instruction and the information of each second instruction stored in the sent information queue, or determine whether there is a data hazard based on the information statistically calculated by the statistical unit (S3). If there is a data hazard, the first instruction is still cached in the instruction queue 1011, and it is periodically re-determined whether there is a data hazard (S4). If there is no data hazard, the first instruction is taken out from the instruction queue 1011 and sent to the arbitration unit 103 (S5). The processing flows of each processing unit 101 are the same and will not be described one by one here.

[0085] The arbitration unit 103 can send each first instruction to the bus control unit 104 in turn according to the priority of the first instructions sent by each processing unit 101 (S6). The bus control unit 104 can generate a target instruction based on the first instruction received from the arbitration unit 103 and send the target instruction to the corresponding target address (S7).

[0086] In addition, the bus control unit 104 can also clear the information corresponding to the second instruction in the sent information queue 1012 (S8), and clear the bypass information (S9) when receiving the processing completion information for a certain second instruction. The detection unit can also write the first instruction information into the sent information queue 1012 when the first instruction is successfully sent through the bus control unit 104 (S10).

[0087] The execution order of the above steps is not limited to the order shown in the figure. For example, the order of steps S8 and S9 can be interchanged, and the order of step S10 and step S8 or S9 can be interchanged, etc.

[0088] The embodiments of the present disclosure perform data hazard processing in multiple thread groups in parallel. When data hazards occur in some thread groups, other thread groups without data hazards can still send instructions. In addition, the present disclosure allows out-of-order issue of consecutive instructions of the same read / write type. Out-of-order issue can effectively improve the system memory access performance. The method of the present disclosure can be used to achieve efficient data hazard processing in multiple thread groups, improve the memory access performance of the system, and utilize the scalability and variations of the method of the present disclosure to reduce power consumption and reduce the crossbar complexity.

[0089] As Figure 8As shown, the present disclosure further provides an artificial intelligence chip, including: a data processing circuit 801; and a control unit 802 configured to send the first instruction to an instruction queue in the data processing circuit 801.

[0090] The data processing circuit 801 may adopt the data processing circuit described in any embodiment of the present disclosure. For the specific details of the data processing circuit 801 in this embodiment, please refer to the foregoing embodiments and will not be elaborated herein.

[0091] See Figure 9 , an embodiment of the present disclosure further provides a data processing method, which is applied to a detection unit in the data processing circuit described in any embodiment of the present disclosure. The method includes:

[0092] Step 901: Detect whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction;

[0093] Step 902: When the instruction sending condition is met, take out the first instruction from the instruction queue and send it;

[0094] The information includes at least one of read / write type information and address information.

[0095] Optionally, the detecting whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction includes: when the sent information queue is not full, detecting whether the first instruction meets the instruction sending condition based on the read / write type information of the first instruction, the read / write type information of the second instruction, the address information of the first instruction, and the address information of the second instruction; when the sent information queue is full, detecting whether the first instruction meets the instruction sending condition based on the address information of the first instruction and the address information of the second instruction.

[0096] Optionally, when the sent information queue is not full, detecting whether the first instruction meets the instruction sending condition based on the read / write type information of the first instruction, the read / write type information of the second instruction, the address information of the first instruction, and the address information of the second instruction includes: when the read / write type information of any second instruction in the sent information queue is different from the read / write type information of the first instruction and the address information is the same as the address information of the first instruction, determining that the first instruction does not meet the instruction sending condition; when the address information of each second instruction in the sent information queue is different from the address information of the first instruction, or when the target address information and the target read / write type information of the same second instruction in the sent information queue do not simultaneously meet the condition that the target address information is the same as the address information of the first instruction and the target read / write type information is different from the read / write type information of the first instruction, determining that the first instruction meets the instruction sending condition.

[0097] Optionally, when the sent information queue is full, detecting whether the first instruction meets the instruction sending condition based on the address information of the first instruction and the address information of the second instruction includes: when there is at least one instruction in the sent information queue whose read / write type information is different from the read / write type information of the first instruction, determining that the first instruction does not meet the instruction sending condition; when the read / write type information of each second instruction in the sent information queue is the same as the read / write type information of the first instruction, determining that the first instruction meets the instruction sending condition.

[0098] Optionally, different processing units are used to process instructions sent by different thread groups.

[0099] Optionally, the circuit further includes a statistical unit for statistically calculating the following information: the total number of the second instructions; the number of instructions that have been processed and completed among each second instruction; and the read / write type information of each second instruction; detecting whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction includes: when all the information in the sent information queue is invalid and there is a second instruction whose read / write type information is different from the read / write type information of the first instruction, detecting whether the first instruction meets the instruction sending condition based on the information statistically calculated by the statistical unit.

[0100] Optionally, detecting whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction includes: when all the information in the sent information queue is invalid and there is a second instruction whose read / write type information is different from the read / write type information of the first instruction, if each second instruction has been processed and completed, determining that the first instruction meets the instruction sending condition.

[0101] Optionally, the information of the first instruction is written into the sent information queue when the sending of the first instruction is successful.

[0102] For details of the above method embodiments, refer to the embodiments of the foregoing data processing circuit, which will not be elaborated here.

[0103] See Figure 10 , embodiments of the present disclosure further provide a data processing device, which is applied to a detection unit in the data processing circuit according to any embodiment of the present disclosure. The device includes:

[0104] A detection module 1001, configured to detect whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction;

[0105] A sending module 1002, configured to, when the instruction sending condition is met, take out the first instruction from the instruction queue for sending;

[0106] The information includes at least one of read / write type information and address information.

[0107] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be elaborated here.

[0108] From the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of this specification can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions of the embodiments of this specification, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments of this specification.

[0109] The systems, devices, modules or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email receiving and sending device, a game console, a tablet computer, a wearable device, or a combination of any several of these devices.

[0110] Each embodiment in this specification is described in a progressive manner. For the identical or similar parts among the embodiments, reference can be made to each other, and the differences between each embodiment and other embodiments are emphasized. In particular, for the apparatus embodiments, since they are basically similar to the method embodiments, they are described relatively simply, and for the relevant parts, reference can be made to the description of the method embodiments. The apparatus embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated. When implementing the solutions of the embodiments of this specification, the functions of each module can be implemented in one or more software and / or hardware. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the solutions of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.

[0111] The above is only the specific implementation manner of the embodiments of this specification. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the embodiments of this specification, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the embodiments of this specification.

Claims

1. A data processing circuit, characterized in that, The circuit includes a plurality of processing units, and each of the plurality of processing units includes: an instruction queue for caching the received first instruction; a sent information queue for caching the information of each of at least one second instruction that has been sent; a detection unit for detecting whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction, and taking out the first instruction from the instruction queue for sending when the instruction sending condition is met; The information includes at least one of read / write type information and address information; The detection unit is used for: when the sent information queue is not full, detecting whether the first instruction meets the instruction sending condition based on the read / write type information of the first instruction, the read / write type information of the second instruction, the address information of the first instruction, and the address information of the second instruction; when the sent information queue is full, detecting whether the first instruction meets the instruction sending condition based on the address information of the first instruction and the address information of the second instruction.

2. The circuit according to claim 1, wherein When the sent information queue is not full, the detection unit is used for: when the read / write type information of any second instruction in the sent information queue is different from the read / write type information of the first instruction and the address information is the same as the address information of the first instruction, determining that the first instruction does not meet the instruction sending condition; when the address information of each second instruction in the sent information queue is different from the address information of the first instruction, or when the target address information and the target read / write type information of the same second instruction in the sent information queue do not simultaneously meet the conditions that the target address information is the same as the address information of the first instruction and the target read / write type information is different from the read / write type information of the first instruction, determining that the first instruction meets the instruction sending condition.

3. The circuit according to claim 1 or 2, characterized in that, When the sent information queue is full, the detection unit is used for: when there is at least one instruction in the sent information queue whose read / write type information is different from the read / write type information of the first instruction, determining that the first instruction does not meet the instruction sending condition; when the read / write type information of each second instruction in the sent information queue is the same as the read / write type information of the first instruction, determining that the first instruction meets the instruction sending condition.

4. The circuit according to claim 1, characterized in that, Different processing units are used to process the instructions sent by different thread groups.

5. The circuit according to claim 4, wherein The circuit further includes a first instruction distribution unit for: receiving the first instructions sent by each thread group, and each first instruction sent by a thread group carries the identification information of the corresponding thread group; respectively distributing the first instructions sent by each thread group to the corresponding processing unit based on the identification information carried in the first instructions sent by each thread group.

6. The circuit according to claim 4 or 5, characterized in that, The instructions sent by different processing units have different priorities; the circuit further includes: An instruction arbitration unit for receiving first instructions sent by each processing unit and sequentially sending the first instructions sent by each processing unit based on the priorities of the first instructions sent by each processing unit.

7. The circuit according to claim 1, characterized in that, The first instruction includes the storage address of bypass information, and the bypass information corresponding to the first instruction is stored under the storage address of the bypass information; the circuit further includes: A second instruction distribution unit for decoupling the original instruction carrying the bypass information to obtain the decoupled original instruction and the bypass information, storing the bypass information under the storage address of the bypass information, generating the first instruction based on the decoupled original instruction and the storage address of the bypass information, and sending the first instruction to the instruction queue; A bus control unit for generating a target instruction based on the first instruction and the storage address of the bypass information and sending the target instruction.

8. The circuit according to claim 7, wherein The bus control unit is further used for: Clearing the information of the processed second instructions from the sent information queue.

9. The circuit according to claim 7 or 8, characterized in that, The first instruction includes a write instruction, and the bypass information includes first bypass information corresponding to the write instruction; The circuit further includes: A first storage unit for storing the first bypass information.

10. The circuit according to claim 9, characterized in that, The bus control unit is used for: Extracting the storage address of the first bypass information from the write instruction, obtaining the first bypass information from the storage address of the first bypass information, generating the target instruction based on the first bypass information and the write instruction, and sending the target instruction.

11. The circuit according to claim 8, wherein The first instruction includes a read instruction, and the bypass information includes second bypass information corresponding to the read instruction; the circuit further includes: A second storage unit for storing the second bypass information.

12. The circuit according to claim 11, wherein The bus control unit is further used for: Using the read instruction as the target instruction.

13. The circuit according to claim 11 or 12, characterized in that, The bus control unit is further used for: Receiving the target data read by the read instruction, where the target data carries the storage address of the second bypass information; Writing the target data into the storage address of the second bypass information in the second storage unit.

14. The circuit according to claim 8, characterized in that, The bus control unit is further used for: Clearing the bypass information corresponding to the first instruction from the storage address of the bypass information when the first instruction is processed.

15. The circuit according to claim 1, wherein When the total number of the second instructions exceeds the length of the sent information queue, setting all the information in the sent information queue to invalid.

16. The circuit according to claim 15, wherein The circuit further includes a statistical unit for statistically calculating the following information: The total number of the second instructions; The number of processed instructions in each second instruction; and The read / write type information of each second instruction; The detection unit is used for: When all the information in the sent information queue is invalid and there is a second instruction with a read / write type information different from that of the first instruction, detecting whether the first instruction meets the instruction sending condition based on the information statistically calculated by the statistical unit.

17. The circuit according to claim 16, characterized in that, When each piece of information in the sent information queue is invalid and there is a second instruction with a read / write type information different from that of the first instruction, if all the second instructions are processed, it is determined that the first instruction meets the instruction sending condition.

18. The circuit according to claim 1, wherein The information of the first instruction is written into the sent information queue when the first instruction is successfully sent.

19. An artificial intelligence chip, characterized in that, Including: The data processing circuit according to any one of claims 1 to 18; And A control unit for sending the first instruction to the instruction queue in the data processing circuit.

20. A data processing method, characterized in that, Applied to the detection unit in the data processing circuit according to any one of claims 1 to 18, the method includes: Detecting whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction; When the instruction sending condition is met, taking out the first instruction from the instruction queue for sending; The information includes at least one of read / write type information and address information.

21. A data processing device, characterized in that, Applied to the detection unit in the data processing circuit according to any one of claims 1 to 18, the device includes: A detection unit for detecting whether the first instruction meets the instruction sending condition based on the information of the first instruction and the information of each second instruction; A sending unit for taking out the first instruction from the instruction queue for sending when the instruction sending condition is met; The information includes at least one of read / write type information and address information.

Images