Data processing circuitry for neural networks, neural network circuitry, and processors

Abstract

The application provides a data processing circuit, a neural network circuit and a processor for a neural network. The data processing circuit comprises M computing units and an output unit. The M computing units are connected with each other in pairs. The M computing units are connected with the output unit respectively. The i-th computing unit is configured to receive a first instruction queue and an operand, determine a first instruction in the first instruction queue as a target instruction, the target instruction matching the i-th computing unit, delete the target instruction to form a second instruction queue, process the operand, and output the second instruction queue and the processed operand to the output unit and the M computing units. The first instruction queue comprises a source instruction queue or the second instruction queue. The output unit is configured to receive the second instruction queue, determine a first instruction in the second instruction queue as an output instruction, and output the processed operand. The technical scheme of the application improves the calculation efficiency of the neural network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of chip heat dissipation, and in particular to a data processing circuit for a neural network, a neural network circuit and a processor. BACKGROUND

[0002] With the continuous progress of science and technology, the types and application fields of deep learning algorithms are also constantly expanding. As the basis for implementing deep learning algorithms, the importance of neural networks is self-evident.

[0003] At present, people have higher and higher requirements for the processing speed of deep learning algorithms. As a result, neural network processors dedicated to neural network computing have emerged. These neural network processors often need to perform a series of continuous calculations on data in the process of implementing deep learning algorithm calculations. In such a continuous calculation process, the data storage unit needs to be read and written multiple times. Such high-frequency memory access reduces the calculation efficiency and increases the total power consumption.

[0004] Therefore, how to improve the calculation efficiency of the neural network is a problem to be solved. SUMMARY

[0005] The purpose of the present application is to provide a data processing circuit for a neural network, a neural network circuit and a processor, thereby improving the calculation efficiency of the neural network.

[0006] In a first aspect, the present application provides a data processing circuit for a neural network. The data processing circuit comprises: M calculation units and an output unit. The M calculation units are connected to each other in pairs. The M calculation units are connected to the output unit respectively. M is an integer greater than 1. The i-th calculation unit in the M calculation units is configured to: receive a first instruction queue and an operand corresponding to the first instruction queue; determine a first instruction in the first instruction queue as a target instruction, wherein the target instruction matches the i-th calculation unit; delete the target instruction from the first instruction queue to form a second instruction queue; process the operand corresponding to the first instruction queue; and output the second instruction queue and the processed operand to the output unit and the M calculation units. The first instruction queue comprises a source instruction queue or a second instruction queue. i is an integer, and the value range of i is 1 to M. The output unit is configured to: receive the second instruction queue; determine a first instruction in the second instruction queue as an output instruction; and output the processed operand.

[0007] According to some possible embodiments, the output end of each of the M calculation units can be connected to the input end of the M calculation units and the input end of the output unit.

[0008] According to some possible implementation, each instruction in the first instruction queue and the second instruction queue can have K bits. K is a positive integer, and K satisfies 2 K ≥ M + 1.

[0009] According to some possible implementation, the i-th computing unit can include a multiplexing unit and a basic operation unit. The multiplexing unit is connected with the basic operation unit. The multiplexing unit is configured to: receive the first instruction queue and the operand corresponding to the first instruction queue; determine whether the first instruction in the first instruction queue is the target instruction; and, in response to the first instruction in the first instruction queue being the target instruction, output the first instruction queue and the operand corresponding to the first instruction queue to the basic operation unit. The basic operation unit is configured to: delete the target instruction from the first instruction queue to form the second instruction queue; process the operand corresponding to the first instruction queue; and output the second instruction queue and the processed operand to the output unit and the M computing units.

[0010] According to some possible implementation, the i-th computing unit can include at least one of: an ALU unit, an IMAC unit, an FADD unit, an FMUL unit, an I2F unit, an F2I unit, and an RSS unit.

[0011] According to some possible implementation, the M computing units can be used for at least one of: quantization calculation, dequantization calculation.

[0012] According to some possible implementations, in the case that M computing units are used for quantization computation, the M computing units can include an I2F unit, an FMUL unit, an FADD unit, and an F2I unit. The I2F unit is configured to: determine that a first instruction in a first instruction queue from a data storage unit is a fixed-point to floating-point conversion instruction; perform fixed-point to floating-point conversion on an operand from the data storage unit; delete the fixed-point to floating-point conversion instruction in the first instruction queue to form a second instruction queue; input the conversion result as an operand and the second instruction queue as a first instruction queue to the FMUL unit. The FMUL unit is configured to: determine that a first instruction in a first instruction queue from the I2F unit is a floating-point multiplication instruction; perform floating-point multiplication operation on an operand from the I2F unit; delete the floating-point multiplication instruction in the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as a first instruction queue to the FADD unit. The FADD unit is configured to: determine that a first instruction in a first instruction queue from the FMUL unit is a floating-point addition instruction; perform floating-point addition operation on an operand from the FMUL unit; delete the floating-point addition instruction in the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as a first instruction queue to the F2I unit. The F2I unit is configured to: determine that a first instruction in a first instruction queue from the FADD unit is a floating-point to fixed-point conversion instruction; perform floating-point to fixed-point conversion on an operand from the FADD unit; delete the fixed-point to floating-point conversion instruction in the first instruction queue to form a second instruction queue; and input the conversion result as an operand and the second instruction queue to the output unit.

[0013] According to some possible implementations, in the case that M computing units are used for quantization computation, the M computing units can include an IMAC unit, an RSS unit, and an ALU unit. The IMAC unit is configured to determine that a first instruction in a first instruction queue from a data storage unit is a fixed-point multiplication instruction, perform a fixed-point multiplication operation on an operand from the data storage unit, delete the fixed-point multiplication instruction in the first instruction queue to form a second instruction queue, and input the operation result as the operand and the second instruction queue as the first instruction queue to the RSS unit. The RSS unit is configured to determine that a first instruction in a first instruction queue from the IMAC unit is a shift instruction, perform a shift operation on an operand from the IMAC unit, delete the shift instruction in the first instruction queue to form a second instruction queue, and input the shift result as the operand and the second instruction queue as the first instruction queue to the ALU unit. The ALU unit is configured to determine that a first instruction in a first instruction queue from the RSS unit is a fixed-point addition instruction, perform a fixed-point addition operation on an operand from the RSS unit, delete the fixed-point addition instruction in the first instruction queue to form a second instruction queue, and input the operation result as the operand and the second instruction queue to an output unit.

[0014] According to some possible implementations, in the case that M computing units are used for dequantization computation, the M computing units can include an ALU unit, an I2F unit, and an FMUL unit. The ALU unit is configured to determine that a first instruction in a first instruction queue from a data storage unit is a fixed-point subtraction instruction, perform a fixed-point subtraction operation on an operand from the data storage unit, delete the fixed-point subtraction instruction in the first instruction queue to form a second instruction queue, and input the operation result as the operand and the second instruction queue as the first instruction queue to the I2F unit. The I2F unit is configured to determine that a first instruction in a first instruction queue from the ALU unit is a fixed-point-to-floating-point conversion instruction, perform a fixed-point-to-floating-point conversion on an operand from the ALU unit, delete the fixed-point-to-floating-point conversion instruction in the first instruction queue to form a second instruction queue, and input the conversion result as the operand and the second instruction queue as the first instruction queue to the FMUL unit. The FMUL unit is configured to determine that a first instruction in a first instruction queue from the I2F unit is a floating-point multiplication instruction, perform a floating-point multiplication operation on an operand from the I2F unit, delete the floating-point multiplication instruction in the first instruction queue to form a second instruction queue, and input the operation result as the operand and the second instruction queue to an output unit.

[0015] According to some possible implementation, in case that M computing units are used for inverse quantization calculation, the M computing units can include an ALU unit, an IMAC unit and an RSS unit. The ALU unit is configured to: determine that a first instruction in a first instruction queue from the data storage unit is a fixed-point subtraction instruction; perform a fixed-point subtraction operation on the operands from the data storage unit; delete the fixed-point subtraction instruction in the first instruction queue to form a second instruction queue; and input the operation result as the operands and the second instruction queue as the first instruction queue into the IMAC unit. The IMAC unit is configured to: determine that a first instruction in a first instruction queue from the ALU unit is a fixed-point multiplication instruction; perform a fixed-point multiplication operation on the operands from the ALU unit; delete the fixed-point multiplication instruction in the first instruction queue to form a second instruction queue; and input the operation result as the operands and the second instruction queue as the first instruction queue into the RSS unit. The RSS unit is configured to: determine that a first instruction in a first instruction queue from the IMAC unit is a shift instruction; perform a shift operation on the operands from the IMAC unit; delete the shift instruction in the first instruction queue to form a second instruction queue; and input the shift result as the operands and the second instruction queue into the output unit.

[0016] In a second aspect, the present application provides a neural network circuit. The neural network circuit includes a data processing circuit and a data storage unit. The data processing circuit is as described in the first aspect and any implementation thereof. The data storage unit is connected with the data processing circuit and is configured to store operands.

[0017] In a third aspect, the present application provides a neural network processor. The neural network chip includes at least one neural network circuit and a control circuit. The neural network circuit is as described in the second aspect. The control circuit is connected with the neural network circuit and is configured to control the neural network circuit.

[0018] In the data processing circuit according to this application, after receiving the first instruction queue and the operands corresponding to the first instruction queue, each computing unit processes the operands if the first instruction in the first instruction queue matches its own, deletes the first instruction, and then outputs the processed operands and the new first instruction queue to the input terminals of all computing units and the output unit; this continues until only output instructions remain in the first instruction queue, at which point the processed operands are output through the output unit. Thus, when the first instruction queue contains multiple instructions before the output instructions, each computing unit matching these instructions can process the operands sequentially according to the order of the instructions, and output the processed operands through the output unit after all processing is complete. In this way, each computing unit processes the operands sequentially in a pipelined manner according to the order of multiple instructions in the first instruction queue, without needing to store and retrieve intermediate results obtained by each computing unit, thereby reducing the number of memory accesses, lowering the memory access frequency, improving computational efficiency, and reducing overall power consumption.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 A schematic diagram of the architecture of a neural network processor in the related technology is shown.

[0022] Figure 2 A schematic structural diagram of a data processing circuit for a neural network according to an embodiment of this application is shown.

[0023] Figure 3 A schematic structural diagram of the source instruction queue according to an embodiment of this application is shown.

[0024] Figure 4 A schematic structural diagram of the computing unit in the data processing circuit of this application embodiment is shown.

[0025] Figure 5 A schematic structural diagram of a multiplexing unit in a data processing circuit according to an embodiment of this application is shown.

[0026] Figure 6 A schematic diagram of an exemplary structure of a data processing circuit for a neural network according to an embodiment of this application is shown.

[0027] Figure 7A schematic diagram of the data processing circuit implementing the first quantization operator according to an embodiment of this application is shown.

[0028] Figure 8 A schematic diagram of the data processing circuit implementing the second quantization operator according to an embodiment of this application is shown.

[0029] Figure 9 A schematic diagram of the data processing circuit implementing the first inverse quantization operator according to an embodiment of this application is shown.

[0030] Figure 10 A schematic diagram of the data processing circuit implementing the second inverse quantization operator according to an embodiment of this application is shown. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of subject matter consistent with some aspects of this application as detailed in the appended claims. Furthermore, the proportional relationships between the various components in the drawings are merely illustrative and not actual proportional relationships.

[0032] With the continuous advancement of science and technology, the types and application areas of deep learning algorithms are constantly expanding. As the foundation for implementing deep learning algorithms, the importance of neural networks is self-evident.

[0033] Currently, people have increasingly higher requirements for the processing speed of deep learning algorithms. As a result, neural network processors specifically designed for neural network computation have emerged.

[0034] Figure 1 A schematic diagram of the architecture of a neural network processor in related technologies is shown. For example... Figure 1 As shown, the neural network processor 1 includes at least a control unit 10 and at least one processing unit 20.

[0035] The control unit 10 is used to control the processing unit 20 to perform data processing. Specifically, the control unit 10 can send instructions to the processing unit 20 to control the processing unit 20.

[0036] The processing unit 20 is used to store and process data under the control of the control unit 10. The number of processing units 20 can be one or more. In the field of neural networks, the data to be processed is usually a tensor. A tensor is a data unit in a neural network and has four dimensions: N (batch), C (channel), H (height), and W (width). Therefore, tensors can be stored in the NCHW format. When the neural network processor 1 processes a tensor, the tensor is stored and processed in different processing units 20 according to channels. That is, each processing unit 20 in the neural network processor 1 can store and process the data in one or more channels of the tensor.

[0037] Specifically, the processing unit 20 includes a data processing unit 100, a data storage unit 200, and a bus 300. The bus 300 is used to transmit data and / or instructions. The data processing unit 100 and the data storage unit 200 are interconnected through the bus 300. In each processing unit 20, the data storage unit 200 is used to store the data in one or more channels of the tensor. The data stored in the data storage unit 200 can be referred to as operands. Under the action of an instruction from the control unit 10, the data processing unit 100 in this processing unit 20 processes the operands.

[0038] As Figure 1 shown, a tensor can be divided into Channel 1, Channel 2,..., Channel N. In this case, the data in Channel 1 is processed by the first processing unit, the data in Channel 2 is processed by the second processing unit, and so on. Of course, in practical applications, there may be a situation where the number of channels in a tensor is greater than the number of processing units. For example, a tensor has N channels and is processed using K (K < N) processing units. In this case, Channels 1 to Channel K of the tensor are respectively processed by K processing units; afterwards, Channel K + 1 is then processed by the first processing unit, and Channel K + 2 is processed by the second processing unit.

[0039] In the process of such a neural network processor implementing the calculation of deep learning algorithms, a series of consecutive calculations on operands are often required. However, in the process of consecutive calculations, the calculation results need to be stored in the data storage unit after each calculation and read from the data storage unit at the start of the next calculation. This means multiple reads and writes to the data storage unit. Such high-frequency memory access reduces the calculation efficiency and increases the total power consumption.

[0040] Therefore, how to improve the calculation efficiency of neural networks is an urgent problem to be solved.

[0041] This application provides a data processing circuit for neural networks. Figure 1 Similar to the data processing unit in the previous example, the data processing circuit in this embodiment functions to store and process operands.

[0042] Figure 2 A schematic structural diagram of a data processing circuit for a neural network according to an embodiment of this application is shown. Figure 2 As shown, the data processing circuit 100' includes M computing units 110 and an output unit 120. The M computing units 110 are connected in pairs. Each of the M computing units 110 is connected to the output unit 120. M is an integer greater than 1. The i-th computing unit 110 among the M computing units 110 is configured to: receive a first instruction queue and operands corresponding to the first instruction queue; determine the first instruction in the first instruction queue as the target instruction, wherein the target instruction matches the i-th computing unit; delete the target instruction from the first instruction queue to form a second instruction queue; process the operands corresponding to the first instruction queue; and output the second instruction queue and the processed operands to the output unit 120 and the M computing units 110. The first instruction queue includes either a source instruction queue or a second instruction queue. i is an integer, and the value of i ranges from 1 to M. The output unit 120 is configured to: receive the second instruction queue; determine the first instruction in the second instruction queue as the output instruction; and output the processed operands.

[0043] In one embodiment, the output of each of the M computing units 110 can be connected to the input of all M computing units 110. That is, the second instruction queue and operands output by each computing unit 110 can reach the input of all M computing units 110. Thus, when the second instruction queue and operands output from one computing unit 110 reach the input of another computing unit 110, they can be received by that other computing unit 110 as the first instruction queue and the operands corresponding to the first instruction queue, respectively. It is understood that because any two computing units 110 among the M computing units 110 have the above connection relationship, the instruction queue and operands can be sent and received between any two computing units 110.

[0044] In one embodiment, the output of each of the M computing units 110 can be connected to the input of the output unit 120. That is, the second instruction queue and operands output by each computing unit 110 can reach the input of the output unit 120. Therefore, the second instruction queue and operands output from each computing unit 110 can all reach the input of the output unit 120.

[0045] Before processing the operands, the data processing circuit 100' can receive a source instruction queue from an external source and operands corresponding to the source instruction queue (also called source operands). In practical applications, the source instruction queue and operands can have the same or different sources. In one embodiment, the source instruction queue can come from a control circuit, and the operands can come from a data storage unit. In another embodiment, the source instruction queue can come from an instruction storage unit, and the operands can come from a data storage unit. In this case, the instruction storage unit and the data storage unit can belong to the same storage circuit or different storage circuits. To receive the source instruction queue from an external source, the data processing circuit 100' can have an input terminal IN. On one hand, the input terminal IN can be connected to at least one of the control circuit, data storage unit, and instruction storage unit located outside the data processing circuit 100' to input the source instruction queue and operands. On the other hand, the input terminal IN can be connected to M calculation units 110 and an output unit 120 to input the source instruction queue and operands from the external source to the M calculation units 110 and the output unit 120.

[0046] Figure 3 A schematic structural diagram of the source instruction queue according to an embodiment of this application is shown. Figure 3 As shown, in one embodiment, the source instruction queue may include multiple instructions. These instructions are arranged and stored sequentially. These instructions include at least one operation instruction CMD and one output instruction RES. Each operation instruction CMD is used to perform corresponding processing on the operands. The output instruction RES is used to output the final operands from the data processing circuit 100'. Specifically, at least one operation instruction CMD may include a first operation instruction CMD1, a second operation instruction CMD2, a third operation instruction CMD3, ..., a t-th operation instruction CMDt. Here, t may be a positive integer.

[0047] It is understood that an instruction can consist of one or more fields. In one embodiment, an instruction may include one field. This field contains an encoding corresponding to the operation type of the instruction. In one embodiment, an instruction may include multiple fields. These multiple fields include at least an information field and an encoding field. The type field is used to represent relevant information about the instruction. The encoding field contains an encoding corresponding to the operation type of the instruction. It should be noted that the instructions in the embodiments of this application may also include other content, and the embodiments of this application do not specifically limit this.

[0048] To achieve interconnection between the M computing units 110 and the output unit 120, an interconnection region 130 may be provided within the data processing circuit 100'. Multiple connection lines are arranged within the interconnection region 130. These connection lines can be used to establish pairwise connections between the M computing units 110, and connections between each of the M computing units 110 and the output unit 120. Furthermore, the connection lines within the interconnection region 130 can also be used to establish connections from the input terminal IN to the M computing units 110 and the output unit 120. It is understood that the connection lines used to achieve the above connections may not be located within a specific region, and this embodiment does not impose specific limitations on this.

[0049] For each of the M computing units 110, its input terminal is connected to the M computing units 110 (including itself) and the input terminal IN. Specifically, the input terminals of the computing unit 110 may include M+1 input channels. Each input channel is connected to one of the M computing units 110 and the input terminal IN.

[0050] In one embodiment, each instruction in the first instruction queue and the second instruction queue may have K bits. For each computing unit 110, both the received first instruction queue and the output second instruction queue include at least one instruction. Each instruction is matched with one of the M computing units 110 or the output unit 120. In other words, each instruction is used to identify the corresponding computing unit 110 or the output unit 120. Since the data processing circuit 100' has M computing units 110 and one output unit 120, each instruction is encoded using at least K bits. K is a positive integer, and K satisfies 2^(K-1) / 2^(K-1). K ≥M+1.

[0051] Here, the function of the i-th computing unit 110 among the M computing units 110 is described in detail. It can be understood that i is any integer ranging from 1 to M. That is to say, the following description applies to any computing unit 110 among the M computing units 110.

[0052] The input terminal of the i-th computation unit 110 is connected to the input terminal IN and all M computation units 110. The input terminal of the i-th computation unit 110 may have a source instruction queue from IN and / or a second instruction queue from at least one of the M computation units 110, as well as operands corresponding to the source instruction queue and / or the second instruction queue. The i-th computation unit 110 receives a first instruction queue. The first instruction queue is either the source instruction queue or the second instruction queue, and the first instruction in the first instruction queue matches the i-th computation unit 110. In this case, the first instruction in the first instruction queue that matches the i-th computation unit 110 is called the target instruction. After determining the target instruction, the i-th computation unit 110 removes the target instruction from the first instruction queue. The first instruction queue after removing the target instruction is used as the second instruction queue. Simultaneously, the i-th computation unit receives the operands corresponding to the first instruction queue and processes the operands. Finally, the i-th computation unit 110 outputs the second instruction queue and the processed operands. Due to the connection between the M computing units 110 and the output unit 120, the second instruction queue and the processed operands will be output to the M computing units 110 and the output unit 120.

[0053] Figure 4 A schematic structural diagram of the computing unit in the data processing circuit of an embodiment of this application is shown. Figure 4 In one embodiment, the computing unit 110 may include a multiplexing unit (MUX) and a basic arithmetic unit (OPN).

[0054] The multiplexing unit MUX is configured to: receive a first instruction queue and operands corresponding to the first instruction queue; determine whether the first instruction in the first instruction queue is a target instruction; and, in response to the first instruction in the first instruction queue being a target instruction, output the first instruction queue and operands corresponding to the first instruction queue to the basic arithmetic unit OPN.

[0055] As can be seen, the multiplexing unit (MUX) needs to determine whether the first instruction in the first instruction queue is the target instruction. In other words, the multiplexing unit (MUX) needs to determine the input channel containing the first instruction queue with the target instruction from M+1 input channels.

[0056] Figure 5 A schematic structural diagram of a multiplexing unit in a data processing circuit according to an embodiment of this application is shown. Figure 5As shown, a multiplexing unit (MUX) may include M+1 input channels and one output. Each of the M+1 input channels can be used to input an instruction queue and its corresponding operand. For example, the first input channel is used to input instruction queue Q_1 and operand OPD_1, the second input channel is used to input instruction queue Q_2 and operand OPD_2, ..., the Mth input channel is used to input instruction queue Q_m and operand OPD_m, and the M+1th input channel is used to input instruction queue Q_m+1 and operand OPD_m+1. The output instruction queue Q and operand OPD are one of the instruction queues Q1, Q2, ..., Q_m, Q_m+1 in each input channel, along with its corresponding operand.

[0057] Understandably, it is necessary to ensure that at any given time, among the M+1 input channels, only one input channel contains the first instruction queue, where the first instruction is the target instruction. In this way, the multiplexing unit (MUX) can output the first instruction queue containing the target instruction and its corresponding operands through the output terminal, based on the target instruction.

[0058] The basic arithmetic unit OPN is configured to: delete the target instruction from the first instruction queue to form a second instruction queue; process the operands corresponding to the first instruction queue; and output the second instruction queue and the processed operands to the output unit 120 and the M calculation units 110.

[0059] The M computational units 110 in the data processing circuit 100' can be computational units with different functions. In one embodiment, the i-th computational unit 110 may include at least one of the following: an ALU (Arithmetic Logic Unit) unit, an IMAC (Fixed-Point Multiply-Accumulate) unit, a FADD (Floating-Point Addition) unit, an FMUL (Floating-Point Multiplication) unit, an I2F (Fixed-Point to Floating-Point Conversion) unit, an F2I (Floating-Point to Fixed-Point Conversion) unit, and an RSS (Fixed-Point Shift) unit. Specifically, the basic arithmetic unit OPN in the i-th computational unit 110 may be one of the following: an ALU arithmetic unit, an IMAC arithmetic unit, a FADD arithmetic unit, an FMUL arithmetic unit, an I2F arithmetic unit, an F2I arithmetic unit, and an RSS arithmetic unit. The ALU arithmetic unit is used to perform fixed-point operand addition / subtraction operations, fixed-point / floating-point operand comparison operations, etc. The IMAC arithmetic unit is used to perform fixed-point operand multiplication operations, multiply-accumulate operations of three operands, etc. The FADD arithmetic unit is used to perform floating-point operand addition operations, etc. The FMUL arithmetic unit is used for operations such as floating-point multiplication. The I2F arithmetic unit is used for fixed-point to floating-point conversion. The F2I arithmetic unit is used for floating-point to fixed-point conversion. The RSS arithmetic unit is used for shifting fixed-point operands. It should be noted that the calculation unit 110 in the data processing circuit 100' may also include other basic arithmetic units, or the data processing circuit 100' may not have one or more of the above-mentioned calculation units 110. This application embodiment does not specifically limit this.

[0060] It should be noted that in the data processing circuit 100' of this application embodiment, the operands may include data from one or more tensors. For example, an operand may include data from one tensor. In this case, an I2F operation unit, an F2I operation unit, an RSS operation unit, etc., can be used to process the operand. As another example, the operands may include data from two tensors. In other words, the number of operands is two. In this case, an ALU operation unit, an IMAC operation unit, a FADD operation unit, an FMUL operation unit, an RSS operation unit, etc., can be used to perform operations between the two operands, or an I2F operation unit, an F2I operation unit, etc., can be used to process one of the two operands. Furthermore, the operands may include data from three or more tensors.

[0061] Further, in one embodiment, the output unit 120 in the data processing circuit 100' may include a multiplexing unit MUX and an output circuit OUT. The multiplexing unit MUX may include M input channels and one output terminal. In the M input channels, each input channel can be used to input a second instruction queue and the operands corresponding to that instruction queue. It is understood that it is necessary to ensure that at any given time, in the M input channels, only one input channel has a second instruction queue where the first instruction is an output instruction. Thus, the multiplexing unit MUX can output the second instruction queue containing the output instruction and the operands corresponding to that second instruction queue through the output terminal, based on the output instruction. It is understood that since the output instruction is usually the last instruction in the source instruction queue, the second instruction queue determined by the multiplexing unit MUX may only have one instruction, i.e., the output instruction. Of course, it is understood that the multiplexing unit MUX may also include M+1 input channels. In these M+1 input channels, the M input channels are respectively connected to the M computing units 110, and one input channel is connected to the input terminal IN of the data processing circuit 100'. In M+1 input channels, only the first instruction in the source instruction queue or the second instruction queue of one input channel is an output instruction.

[0062] The output circuit OUT is used to output the operands upon receiving a second instruction queue and operands from the multiplexing unit MUX. In one embodiment, the output circuit OUT can output the operands to a data storage unit.

[0063] Figure 6 A schematic diagram of an exemplary structure of a data processing circuit for a neural network according to an embodiment of this application is shown. The data processing circuit 100' will be described exemplarily using this exemplary structure as an example.

[0064] like Figure 6 As shown, the data processing circuit 100' includes an input terminal IN, seven calculation units 110, and an output unit 120.

[0065] The input terminal IN is configured to input source operation instructions and operands. Input terminal IN is connected to the input terminals of all multiplexer units (MUX) in the data processing circuit 100'. Therefore, source operation instructions and operands can reach the input terminals of all multiplexer units (MUX).

[0066] Each computing unit 110 includes a multiplexing unit (MUX) and a basic arithmetic unit (OPN). The seven basic arithmetic units (OPNs) in the computing units 110 are: ALU, IMAC, FADD, FMUL, I2F, F2I, and RSS. The multiplexing unit (MUX) has eight input channels. These eight input channels are connected to the input terminal IN and the output terminals of the seven computing units 110, respectively. The output terminal of the multiplexing unit (MUX) is connected to the corresponding basic arithmetic unit (OPN). The multiplexing unit (MUX) is configured to pass the instruction queue and operands corresponding to the computing unit 110 (i.e., the basic arithmetic unit (OPN)) at its input terminal to the corresponding basic arithmetic unit (OPN). The basic arithmetic unit (OPN) is configured to delete the first instruction in the instruction queue, process the operands, and then output the second instruction queue and the processed operands.

[0067] Output unit 120 includes a multiplexer unit MUX and an output circuit OUT. The multiplexer unit MUX has eight input channels. These eight input channels are connected to the input terminal IN and the output terminals of the seven computing units 110, respectively. The output terminal of the multiplexer unit MUX is connected to the output circuit OUT. The multiplexer unit MUX is configured to pass the instruction queue and operands corresponding to the output unit 120 at the input terminal to the output circuit OUT.

[0068] Here, combined with Figure 2 The data processing circuit shown and Figure 3 The source instruction queue shown illustrates the workflow of the data processing circuit 100'. The data processing circuit 100' receives the source instruction queue and operands. The source instruction queue may sequentially include multiple instructions. These instructions include operation instructions CMD1, CMD2, ..., CMDt, as well as output instructions RES. In this case, the workflow of the data processing circuit includes the following steps.

[0069] First, a calculation unit 110 in the data processing circuit 100' determines the source instruction queue as the first instruction queue. That is, the first instruction CMD1 in the source instruction queue is the target instruction that matches the calculation unit 110. Next, the calculation unit 110 processes the operands corresponding to the source instruction queue to obtain the processed operands. Then, the calculation unit 110 deletes the first instruction CMD1 from the source instruction queue to form a second instruction queue. Finally, the calculation unit 110 outputs the formed second instruction queue and the processed operands.

[0070] Next, another computing unit 110 in the data processing circuit 100' determines the second instruction queue obtained in the first step as the first instruction queue. That is, the first instruction CMD2 in the first instruction queue is the target instruction that matches this computing unit 110. Then, this computing unit 110 processes the operands corresponding to the first instruction queue (i.e., the operands obtained in the first step) to obtain processed operands. Then, this computing unit 110 deletes the first instruction CMD2 from the first instruction queue to form the second instruction queue. This computing unit 110 finally outputs the formed second instruction queue and the processed operands.

[0071] Subsequently, following this pattern, the calculation unit 110 in the data processing circuit 100' processes the operands sequentially according to the instructions CMD3 to CMDt in the obtained second instruction queue. It should be noted that in some cases, the multiple operation instructions CMD1, CMD2, ..., CMDt in the source instruction queue may be different from each other. Correspondingly, different calculation units 110 in the data processing circuit 100' process the operands sequentially according to these operation instructions CMD1, CMD2, ..., CMDt. However, in other cases, the multiple operation instructions CMD1, CMD2, ..., CMDt in the source instruction queue may have two or more identical operation instructions. In this case, under the control of the same operation instruction, the same calculation unit 110 in the data processing circuit 100' can process the operands multiple times. This embodiment of the application does not specifically limit this aspect.

[0072] Finally, the output unit 120 in the data processing circuit 100' determines the first instruction in the second instruction queue, which has been processed by the calculation unit 110 corresponding to CMDt, as the output instruction. Subsequently, the output unit 120 outputs the operand corresponding to the second instruction queue (i.e., the operand processed by the calculation unit 110 corresponding to CMDt).

[0073] Thus, under the influence of the source instruction queue, the data processing circuit 100' has completed the processing of the operands.

[0074] In the data processing circuit according to this application, after receiving the first instruction queue and the operands corresponding to the first instruction queue, each computing unit processes the operands if the first instruction in the first instruction queue matches its own, deletes the first instruction, and then outputs the processed operands and the new first instruction queue to the input terminals of all computing units and the output unit; this continues until only output instructions remain in the first instruction queue, at which point the processed operands are output through the output unit. Thus, when the first instruction queue contains multiple instructions before the output instructions, each computing unit matching these instructions can process the operands sequentially according to the order of the instructions, and output the processed operands through the output unit after all processing is complete. In this way, each computing unit processes the operands sequentially in a pipelined manner according to the order of multiple instructions in the first instruction queue, without needing to store and retrieve intermediate results obtained by each computing unit, thereby reducing the number of memory accesses, lowering the memory access frequency, improving computational efficiency, and reducing overall power consumption.

[0075] In practical applications, the data processing circuit in the embodiments of this application can perform various processing on operands.

[0076] In one embodiment, the data processing circuit can be configured to perform quantization and / or dequantization calculations on operands. Quantization refers to the mapping process from the original numerical range to a preset numerical range. Conversely, dequantization refers to the mapping process from the preset numerical range back to the original numerical range. The original numerical range refers to the original numerical range of a number. The original numerical range can depend on the type and number of bits of the number. For example, different types such as floating-point numbers, signed integers, and unsigned integers typically have different original numerical ranges. Similarly, the original numerical range of the number is different when the number of bits is 32-bit, 16-bit, or 8-bit. The preset numerical range refers to the numerical range of the number after processing. The preset numerical range can be determined according to actual needs. Similarly, the preset numerical range can also depend on the type and number of bits. For example, different types such as floating-point numbers, signed integers, and unsigned integers typically have different preset numerical ranges. Similarly, the preset numerical range of the number is different when the number of bits is 32-bit, 16-bit, or 8-bit. It is understandable that the original numerical range and the preset numerical range are usually different.

[0077] It should be noted that during the quantization / dequantization process, the operands of the input data processing circuit can include a first operand and a second operand. The first operand is the operand to be quantized / dequantized. The second operand is used to assist in the quantization / dequantization of the first operand. It is understood that, in this technical field, the tensor to which the first operand belongs can be stored channel-wise. Therefore, the second operand can remain unchanged for all elements of the tensor within the same channel.

[0078] In this embodiment of the application, two quantization operators are defined for quantization calculation: a first quantization operator and a second quantization operator. Furthermore, in this embodiment of the application, two quantization operators are also defined for inverse quantization calculation: a first inverse quantization operator and a second inverse quantization operator.

[0079] Figure 7 A schematic diagram of a data processing circuit implementing a first quantization operator according to an embodiment of this application is shown. Figure 7 As shown, the first quantization operator is implemented based on I2F units, FMUL units, FADD units, and F2I units. Accordingly, the data processing circuit includes at least: I2F units, FMUL units, FADD units, F2I units, and an output unit.

[0080] The I2F unit is configured to: determine that the first instruction in the first instruction queue from the data storage unit is a fixed-point to floating-point conversion instruction; perform fixed-point to floating-point conversion on the operands from the data storage unit; delete the fixed-point to floating-point conversion instruction from the first instruction queue to form a second instruction queue; and input the conversion result as an operand and the second instruction queue as the first instruction queue into the FMUL unit.

[0081] Here, the first instruction queue from the data storage unit is the source instruction queue. The multiple operation instructions in the source instruction queue are, in order: fixed-point to floating-point conversion instructions, floating-point multiplication instructions, floating-point addition instructions, and floating-point to fixed-point conversion instructions. The source instruction queue and the corresponding operand OPD are input through the input terminals of the data processing circuit. The operand OPD is fixed-point data. For example, the operand OPD can be a 32-bit, 16-bit, or 8-bit fixed-point number.

[0082] Specifically, the I2F unit determines that the first instruction in the first instruction queue (i.e., the source instruction queue) is a fixed-point to floating-point conversion instruction. Because the fixed-point to floating-point conversion instruction matches the I2F unit, the I2F unit performs a fixed-point to floating-point conversion on the operand OPD corresponding to the first instruction queue. After the fixed-point to floating-point conversion, the operand OPD becomes a floating-point number. For example, after the fixed-point to floating-point conversion, the operand OPD can be a 32-bit floating-point number. Furthermore, the I2F unit removes the fixed-point to floating-point conversion instruction from the first instruction queue to form the second instruction queue.

[0083] The FMUL unit is configured to: determine that the first instruction in the first instruction queue from the I2F unit is a floating-point multiplication instruction; perform a floating-point multiplication operation on the operands from the I2F unit; delete the floating-point multiplication instruction from the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as the first instruction queue into the FADD unit.

[0084] Specifically, the FMUL unit determines that the first instruction in the first instruction queue from the I2F unit (i.e., the second instruction queue formed by the I2F unit) is a floating-point multiplication instruction. Since the floating-point multiplication instruction corresponds to the FMUL unit, the FMUL unit performs a multiplication operation on the operand OPD from the I2F unit and another operand OPD1 to obtain the processed operand OPD. Because operand OPD is a floating-point number, and the other operand OPD1 is also a floating-point number, the processed operand OPD remains a floating-point number. Furthermore, the FMUL unit removes the floating-point multiplication instruction from the first instruction queue to form the second instruction queue.

[0085] The FADD unit is configured to: determine that the first instruction in the first instruction queue from the FMUL unit is a floating-point addition instruction; perform a floating-point addition operation on the operand from the FMUL unit; delete the floating-point addition instruction from the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as the first instruction queue into the F2I unit.

[0086] Specifically, the FADD unit determines that the first instruction in the first instruction queue from the FMUL unit (i.e., the second instruction queue formed by the FMUL unit) is a floating-point addition instruction. Since the floating-point addition instruction corresponds to the FADD unit, the FADD unit performs addition on the operand OPD and another operand OPD2 from the FMUL unit to obtain the processed operand OPD. Because operand OPD is a floating-point number, and another operand OPD2 is also a floating-point number, the processed operand OPD remains a floating-point number. Furthermore, the FADD unit removes floating-point multiplication and addition instructions from the first instruction queue to form the second instruction queue.

[0087] The F2I unit is configured to: determine that the first instruction in the first instruction queue from the FADD unit is a floating-point to fixed-point conversion instruction; perform floating-point to fixed-point conversion on the operand from the FADD unit; delete the fixed-point to floating-point conversion instruction from the first instruction queue to form a second instruction queue; and use the conversion result as an operand and the second instruction queue input / output unit.

[0088] Specifically, the F2I unit determines that the first instruction in the first instruction queue from the FADD unit (i.e., the second instruction queue formed by the FADD unit) is a floating-point to fixed-point conversion instruction. Because the floating-point to fixed-point conversion instruction corresponds to the F2I unit, the F2I unit performs a floating-point to fixed-point conversion on the operand OPD from the FADD unit to obtain the processed operand OPD. The processed operand OPD is a fixed-point number. Furthermore, the F2I unit deletes the floating-point to fixed-point conversion instruction from the first instruction queue to form the second instruction queue.

[0089] It is understandable that the operand OPD output by the F2I unit is the quantization calculation result of the operand OPD from the data storage unit by the first quantization operator. Subsequently, the output unit in the data processing circuit can output the operand OPD from the F2I unit to the data storage unit based on the only remaining output instructions in the second instruction queue.

[0090] As can be seen, in the process of quantizing operands using the first quantization operator implemented based on the data processing circuit of this application embodiment, it is only necessary to read the operands from the data processing unit once initially and write the quantization result to the data storage unit after the quantization calculation is completed. No operand writing and reading are required after each of the I2F unit, FMUL unit, and FADD unit has finished processing. Thus, compared with the data processing circuit in related technologies, the data processing circuit of this application embodiment can reduce three read / write operations to the data storage unit, and the quantization calculation of the first quantization operator is accelerated by 300%.

[0091] Figure 8 A schematic diagram of the data processing circuit implementing the second quantization operator according to an embodiment of this application is shown. Figure 7 As shown, the second quantization operator is implemented based on the IMAC unit, RSS unit, and ALU unit. Accordingly, the data processing circuit includes at least the following: IMAC unit, RSS unit, ALU unit, and output unit.

[0092] The IMAC unit is configured to: determine that the first instruction in the first instruction queue from the data storage unit is a fixed-point multiplication instruction; perform fixed-point multiplication on the operands from the data storage unit; delete the fixed-point multiplication instruction from the first instruction queue to form a second instruction queue; and input the result of the operation as an operand and the second instruction queue as the first instruction queue into the RSS unit.

[0093] Here, the first instruction queue from the data storage unit is the source instruction queue. The multiple operation instructions in the source instruction queue are, in order: fixed-point multiplication instructions, shift instructions, and fixed-point addition instructions. The source instruction queue and the corresponding operands OPD are input through the input terminals of the data processing circuit. The operands OPD are fixed-point data. For example, the operands OPD can be 32-bit, 16-bit, or 8-bit fixed-point numbers.

[0094] Specifically, the IMAC unit determines that the first instruction in the first instruction queue (i.e., the source instruction queue) is a fixed-point multiplication instruction. Because the fixed-point multiplication instruction matches the IMAC unit, the IMAC unit performs a fixed-point multiplication operation on operand OPD and another operand OPD1 to obtain the processed operand OPD. Since operand OPD is a fixed-point number, and the other operand OPD1 is also a fixed-point number, the processed operand OPD is also a fixed-point number. Furthermore, the IMAC unit removes the fixed-point multiplication instruction from the first instruction queue to form the second instruction queue.

[0095] The RSS unit is configured to: determine that the first instruction in the first instruction queue from the IMAC unit is a shift instruction; shift the operand from the IMAC unit; delete the shift instruction from the first instruction queue to form a second instruction queue; and input the shift result as an operand and the second instruction queue as the first instruction queue into the ALU unit.

[0096] Specifically, the RSS unit determines that the first instruction in the first instruction queue from the IMAC unit (i.e., the second instruction queue formed by the IMAC unit) is a shift instruction. Because the shift instruction corresponds to the RSS unit, the RSS unit, under the action of another operand OPD2, shifts the operand OPD from the IMAC unit to obtain the processed operand OPD. The processed operand OPD remains a fixed-point number. Furthermore, the RSS unit deletes the shift instruction from the first instruction queue to form the second instruction queue.

[0097] The ALU unit is configured to: determine that the first instruction in the first instruction queue from the RSS unit is a fixed-point addition instruction; perform fixed-point addition on the operands from the RSS unit; delete the fixed-point addition instruction from the first instruction queue to form a second instruction queue; and use the result of the operation as an operand and the second instruction queue as an input / output unit.

[0098] Specifically, the ALU unit determines that the first instruction in the first instruction queue from the RSS unit (i.e., the second instruction queue formed by the RSS unit) is a fixed-point addition instruction. Because the fixed-point addition instruction corresponds to the ALU unit, the ALU unit performs fixed-point addition on the operand OPD and the second operand OPD3 from the RSS unit to obtain the processed operand OPD. Since operand OPD is a fixed-point number, and the second operand OPD3 is also a fixed-point number, the processed operand OPD is also a fixed-point number. Furthermore, the ALU unit removes the fixed-point addition instruction from the first instruction queue to form the second instruction queue.

[0099] It is understandable that the operand OPD output by the ALU unit is the quantization calculation result of the operand OPD from the data storage unit by the second quantization operator. Afterwards, the output unit in the data processing circuit can output the operand OPD from the ALU unit to the data storage unit based on the only remaining output instructions in the second instruction queue.

[0100] As can be seen, in the process of quantizing operands using the second quantization operator implemented based on the data processing circuit of this application embodiment, it is only necessary to read the operands from the data processing unit once initially and write the quantization result to the data storage unit after the quantization calculation is completed. No operand writing and reading are required after each calculation unit in the IMAC unit and RSS unit has finished processing. Thus, compared with the data processing circuit in related technologies, the data processing circuit of this application embodiment can reduce two read / write operations to the data storage unit, and the implementation of the quantization calculation of the second quantization operator is accelerated by 200%.

[0101] Figure 9 A schematic diagram of the data processing circuit implementing the first inverse quantization operator according to an embodiment of this application is shown. Figure 9 As shown, the first inverse quantization operator is implemented based on an ALU unit, an I2F unit, and an FMUL unit. Accordingly, the data processing circuit includes at least an ALU unit, an I2F unit, an FMUL unit, and an output unit.

[0102] The ALU unit is configured to: determine that the first instruction in the first instruction queue from the data storage unit is a fixed-point subtraction instruction; perform a fixed-point subtraction operation on the operand from the data storage unit; delete the fixed-point subtraction instruction from the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as the first instruction queue into the I2F unit.

[0103] Here, the first instruction queue from the data storage unit is the source instruction queue. The multiple operation instructions in the source instruction queue are, in order: fixed-point subtraction instructions, fixed-point-to-floating-point conversion instructions, and floating-point multiplication instructions. The source instruction queue and the corresponding operand OPD are input through the input terminals of the data processing circuit. The operand OPD is fixed-point data. For example, the operand OPD can be a 32-bit, 16-bit, or 8-bit fixed-point number.

[0104] Specifically, the ALU unit determines that the first instruction in the first instruction queue (i.e., the source instruction queue) is a fixed-point subtraction instruction. Because the fixed-point subtraction instruction matches the ALU unit, the ALU unit performs a fixed-point subtraction operation on the operand OPD corresponding to the first instruction queue and another operand OPD1. Since operand OPD is a fixed-point number, and the other operand OPD1 is also a fixed-point number, the processed operand OPD remains a fixed-point number. Furthermore, the ALU unit removes the fixed-point subtraction instruction from the first instruction queue to form the second instruction queue.

[0105] The I2F unit is configured to: determine that the first instruction in the first instruction queue from the ALU unit is a fixed-point to floating-point conversion instruction; perform fixed-point to floating-point conversion on the operands from the ALU unit; delete the fixed-point to floating-point conversion instruction from the first instruction queue to form a second instruction queue; and input the conversion result as an operand and the second instruction queue as the first instruction queue into the FMUL unit.

[0106] Specifically, the I2F unit determines that the first instruction in the first instruction queue from the ALU unit (i.e., the second instruction queue formed by the ALU unit) is a fixed-point to floating-point conversion instruction. Because the fixed-point to floating-point conversion instruction corresponds to the I2F unit, the I2F unit performs a fixed-point to floating-point conversion on the operand OPD from the ALU unit to obtain the processed operand OPD. The converted operand OPD is, for example, a 32-bit floating-point number. Furthermore, the I2F unit removes the fixed-point to floating-point conversion instruction from the first instruction queue to form the second instruction queue.

[0107] The FMUL unit is configured to: determine that the first instruction in the first instruction queue from the I2F unit is a floating-point multiplication instruction; perform floating-point multiplication on the operands from the I2F unit; delete the floating-point multiplication instruction from the first instruction queue to form a second instruction queue; and use the result of the operation as an operand and the second instruction queue as an input / output unit.

[0108] Specifically, the FMUL unit determines that the first instruction in the first instruction queue from the I2F unit (i.e., the second instruction queue formed by the I2F unit) is a floating-point multiplication instruction. Since the floating-point multiplication instruction corresponds to the FMUL unit, the FMUL unit performs a multiplication operation on the operand OPD from the I2F unit and another operand OPD2 to obtain the processed operand OPD. Because operand OPD is a floating-point number, and another operand OPD2 is also a floating-point number, the processed operand OPD remains a floating-point number. Furthermore, the FMUL unit removes the floating-point multiplication instruction from the first instruction queue to form the second instruction queue.

[0109] It is understandable that the operand OPD output by the FMUL unit is the dequantization result of the first dequantization operator on the operand OPD from the data storage unit. Afterwards, the output unit in the data processing circuit can output the operand OPD from the FMUL unit to the data storage unit based on the only remaining output instructions in the second instruction queue.

[0110] As can be seen, in the process of performing dequantization calculation on operands using the first dequantization operator implemented based on the data processing circuit of this application embodiment, it is only necessary to read the operands from the data processing unit once initially and write the dequantization result to the data storage unit after the quantization calculation is completed. No operand writing and reading are required after each calculation unit in the ALU unit and I2F unit has finished processing. Thus, compared with the data processing circuit in related technologies, the data processing circuit of this application embodiment can reduce two read / write operations on the data storage unit, and the implementation of the dequantization calculation of the first dequantization operator is accelerated by 200%.

[0111] Figure 10 A schematic diagram of the data processing circuit implementing the second inverse quantization operator according to an embodiment of this application is shown. Figure 10 As shown, the second inverse quantization operator is implemented based on an ALU unit, an IMAC unit, and an RSS unit. Accordingly, the data processing circuit includes at least an ALU unit, an IMAC unit, an RSS unit, and an output unit.

[0112] The ALU unit is configured to: determine that the first instruction in the first instruction queue from the data storage unit is a fixed-point subtraction instruction; perform a fixed-point subtraction operation on the operand from the data storage unit; delete the fixed-point subtraction instruction from the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as the first instruction queue into the IMAC unit.

[0113] Here, the first instruction queue from the data storage unit is the source instruction queue. The multiple operation instructions in the source instruction queue are, in order: fixed-point subtraction instructions, fixed-point multiplication instructions, and shift instructions. The source instruction queue and the corresponding operand OPD are input through the input terminals of the data processing circuit. The operand OPD is fixed-point data. For example, the operand OPD can be a 32-bit, 16-bit, or 8-bit fixed-point number.

[0114] Specifically, the ALU unit determines that the first instruction in the first instruction queue (i.e., the source instruction queue) from the data storage unit is a fixed-point subtraction instruction. Because the fixed-point subtraction instruction matches the ALU unit, the ALU unit performs a fixed-point subtraction operation on operand OPD and another operand OPD1 to obtain the processed operand OPD. The processed operand OPD is a fixed-point number. Furthermore, the ALU unit removes the fixed-point subtraction instruction from the first instruction queue to form the second instruction queue.

[0115] The IMAC unit is configured to: determine that the first instruction in the first instruction queue from the ALU unit is a fixed-point multiplication instruction; perform fixed-point multiplication on the operands from the ALU unit; delete the fixed-point multiplication instruction from the first instruction queue to form a second instruction queue; and input the result of the operation as an operand and the second instruction queue as the first instruction queue into the RSS unit.

[0116] Specifically, the IMAC unit determines that the first instruction in the first instruction queue (i.e., the second instruction queue from the IMAC unit) is a fixed-point multiplication instruction. Because the fixed-point multiplication instruction matches the IMAC unit, the IMAC unit performs a fixed-point multiplication operation on operand OPD and another operand OPD2 to obtain the processed operand OPD. Since operand OPD is a fixed-point number, and another operand OPD2 is also a fixed-point number, the processed operand OPD is a fixed-point number. Furthermore, the IMAC unit removes the fixed-point multiplication instruction from the first instruction queue to form the second instruction queue.

[0117] The RSS unit is configured to: determine that the first instruction in the first instruction queue from the IMAC unit is a shift instruction; shift the operand from the IMAC unit; delete the shift instruction from the first instruction queue to form a second instruction queue; and use the shift result as an operand and the second instruction queue as an input / output unit.

[0118] Specifically, the RSS unit determines that the first instruction in the first instruction queue from the IMAC unit (i.e., the second instruction queue formed by the IMAC unit) is a shift instruction. Because the shift instruction corresponds to the RSS unit, the RSS unit, under the action of the operand OPD3, shifts the operand OPD from the IMAC unit to obtain the processed operand OPD. The processed operand OPD remains a fixed-point number. Furthermore, the RSS unit removes the shift instruction from the first instruction queue to form the second instruction queue.

[0119] It is understandable that the operand OPD output by the RSS unit is the dequantization result of the first dequantization operator on the operand OPD from the data storage unit. Afterwards, the output unit in the data processing circuit can output the operand OPD from the RSS unit to the data storage unit based on the only remaining output instructions in the second instruction queue.

[0120] As can be seen, in the process of performing dequantization calculation on operands using the second dequantization operator implemented based on the data processing circuit of this application embodiment, it is only necessary to read the operands from the data processing unit once initially and write the dequantization result to the data storage unit after the quantization calculation is completed. No operand writing and reading are required after each calculation unit in the ALU unit and I2F unit has finished processing. Thus, compared with the data processing circuit in related technologies, the data processing circuit of this application embodiment can reduce two read / write operations on the data storage unit, and the implementation of the dequantization calculation of the second dequantization operator is accelerated by 200%.

[0121] It should be noted that, for the quantization and dequantization calculations in the above examples, the second operands OPD1, OPD2, and OPD3 are a collective term for all operands other than operand OPD. In practical applications, the second operands OPD1, OPD2, and OPD3 can come from data in the same tensor or from data in different tensors. Furthermore, in one embodiment, at least one of the second operands OPD1, OPD2, and OPD3 can vary according to the channel. That is, for operand OPD within the same channel of the tensor, at least one of the corresponding second operands OPD1, OPD2, and OPD3 can remain unchanged. When traversing all the data in one channel of the tensor and switching to another channel of the tensor, at least one of the second operands OPD1, OPD2, and OPD3 can change.

[0122] Understandably, data from different channels in a tensor can be processed by different computational units. Therefore, in the process of a computational unit performing quantization and dequantization calculations on operands within a single channel, the value of the second operand can be fixed.

[0123] Based on the same inventive concept, embodiments of this application provide a neural network circuit. This neural network circuit includes a data processing circuit and a data storage unit. The data processing circuit is the same as that described in this application. The data storage unit is connected to the data processing circuit and configured to store operands.

[0124] In a third aspect, this application provides a neural network processor. The neural network chip includes at least one neural network circuit and a control circuit. The neural network circuit is the neural network circuit as described in the embodiments of this application. The control circuit is connected to the neural network circuit and configured to control the neural network circuit.

[0125] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only.

[0126] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data processing circuit for neural networks, characterized in that, include: M computational units and output units; The M computing units are connected in pairs, and each of the M computing units is connected to the output unit, where M is an integer greater than 1. The i-th computing unit among the M computing units is configured to: receive a first instruction queue and operands corresponding to the first instruction queue; determine the first instruction in the first instruction queue as a target instruction, wherein the target instruction matches the i-th computing unit; delete the target instruction from the first instruction queue to form a second instruction queue; process the operands corresponding to the first instruction queue; and output the second instruction queue and the processed operands to the output unit and the M computing units; wherein the first instruction queue includes either a source instruction queue or the second instruction queue; i is an integer, and the value of i ranges from 1 to M; The output unit is configured to: receive the second instruction queue; determine the first instruction in the second instruction queue as an output instruction; and output the processed operand.

2. The data processing circuit according to claim 1, characterized in that, The output of each of the M computing units is connected to the input of the M computing units and the input of the output unit.

3. The data processing circuit according to claim 1, characterized in that, Each instruction in the first instruction queue and the second instruction queue has K bits, where K is a positive integer and K satisfies 22. K ≥M+1.

4. The data processing circuit according to claim 1, characterized in that, The i-th computing unit includes: a multiplexing unit and a basic arithmetic unit; wherein the multiplexing unit is connected to the basic arithmetic unit; The multiplexing unit is configured to: receive the first instruction queue and the operands corresponding to the first instruction queue; determine whether the first instruction in the first instruction queue is the target instruction; and, in response to the first instruction in the first instruction queue being the target instruction, output the first instruction queue and the operands corresponding to the first instruction queue to the basic arithmetic unit. The basic arithmetic unit is configured to: delete the target instruction from the first instruction queue to form the second instruction queue; process the operands corresponding to the first instruction queue; and output the second instruction queue and the processed operands to the output unit and the M computing units.

5. The data processing circuit according to claim 1, characterized in that, The i-th computation unit includes at least one of the following: an arithmetic logic unit (ALU), a fixed-point multiplication unit (IMAC), a floating-point addition unit (FADD), a floating-point multiplication unit (FMUL), a fixed-point to floating-point conversion unit (I2F), a floating-point to fixed-point conversion unit (F2I), and a shift unit (RSS).

6. The data processing circuit according to claim 1, characterized in that, The M computing units are used for at least one of the following: quantization calculation and dequantization calculation.

7. The data processing circuit according to claim 6, characterized in that, When the M computing units are used for quantization calculations, the M computing units include I2F units, FMUL units, FADD units, and F2I units; The I2F unit is configured to: determine that the first instruction in the first instruction queue from the data storage unit is a fixed-point to floating-point conversion instruction; perform fixed-point to floating-point conversion on the operands from the data storage unit; delete the fixed-point to floating-point conversion instruction from the first instruction queue to form a second instruction queue; and input the conversion result as an operand and the second instruction queue as the first instruction queue into the FMUL unit. The FMUL unit is configured to: determine that the first instruction in the first instruction queue from the I2F unit is a floating-point multiplication instruction; perform a floating-point multiplication operation on the operands from the I2F unit; delete the floating-point multiplication instruction from the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as the first instruction queue into the FADD unit. The FADD unit is configured to: determine that the first instruction in the first instruction queue from the FMUL unit is a floating-point addition instruction; perform floating-point addition on the operands from the FMUL unit; delete the floating-point addition instruction from the first instruction queue to form a second instruction queue; and input the result of the operation as an operand and the second instruction queue as the first instruction queue into the F2I unit. The F2I unit is configured to: determine that the first instruction in the first instruction queue from the FADD unit is a floating-point to fixed-point conversion instruction; perform floating-point to fixed-point conversion on the operands from the FADD unit; delete the fixed-point to floating-point conversion instruction from the first instruction queue to form a second instruction queue; and input the conversion result as an operand and the second instruction queue into the output unit.

8. The data processing circuit according to claim 6, characterized in that, When the M computing units are used for quantization calculations, the M computing units include an IMAC unit, an RSS unit, and an ALU unit; The IMAC unit is configured to: determine that the first instruction in the first instruction queue from the data storage unit is a fixed-point multiplication instruction; perform fixed-point multiplication on the operands from the data storage unit; delete the fixed-point multiplication instruction from the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as the first instruction queue into the RSS unit. The RSS unit is configured to: determine that the first instruction in the first instruction queue from the IMAC unit is a shift instruction; shift the operand from the IMAC unit; delete the shift instruction in the first instruction queue to form a second instruction queue; and input the shift result as an operand and the second instruction queue as the first instruction queue into the ALU unit. The ALU unit is configured to: determine that the first instruction in the first instruction queue from the RSS unit is a fixed-point addition instruction; perform fixed-point addition on the operands from the RSS unit; delete the fixed-point addition instruction from the first instruction queue to form a second instruction queue; and input the result of the operation as an operand and the second instruction queue into the output unit.

9. The data processing circuit according to claim 6, characterized in that, When the M computing units are used for inverse quantization calculations, the M computing units include an ALU unit, an I2F unit, and an FMUL unit; The ALU unit is configured to: determine that the first instruction in the first instruction queue from the data storage unit is a fixed-point subtraction instruction; perform a fixed-point subtraction operation on the operand from the data storage unit; delete the fixed-point subtraction instruction from the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as the first instruction queue into the I2F unit. The I2F unit is configured to: determine that the first instruction in the first instruction queue from the ALU unit is a fixed-point to floating-point conversion instruction; perform fixed-point to floating-point conversion on the operands from the ALU unit; delete the fixed-point to floating-point conversion instruction from the first instruction queue to form a second instruction queue; and input the conversion result as an operand and the second instruction queue as the first instruction queue into the FMUL unit. The FMUL unit is configured to: determine that the first instruction in the first instruction queue from the I2F unit is a floating-point multiplication instruction; perform floating-point multiplication on the operands from the I2F unit; delete the floating-point multiplication instruction from the first instruction queue to form a second instruction queue; and input the result of the operation as an operand and the second instruction queue into the output unit.

10. The data processing circuit according to claim 6, characterized in that, When the M computing units are used for inverse quantization calculation, the M computing units include an ALU unit, an IMAC unit, and an RSS unit; The ALU unit is configured to: determine that the first instruction in the first instruction queue from the data storage unit is a fixed-point subtraction instruction; perform a fixed-point subtraction operation on the operand from the data storage unit; delete the fixed-point subtraction instruction from the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as the first instruction queue into the IMAC unit. The IMAC unit is configured to: determine that the first instruction in the first instruction queue from the ALU unit is a fixed-point multiplication instruction; perform fixed-point multiplication on the operands from the ALU unit; delete the fixed-point multiplication instruction from the first instruction queue to form a second instruction queue; and input the operation result as an operand and the second instruction queue as the first instruction queue into the RSS unit. The RSS unit is configured to: determine that the first instruction in the first instruction queue from the IMAC unit is a shift instruction; shift the operand from the IMAC unit; delete the shift instruction from the first instruction queue to form a second instruction queue; and input the shift result as an operand and the second instruction queue into the output unit.

11. A neural network circuit, characterized in that, include: The data processing circuit as described in any one of claims 1 to 10; as well as A data storage unit is connected to the data processing circuit and configured to store operands.

12. A neural network chip, characterized in that, include: At least one neural network circuit as described in claim 11; as well as A control circuit is connected to the neural network circuit and configured to control the neural network circuit.

Images