Programmable parallel processing devices, neural network chips, and electronic devices

Abstract

This disclosure provides a programmable parallel processing device applied in the field of information processing technology, comprising: a configuration register set for storing configuration information of reusable instructions; an instruction controller for retrieving the configuration information of instructions from the configuration register set and decoding it to obtain control information corresponding to the instructions; and sending the instructions to the instruction execution unit based on the control information; a vector module including a shared register set and multiple dynamically reconfigurable processing units; the vector module for dynamically reconfiguring the multiple processing units into multiple processing unit groups according to the vector width of the vector data to be processed; the shared register set for storing scalar data; a memory access network for transferring data between the data memory and the vector module; and the data memory for exchanging data with external systems through input / output interfaces and storing computational data. This application also discloses a neural network chip and an electronic device.

Description

Technical Field

[0001] This application relates to the field of information processing technology, and in particular to a programmable parallel processing device, a neural network chip, and an electronic device. Background Technology

[0002] To achieve real-time and efficient processing of deep neural network algorithms, programmable parallel processing devices for deep neural network computation have become a research hotspot. Currently, most programmable parallel processing devices for deep neural network computation use matrix computation and vector dot product operations to accelerate deep neural networks, but they still cannot flexibly and efficiently cope with the rich variations in feature map size, convolution window size, and network layer structure in deep neural network computation. Summary of the Invention

[0003] The main objective of this application is to provide a programmable parallel processing device, a neural network chip, and an electronic device that effectively improves the flexibility and computational efficiency of the device through a reconfigurable parallel processing architecture based on a single instruction multi-vector processing method, and improves instruction coding efficiency and global configuration efficiency in the program through a configuration information expression method based on configuration register groups.

[0004] To achieve the above objectives, a first aspect of this application provides a programmable parallel processing apparatus, comprising:

[0005] Input / output interfaces are used for exchanging data with external systems.

[0006] An instruction memory is used to load instructions from an external source via the input / output interface and to store the instructions.

[0007] A configuration register set is used to store configuration information for the reusable instructions;

[0008] The instruction controller is used to look up the configuration information of the instruction from the configuration register group, decode the configuration information to obtain the control information corresponding to the instruction, and send the instruction to the instruction execution unit based on the control information to execute the instruction;

[0009] The instruction execution unit includes a vector module and a memory access network. The vector module includes a shared register set and multiple dynamically reconfigurable processing units. The vector module is used to dynamically reconfigure the multiple processing units into multiple processing unit groups according to the vector width of the vector data to be processed, so as to perform parallel operations of single instruction multiple vectors. The shared register set is used to store scalar data. The memory access network is used to complete data transfer between the data memory and the vector module.

[0010] The data storage device is used to exchange data with the outside world through the input / output interface and to store computational data.

[0011] In one embodiment of this disclosure, the instructions stored in the instruction memory carry configuration register numbers;

[0012] The instruction controller is specifically configured to retrieve the instruction from the instruction memory, locate the corresponding configuration register from the configuration register group according to the configuration register number included in the instruction, read configuration information from the configuration register, decode the configuration information, and obtain the control information corresponding to the instruction.

[0013] The instruction controller is further configured to decode the configuration information directly encoded in the instruction to obtain the control information corresponding to the instruction.

[0014] In one embodiment of this disclosure, the vector module includes 2 N One processing unit and 2 M There are 2 shared register groups, each of which is 2 N-M All of the aforementioned processing units share the same N and M, where N and M are both positive integers greater than 0;

[0015] The vector module is specifically used in the vector width of 2 L In the case of 2 N The processing unit is divided into 2 N-L The processing unit group;

[0016] Where N and M are both positive integers greater than 0, and N≥L≥M.

[0017] In one embodiment of this disclosure, the processing unit includes:

[0018] The local register set includes a basic arithmetic logic unit, P parallel multiply-accumulate units, and a complex function unit. The P parallel multiply-accumulate units share the input vector data and scalar data, where P is a positive integer.

[0019] The local register group is used to store vector data;

[0020] The basic arithmetic logic unit is used to perform at least the following operations: multiplication, addition, subtraction, comparison, bitwise AND, bitwise OR, bitwise NOT, left shift, and right shift.

[0021] The multiply-accumulate unit is used to perform multiply-accumulate operations;

[0022] The complex function unit is used to fit nonlinear function operations through approximate calculation.

[0023] A single processing unit contains multiple multiply-accumulate operation units, which can further improve the computing power and data sharing rate for neural networks;

[0024] In one embodiment of this disclosure, the processing unit is further configured to access vector data stored in a local register group by the target processing unit, wherein the target processing unit and the processing unit belong to the same processing unit group.

[0025] In one embodiment of this disclosure, the memory access network is further configured to read a plurality of scalar data from the data memory, allocate each of the scalar data to a corresponding processing unit group, and broadcast the scalar data to all the shared register groups contained in the processing unit group so that they can be shared and used by all the processing units in the processing unit group.

[0026] The memory access network is further configured to read a vector data matching the vector width from the data memory, broadcast the vector data to multiple processing unit groups, and write the vector data into all local register groups included in each processing unit group, so that the vector data becomes a vector operand shared by multiple vector operations.

[0027] In one embodiment of this disclosure, it further includes:

[0028] The scalar module is used to execute scalar operation instructions and instruction flow control instructions, wherein the instruction flow control instructions are used to change the order of instruction execution;

[0029] The memory access network is also used to perform data transfer between the data storage and the scalar module.

[0030] In one embodiment of this disclosure, the configuration information of the instruction stored in the configuration register set includes configuration information of the vector width and configuration information of the shared mode of the P parallel multiply-accumulate units.

[0031] A second aspect of this application provides a neural network chip, the neural network chip including the programmable parallel processing device as described in the first aspect of this application.

[0032] A second aspect of this application provides an electronic device, the electronic device including a neural network chip as described in the second aspect of this application.

[0033] As can be seen from the above embodiments of this application, the programmable parallel processing device, neural network chip, and electronic device provided in this application allow the vector module to be dynamically reconstructed according to the vector width of the vector data to be processed. All processing units of the vector module are divided into multiple processing unit groups, and multiple vector operations of corresponding widths are executed in parallel. This can improve the parallelism of vector operations while ensuring hardware utilization efficiency, and accelerate algorithm processing when the width of the vector operation becomes smaller. For convolutional layer operations in deep neural network algorithms, efficient processing can be achieved. Generally, the later the follower network layer, the smaller the feature map size of its convolutional layer. Since the vector width of the vector data to be processed corresponds to the feature map size, when the feature map size of the convolutional layer becomes smaller, the vector module can provide higher vector parallelism to accelerate the convolutional layer calculation. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the structure of a programmable parallel processing device provided in an embodiment of this application;

[0036] Figure 2 This is a schematic diagram of the structure of a processing unit group provided in an embodiment of this application;

[0037] Figure 3 A schematic diagram illustrating a memory access network according to different vector widths loading vector data to multiple processing unit groups, as provided in an embodiment of this application;

[0038] Figure 4 A schematic diagram illustrating a memory access network according to different vector widths loading scalar data into multiple processing unit groups, as provided in an embodiment of this application;

[0039] Figure 5 A schematic diagram illustrating four different data sharing modes of the four multiply-accumulate units in a processing unit provided in an embodiment of this application;

[0040] Figure 6 This is a schematic diagram illustrating local data sharing between each processing unit in a processing unit group provided in an embodiment of this application and multiple neighboring processing units. Detailed Implementation

[0041] To make the purpose, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of a programmable parallel processing device provided in an embodiment of this application. The programmable parallel processing device includes:

[0043] Input / output interface 10 is used for data exchange with the outside world.

[0044] The instruction memory 20 is used to load instructions from the outside through the input / output interface 10 and to store the instructions, including arithmetic instructions, data movement instructions, instruction flow control instructions, and configuration instructions.

[0045] Configuration register group 30 is used to store configuration information for this instruction that can be reused.

[0046] The instruction controller 40 is used to look up the configuration information of the instruction from the configuration register group 30, decode the configuration information to obtain the control information corresponding to the instruction, and send the instruction to the instruction execution unit based on the control information to execute the instruction.

[0047] The instruction execution unit includes a vector module 50 and a memory access network 60. The vector module 50 includes a shared register set 510 and multiple dynamically reconfigurable processing units 520. The vector module 50 is used to dynamically reconfigure the multiple processing units 520 into multiple processing unit groups according to the vector width of the vector data to be processed, so as to perform parallel operations of single instruction multiple vectors. The shared register set 510 is used to store scalar data, and the memory access network 60 is used to complete data transfer between the data memory and the vector module 50.

[0048] The data storage 70 is used to exchange data with the outside through the input / output interface 10 and to store computation data, including input data, output data, and intermediate computation results, as well as other computation-related data.

[0049] In one embodiment of this disclosure, the programmable parallel processing device further includes a scalar module 80 for executing scalar arithmetic instructions and instruction flow control instructions, the instruction flow control instructions being used to change the order of instruction execution. Furthermore, the scalar module 80 includes a scalar register set and a scalar arithmetic logic unit, and the instruction flow control instructions include branch instructions and jump instructions for implementing loop processing in the program. The memory access network 60 is also used to perform data transfer between the data memory 70 and the scalar module 80.

[0050] like Figure 1 As shown, this programmable parallel processing device is a programmable device that can employ a custom reduced instruction set. Instructions corresponding to the computation process of a specific processing algorithm are stored in the instruction memory 20. Data to be processed, intermediate results, and final processing results are stored in the data memory 70. After instruction fetching and decoding, the instruction controller 40 sends the instruction to the instruction execution units, such as the vector module 50, scalar module 80, and memory access network 60, to complete the instruction operation. These instructions include arithmetic instructions, data movement instructions, instruction flow control instructions, and configuration instructions. Arithmetic instructions read operands from various types of data register sets (e.g., shared register set 510, local register set, etc.), send them to the basic arithmetic logic unit for calculation, and then write them back to the shared register set 510. Data movement instructions move data between the shared register set 510 and the data memory 70. Instruction flow control instructions change the order in which the instruction controller 40 fetches instructions. Configuration instructions write specific configuration information to the configuration register set 30, which is then referenced by subsequent instructions.

[0051] In one embodiment of this disclosure, instruction decoding can be divided into two parts. One part involves decoding indirectly referenced configuration information. In this part, the instruction stored in the instruction memory 20 carries a configuration register number. Specifically, the instruction controller 40 retrieves the instruction from the instruction memory 20, searches for the corresponding configuration register in the configuration register group 30 based on the configuration register number included in the instruction, reads the configuration information from the configuration register, and decodes the configuration information to obtain the control information corresponding to the instruction. The other part involves directly decoding the configuration information directly encoded in the instruction. Specifically, the instruction controller 40 further decodes the configuration information directly encoded in the instruction to obtain the control information corresponding to the instruction. In this embodiment, these two parts can be performed simultaneously or not simultaneously; this disclosure does not impose any restrictions on this. Finally, all control information corresponding to the current instruction is obtained.

[0052] In one example of this disclosure, configuration register set 30 may contain four configuration registers that can be directly indexed (by number) by instructions, storing configuration information required by runtime instructions. Instructions may include a 2-bit configuration register number, allowing them to index a register from configuration register set 30 as a source or destination for configuration information. Configuration instructions are responsible for storing specific instruction configuration information in configuration register set 30. This specific configuration information can then be repeatedly referenced by multiple instructions over a period of time, making instruction coding more efficient.

[0053] In one embodiment of this disclosure, the vector module 50 includes 2 N 520 and 2 processing units M There are 2 shared register groups 510, each of which is divided into 2. N-M The processing units 520 share the same vector module 50, where N and M are both positive integers greater than 0. Specifically, this vector module 50 is used in a vector with a width of 2... L In the case of 2 N Each processing unit 520 is divided into 2 N-L There are 256 processing unit groups, where N and M are both positive integers greater than 0, and N ≥ L ≥ M. In one example, N = 8, M = 5, and the vector module 50 contains 256 processing units 520 and 32 shared data register groups, each shared data register group being shared by 8 processing units 520.

[0054] In this embodiment, the vector module 50 performs single-instruction multi-vector operations, performing element-wise vector operations on each vector data. Each vector operation can have either vector data or scalar data as operands, and the result of the operation is vector data. The scalar data is stored in the shared register group 510, and the vector data is stored in the local data register group.

[0055] In one embodiment of this disclosure, the processing unit 520 includes: a local register group 5210, a basic arithmetic logic unit 5220, P parallel multiply-accumulate units 5230, and a complex function unit 5240. The P parallel multiply-accumulate units 5230 share input vector data and scalar data, where P is a positive integer. The local register group 5210 is used to store vector data. The basic arithmetic logic unit 5220 is used to perform at least the following operations: multiplication, addition, subtraction, comparison, bitwise AND, bitwise OR, bitwise NOT, left shift, and right shift. The multiply-accumulate unit 5230 is used to perform multiply-accumulate operations. The complex function unit 5240 is used to fit nonlinear function operations through approximate calculation.

[0056] In this embodiment, the vector module 50 can be dynamically reconstructed according to the vector width of the vector data to be processed. All processing units 520 in the vector module 50 can be divided into multiple processing unit groups, and multiple processing unit groups can execute multiple vector operations of corresponding widths in parallel. The reconstructable feature of the vector module 50 supports improving the parallelism of vector operations while ensuring hardware utilization efficiency when the vector operation width decreases, thus accelerating the processing of the algorithm. It can achieve efficient processing of convolutional layer operations in deep neural network algorithms. Generally, as the neural network layers are further back, the feature map size of the convolutional layer becomes smaller. Since the vector width of the vector data to be processed corresponds to the size of the feature map, when the size of the convolutional layer feature map decreases, the vector module 50 can provide higher vector operation parallelism to accelerate the convolutional layer calculation.

[0057] In one embodiment of this disclosure, the configuration information of the instruction stored in the configuration register group 30 includes the configuration information of the vector width and the shared mode configuration information of the P parallel multiply-accumulate units 5230. According to this embodiment, the operation instructions in the vector module 50 simultaneously use the configuration information of the vector width and the shared mode configuration information of the P parallel multiply-accumulate units 5230. Memory access instructions only use the configuration information of the vector width; the shared mode configuration within the processing unit 520 has no effect on it. These two configurations provide rich flexibility for the vector module 50 and the memory access network 60, enabling the programmable parallel processing device of this disclosure to flexibly and efficiently handle various image processing algorithms. Furthermore, in image processing, such as layer-by-layer processing in deep neural networks, within the processing scope of a single-layer neural network, the operation instructions and memory access instructions of the vector module 50 will only use a few or one configuration options. Therefore, these few or one configuration options that will be used within a certain period are written into the configuration register group 30 and then repeatedly referenced by a large number of instructions, thereby improving the global configuration efficiency of the instructions.

[0058] In one example, the vector width configuration can include six configuration options: 1, 2, 4, 8, 16, and 32, encoded using 3 bits. Vector module 50, for example, includes 256 processing units 520, which can be dynamically reconfigured based on the vector width. Figure 2 As shown, during reconstruction, taking vector widths W = 256, 128, 64, 32, 16, 8 as examples, the 256 processing units 520 are divided into U (U = 1, 2, 4, 8, 16, 32) processing unit groups. Each processing unit group can complete 1 basic arithmetic logic vector operation, P multiply-accumulate vector operations, and 1 complex function vector operation. Therefore, U processing unit groups can complete U basic arithmetic logic vector operations, P * U multiply-accumulate vector operations, and U complex function vector operations.

[0059] In one example, taking a processing unit 520 containing four parallel multiply-accumulate units 5230 as an example, the sharing mode configuration of the four multiply-accumulate units 5230 within the processing unit 520 can include four sharing mode selections, encoded using 2 bits. The four parallel multiply-accumulate units 5230 can share both vector data input and scalar data input. The processing unit 520 supports four different sharing modes; under different sharing modes, the four multiply-accumulate units 5230 employ different vector data sharing and scalar data sharing configurations, such as... Figure 3 As shown. This further improves the data sharing rate in vector module 50.

[0060] According to the example above in this disclosure, the two configuration codes are combined into 5 bits, while the instruction only needs 2 bits to encode the register number in configuration register group 30. Thus, the code length of the instruction is greatly reduced.

[0061] In one embodiment of this disclosure, the processing unit 520 is further configured to access vector data stored in the local register group 5210 of the target processing unit 520, the target processing unit and the processing unit 520 belonging to the same processing unit group. In one example, vector data elements stored in the local data register group of the processing unit 520 in each processing unit group can be shared by four adjacent processing units 520, including the two adjacent processing units 520 on the left and the two adjacent processing units 520 on the right. Within each processing unit 520, the data in the local register group 5210 of the four adjacent processing units 520 and its own local register group 5210 can be selected by a 5-port multiplexer and sent to the subsequent arithmetic unit for computation, such as... Figure 4 As shown. This local data sharing characteristic can improve the efficiency of operations such as convolution, filtering, and pooling, accelerating data access and computation in the horizontal direction.

[0062] In one embodiment of this disclosure, the memory access network 60 is also used to adaptively load and store data for the vector module 50.

[0063] In one embodiment of this disclosure, such as Figure 5 As shown, the memory access network 60 is also used to read a vector data matching the vector width from the data memory 70, broadcast the vector data to multiple processing unit groups, and write the vector data into all local register groups 5210 included in each processing unit group, so that the vector data becomes a shared vector operand for multiple vector operations. In this disclosure, the adaptive memory access network 60 provides flexible and efficient memory data loading functionality while also achieving data reuse through broadcasting, reducing the frequency of memory accesses.

[0064] In one embodiment of this disclosure, such as Figure 6 As shown, the memory access network 60 is also used to read multiple scalar data from the data memory 70, allocate each scalar data to a corresponding processing unit group, and broadcast the scalar data to all shared register groups 510 contained in the processing unit group so that it can be shared by all processing units 520 within the processing unit group. Different processing unit groups use different scalar data. In this disclosure, the adaptive memory access network 60 provides flexible and efficient memory data loading functionality while also achieving data reuse through broadcasting, thereby reducing the frequency of memory access.

[0065] This disclosure also provides a neural network chip, which includes, as described above... Figure 1 The programmable parallel processing device shown.

[0066] This disclosure also provides an electronic device that includes the neural network chip described above. The electronic device includes data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, mobile phones, dashcams, navigators, sensors, cameras, servers, cloud servers, cameras, camcorders, projectors, watches, headphones, mobile storage devices, wearable devices, vehicles, home appliances, and / or medical devices, etc.

[0067] The means of transportation include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, range hoods, etc.; the medical equipment includes MRI machines, ultrasound machines and / or electrocardiographs, etc.

[0068] It should be noted that the functional modules in the various embodiments of this disclosure can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0069] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product.

[0070] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0071] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0072] The above is a description of a programmable parallel processing device, neural network chip, and electronic device provided by the present invention. For those skilled in the art, based on the ideas of the embodiments of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A programmable parallel processing device, characterized in that, include: Input / output interfaces are used for exchanging data with external systems. An instruction memory is used to load instructions from an external source via the input / output interface and to store the instructions. A configuration register group is used to store configuration information for the reusable instructions, including configuration information for the vector width and configuration information for the shared mode of P parallel multiply-accumulate units; The instruction controller is used to look up the configuration information of the instruction from the configuration register group, decode the configuration information to obtain the control information corresponding to the instruction, and send the instruction to the instruction execution unit based on the control information to execute the instruction; The instruction execution unit includes a vector module and a memory access network. The vector module includes a shared register set and multiple dynamically reconfigurable processing units. The vector module is used to dynamically reconfigure the multiple processing units into multiple processing unit groups according to the vector width of the vector data to be processed, so as to perform parallel operations of single instruction multiple vectors. The shared register set is used to store scalar data. The memory access network is used to complete data transfer between the data memory and the vector module. It is also used to read multiple scalar data from the data memory, allocate each scalar data to a corresponding processing unit group, and broadcast the scalar data to all the shared register sets contained in the processing unit group so that it can be shared by all the processing units in the processing unit group. It is also used to read a vector data matching the vector width from the data memory, broadcast the vector data to multiple processing unit groups, and write the vector data to all the local register sets contained in each processing unit group so that the vector data becomes a vector operand shared by multiple vector operations. The data storage device is used to exchange data with the outside world through the input / output interface and to store computational data.

2. The programmable parallel processing device according to claim 1, characterized in that, The instructions stored in the instruction memory carry configuration register numbers; The instruction controller is specifically configured to retrieve the instruction from the instruction memory, locate the corresponding configuration register from the configuration register group according to the configuration register number included in the instruction, read configuration information from the configuration register, decode the configuration information, and obtain the control information corresponding to the instruction. The instruction controller is further configured to decode the configuration information directly encoded in the instruction to obtain the control information corresponding to the instruction.

3. The programmable parallel processing device according to claim 1, characterized in that, The vector module includes 2 N One processing unit and 2 M There are 2 shared register groups, each of which is 2 N-M All of the aforementioned processing units share the same N and M, where N and M are both positive integers greater than 0; The vector module is specifically used in the vector width of 2 L In the case of 2 N The processing unit is divided into 2 N -L The processing unit group; Where N and M are both positive integers greater than 0, and N ≥ L ≥ M.

4. The programmable parallel processing device according to claim 1, characterized in that, The processing unit includes: The local register set includes a basic arithmetic logic unit, P parallel multiply-accumulate units and a complex function unit, wherein the P parallel multiply-accumulate units share input vector data and scalar data, and P is a positive integer; The local register group is used to store vector data; The basic arithmetic logic unit is used to perform at least the following operations: multiplication, addition, subtraction, comparison, bitwise AND, bitwise OR, bitwise NOT, left shift, and right shift. The multiply-accumulate unit is used to perform multiply-accumulate operations; The complex function unit is used to fit nonlinear function operations through approximate calculation.

5. The programmable parallel processing apparatus according to any one of claims 1 to 4, characterized in that, The processing unit is also used to access vector data stored in the local register group of the target processing unit, wherein the target processing unit and the processing unit belong to the same processing unit group.

6. The programmable parallel processing apparatus according to claim 1, characterized in that, Also includes: The scalar module is used to execute scalar operation instructions and instruction flow control instructions, wherein the instruction flow control instructions are used to change the order of instruction execution; The memory access network is also used to perform data transfer between the data storage and the scalar module.

7. A neural network chip, characterized in that, The neural network chip includes a programmable parallel processing device as described in any one of claims 1 to 6.

8. An electronic device, characterized in that, The electronic device includes the neural network chip as described in claim 7.

Images