A data flow architecture neuromorphic processor

Abstract

The present invention discloses a data stream architecture neuromorphic processor, which includes a control unit, a 4D computing unit, a 1D computing unit, an activation unit, a data type conversion unit, a parameter on-chip memory, a 4D data on-chip memory, and a 1D data on-chip memory; the data stream architecture neuromorphic processor is connected to an off-chip memory through a bus, the control unit obtains the configuration information of the neural network algorithm from the off-chip memory, and configures each module according to the configuration information, and configures the computing function of the data stream architecture neuromorphic processor as a certain operator function; through multiple configuration switches by the control unit, the calculations of different operators in the neural network algorithm are sequentially completed, so as to realize the complete calculation of a certain neural network algorithm, and write the final output data into the off-chip memory. Based on the reconfigurable data stream architecture technology, this processor can simultaneously meet the calculations of artificial neural network algorithms and spiking neural network algorithms.

Description

Technical Field

[0001] The present invention relates to a data flow architecture neuromorphic processor, in particular to a neuromorphic processor based on reconfigurable data flow architecture technology and capable of running artificial neural network algorithms and spiking neural network algorithms. Background Art

[0002] The spiking neural network algorithm adopts bionic computing technology and constructs the algorithm by simulating the biological nervous system. The working process of the biological nervous system is roughly as follows: Synapses / dendrites are the input devices of neuron cells, which are connected to many other neuron cells and are responsible for collecting and transmitting signals from other neuron cells; the cell body is the computing device of neuron cells, which calculates all input signals; the axon is the output device of neuron cells, which transmits the calculation results from the cell body to the synapses / dendrites of other neuron systems. Information is transmitted between biological neuron cells through spike signals. The spike signal is emitted by the axon, received by the synapses / dendrites, and accumulated in the cell body. The spike signal can cause the membrane potential in the cell body to increase, and when the membrane potential reaches a certain threshold, a spike output will be triggered.

[0003] Therefore, the spiking neural network is a new type of algorithm different from traditional artificial neural networks (also called numerical neural networks, such as typical CNN or RNN neural network algorithms). The data transmitted by the spiking neural network is spike data (0 or 1), and the spiking neural network needs to drive the generation of spike data through membrane potential data.

[0004] Existing solutions can implement the calculation of the spiking neural network algorithm through a certain dedicated hardware architecture, such as a dedicated computing architecture of memory-computation integrated based on on-chip storage resources. However, the vast majority of dedicated computing hardware for spiking neural networks can only support the calculation of spiking neural network algorithms and cannot support artificial neural network algorithms. In actual algorithm applications, a single spiking neural network algorithm cannot independently complete the calculation task and needs to cooperate with artificial neural network algorithms to implement a complete application solution. Therefore, the processor architecture that only supports the calculation of spiking neural networks has poor flexibility in actual applications.

[0005] The data flow architecture can be used for the efficient calculation of artificial neural network algorithms. However, in existing solutions, the data flow architecture can only be used to implement artificial neural network algorithms (such as typical CNN algorithms). As in the previous patent 1 (invention name: A neural network multi-core tensor processor, application number: 202011423696.0), and the previous patent 2 (invention name: A neural network tensor processor, application number: 202011421828.6), they only support artificial neural network algorithms and cannot support spiking neural network algorithms. Summary of the Invention

[0006] To solve the above problems, the present invention proposes a data flow architecture neuromorphic processor, which can efficiently implement the calculations of artificial neural network algorithms and spiking neural network algorithms under a unified processor architecture.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A data flow architecture neuromorphic processor includes a control unit, a 4D computing unit, a 1D computing unit, an activation unit, a data type conversion unit, a parameter on-chip memory, a 4D data on-chip memory, and a 1D data on-chip memory;

[0009] The data flow architecture neuromorphic processor is connected to an off-chip memory through a bus; the off-chip memory is used to store the parameters, data, and algorithm configuration information required for calculations;

[0010] The control unit is used to obtain the configuration information of the neural network algorithm from the off-chip memory, configure each module according to the configuration information, configure the computing function of the data flow architecture neuromorphic processor as a certain operator function; and through multiple configuration switches, sequentially complete the calculations of different operators in the neural network algorithm, so as to realize the complete calculation of a certain neural network algorithm;

[0011] When performing an operator operation,

[0012] The parameter on-chip memory is used to store the parameters used during the operator operation;

[0013] The 4D data on-chip memory is used to store the 4D data used during the operator operation, and the 4D data is derived from the input data of the off-chip memory and the output of the activation unit;

[0014] The 1D data on-chip memory is used to store the 1D data used during the operator operation, and the 1D data is derived from the output of the activation unit;

[0015] The 4D computing unit is used to read the parameters and 4D data from the parameter on-chip memory and the 4D data on-chip memory, implement the calculation of the parameters and 4D data, or the calculation of the 4D data alone, and output a 4D result;

[0016] The 1D computing unit is used to read the 1D data, parameters, and 4D results from the 1D data on-chip memory, the off-chip memory, and the 4D computing unit, implement the calculation of the 4D result and the 1D data, or the calculation of the 4D result and the parameter, or the calculation of the 4D result alone, and output a 1D result;

[0017] The activation unit is used to perform activation calculations on 1D results, and up to three types of data can be calculated: 4D data, 1D data, and output data. Among them, the 4D data and 1D data are temporary data for the neural network algorithm and are stored in the 4D data on-chip memory and the 1D data on-chip memory respectively during the calculation process of the neural network algorithm; the output data is the final result of the neural network algorithm and is finally sent to the data type conversion unit.

[0018] The data type conversion unit is used to perform conversion processing on data types, and the calculation result of the data type conversion unit is written into the off-chip memory as the final output data.

[0019] Furthermore, in the spiking neural network calculation mode, the data type of the 4D data is spiking data; the data type of the 1D data is membrane potential data, and the data type of the output data is numerical data.

[0020] Furthermore, in the artificial neural network calculation mode, the data types of the 4D data, 1D data, and output data are all numerical data.

[0021] Furthermore, the configuration information includes the data path connection relationships between modules, the behavior modes of each module, and the data types read into or written out by each module, and is used to determine the behaviors of the parameter on-chip memory, the 4D data on-chip memory, the 1D data on-chip memory, the 4D calculation unit, the 1D calculation unit, the activation unit, and the data type conversion unit.

[0022] Furthermore, the 4D calculation unit is used to implement the operation functions of tensor operators, and the tensor operators include convolution, depthwise separable convolution, deconvolution, pooling, and interpolation.

[0023] Furthermore, the 1D calculation unit is used to implement the operation functions of linear operators or non-linear operators, including addition, multiplication, taking the maximum value, and taking the minimum value.

[0024] Furthermore, the activation unit is used to perform activation calculations for artificial neural network algorithms and spiking neural network algorithms.

[0025] Technical effects:

[0026] Compared with existing neural network processors, the feature of the present invention is to provide a unified processor architecture that can satisfy the calculations of both artificial neural network algorithms and spiking neural network algorithms. Description of the Drawings

[0027] Figure 1 is the architecture diagram of the data flow architecture neuromorphic processor of the present invention;

[0028] Figure 2 It is an architecture diagram of a data flow architecture neuromorphic processor in the computing mode of a spiking neural network;

[0029] Figure 3 It is an architecture diagram of a data flow architecture neuromorphic processor in the computing mode of an artificial neural network. Specific embodiments

[0030] To further illustrate the embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used to explain the operating principle of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these contents, those of ordinary skill in the art should be able to understand other possible implementation manners and the advantages of the present invention.

[0031] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0032] Embodiment 1

[0033] The present invention proposes an architecture example of a data flow architecture neuromorphic processor 10, as Figure 1 shown. It includes modules such as a control unit 101, a 4D computing unit 104, a 1D computing unit 105, an activation unit 107, a data type conversion unit 108, a parameter on-chip memory 102, a 4D data on-chip memory 103, and a 1D data on-chip memory 106.

[0034] The data flow architecture neuromorphic processor 10 is connected to an off-chip memory 20 through a bus. The off-chip memory 20 stores the parameters, data, and algorithm configuration information required for computing.

[0035] The control unit 101 obtains the configuration information of the neural network algorithm from the off-chip memory 20. The configuration information determines the behaviors of the parameter on-chip memory 102, the 4D data on-chip memory 103, the 1D data on-chip memory 106, the 4D computing unit 104, the 1D computing unit 105, the activation unit 107, and the data type conversion unit 108, including the data path connection relationships between the modules, the behavior modes of the modules, the types of data read into or written out by the modules (pulse data, 8-bit or 16-bit numerical data, etc.), the sizes of the data read into or written out by the modules (length, width, height, etc.), the addresses of the data read into or written out by the modules, and the source and destination selection of the data read into or written out by the modules.

[0036] The neural network algorithm run by the data flow architecture neuromorphic processor 10 consists of multiple operators of different types. After the control unit 101 configures each module according to the configuration information, the computing function of the data flow architecture neuromorphic processor 10 is configured as a certain operator function (such as pulse convolution, pulse pooling, etc.). Through multiple configuration switches of the control unit 101, the calculations of different operators in the neural network algorithm can be completed in sequence, so as to realize the complete calculation of a certain neural network algorithm.

[0037] The parameter on-chip memory 102 is used to store a part of the complete parameters of the neural network algorithm. The complete parameters of the neural network algorithm are stored in the off-chip memory 20. When the data flow architecture neuromorphic processor 10 performs the operation of a certain operator in the neural network algorithm, the relevant parameters of the operator are read out from the off-chip memory 20 and written into the parameter on-chip memory 102. During the calculation process, the 4D computing unit 104 reads the parameters from the parameter on-chip memory 102 for calculation.

[0038] The 4D data on-chip memory 103 is used to store 4D data. The 4D data refers to the data finally used by the 4D computing unit 104. The input data is stored in the off-chip memory 20. When needed, the input data is read out from the off-chip memory 20 and written into the 4D data on-chip memory 103. During the calculation process, the 4D computing unit 104 reads the 4D data from the 4D data on-chip memory 103 for calculation. The activation unit 107 writes the calculated 4D data result into the 4D data on-chip memory 103.

[0039] The 1D data on-chip memory 106 is used to store 1D data. The 1D data refers to the data finally used by the 1D computing unit 105. During the calculation process, the 1D computing unit 105 reads the 1D data from the 1D data on-chip memory 106 for calculation. The activation unit 107 writes the calculated 1D data result into the 1D data on-chip memory 106.

[0040] The 4D computing unit 104 is used to implement the operation of a tensor operator. Specifically, it realizes the calculation of parameters and 4D data, or the calculation of only 4D data (without parameters). The 4D computing unit 104 uses two inputs of parameters and 4D data, or only one input of 4D data, and calculates to obtain a 4D result. The 4D result is used by the 1D computing unit 105. The 4D computing unit 104 can implement multiple operator functions, including convolution, depthwise separable convolution, deconvolution, pooling, interpolation, etc.

[0041] The 1D calculation unit 105 is used to implement the operation of a linear operator or a non - linear operator. Specifically, in terms of operation, it implements the calculation of the 4D result and 1D data, or the 4D result and parameters, or the calculation of the 4D result alone. The 1D calculation unit 105 uses two inputs of the 4D result and 1D data, or two inputs of the 4D result and parameters, or only one input of the 4D result to calculate and obtain a 1D result. The 1D calculation unit 105 can implement various operator functions, including addition, multiplication, taking the maximum value, taking the minimum value, etc.

[0042] The activation unit 107 is used to implement the activation calculation of the 1D result. The activation calculation includes both the activation calculation of the artificial neural network algorithm and the activation calculation of the spiking neural network algorithm. The activation calculation of the artificial neural network algorithm refers to Relu, PRelu, Swish, Sigmoid, etc. The activation calculation of the spiking neural network algorithm refers to LIF (leaky integrate - and - fire model), IAF (integrate - and - fire model), IBT, etc.

[0043] The activation unit 107 uses the 1D result from the 1D calculation unit 105 as an input and can calculate up to three types of data: 4D data, 1D data, and output data. Among them, the 4D data and 1D data are used as temporary data of the neural network algorithm and are respectively stored in the 4D data - slice on - chip memory 103 and the 1D data - slice on - chip memory 106 during the calculation process of the neural network algorithm. The output data is used as the final result of the neural network algorithm and is finally sent to the data type conversion unit 108.

[0044] The data type conversion unit 108 is used to implement the conversion processing of data types. For example, it converts 8 - bit numerical data to 16 - bit numerical data, converts 1 - bit pulse data to 8 - bit numerical data, etc. The calculation result of the data type conversion unit 108 is written into the off - chip memory 20 as the final output data.

[0045] The data - flow architecture neuromorphic processor 10 can support both the spiking neural network algorithm and the artificial neural network algorithm simultaneously. The data - flow architecture neuromorphic processor 10 implements the calculation of the spiking neural network algorithm through the spiking neural network calculation mode and implements the calculation of the artificial neural network algorithm through the artificial neural network calculation mode. The two modes are determined by the control unit 101 according to the algorithm configuration information.

[0046] Embodiment 2

[0047] The spiking neural network calculation mode is as Figure 2 shown.

[0048] In the spiking neural network computing mode, the data type of the input data is spiking data. The spiking data refers to the data type with a data bit width of 1 bit or 2 bits. Among them, when the spiking data bit width is 1 bit, 0 represents no spike, and 1 represents a positive spike; when the spiking data bit width is 2 bits, 0 represents no spike, 1 represents a positive spike, 2 represents a negative spike, and 3 is meaningless. The data type of the output data is numerical data. The numerical data refers to the data with a bit width of 8 bits or 16 bits, and this data is usually an 8-bit signed integer type or a 16-bit half-precision floating-point type.

[0049] In the spiking neural network computing mode, the 4D on-chip memory 103 stores spiking data. The 1D on-chip memory 106 stores membrane potential data. The membrane potential data refers to the data with a bit width of 8 bits or 16 bits, and this data is usually an 8-bit signed integer type or a 16-bit half-precision floating-point type.

[0050] In the spiking neural network computing mode, the 4D computing unit 104 uses two inputs, namely parameters and spiking data, or only uses one input of spiking data to calculate and obtain a 4D result. The 1D computing unit 105 uses two inputs, namely the 4D result and the membrane potential data, to calculate and obtain a 1D result. The activation unit 107 obtains two results, namely the membrane potential data and the spiking data at the same time. Among them, the membrane potential data is written into the 1D on-chip memory 106, and the spiking data is written into the 4D on-chip memory 103 or the data type conversion unit 108 according to specific configurations.

[0051] In the spiking neural network computing mode, the data type conversion unit 108 implements the conversion function from spiking data to numerical data.

[0052] Embodiment 3

[0053] The artificial neural network computing mode is as Figure 3 shown.

[0054] In the artificial neural network computing mode, the data type of the input data is numerical data. The numerical data refers to the data with a bit width of 8 bits or 16 bits, and this data is usually an 8-bit signed integer type or a 16-bit half-precision floating-point type.

[0055] In the artificial neural network computing mode, the 4D on-chip memory 103 stores numerical data. The 1D on-chip memory 106 stores numerical data.

[0056] In the artificial neural network computing mode, the 4D computing unit 104 uses two inputs, namely parameters and numerical data, or only one input of numerical data to calculate a 4D result. The 1D computing unit 105 uses two inputs, namely the 4D result and numerical data, or the 4D result and parameters, or only one input of the 4D result to calculate a 1D result. The activation unit 107 calculates a result of numerical data, and the numerical data result can be written into the 1D on-chip memory 106, the 4D on-chip memory 103 or the data type conversion unit 108 according to specific configurations.

[0057] In the artificial neural network computing mode, the data type conversion unit 108 implements the type conversion of numerical data, such as the conversion from integer type to floating point type.

[0058] In summary, the data flow architecture neuromorphic processor 10 of the present invention provides a unified processor architecture that can support both spiking neural network algorithms and artificial neural network algorithms simultaneously.

[0059] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, those skilled in the art should understand that various changes can be made to the present invention in terms of form and details without departing from the spirit and scope of the present invention defined by the appended claims, and all of them fall within the protection scope of the present invention.

Claims

1. A data stream architecture neuromorphic processor, characterized in that, It includes a control unit, a 4D computing unit, a 1D computing unit, an activation unit, a data type conversion unit, a parameter on-chip memory, a 4D data on-chip memory, and a 1D data on-chip memory; The data flow architecture neuromorphic processor is connected to an off-chip memory through a bus; the off-chip memory is used to store the parameters, data, and algorithm configuration information required for computing; The control unit is used to obtain the configuration information of the neural network algorithm from the off-chip memory, configure each module according to the configuration information, configure the computing function of the data flow architecture neuromorphic processor as a certain operator function; and through multiple configuration switches, sequentially complete the computing of different operators in the neural network algorithm, so as to realize the complete computing of a certain neural network algorithm; When performing an operator operation, The parameter on-chip memory is used to store the parameters used during the operator operation; The 4D data on-chip memory is used to store the 4D data used during the operator operation, and the 4D data is sourced from the input data of the off-chip memory and the output of the activation unit; The 1D data on-chip memory is used to store the 1D data used during the operator operation, and the 1D data is sourced from the output of the activation unit; The 4D computing unit is used to implement the operation of a tensor operator, read 4D parameters and 4D data from the parameter on-chip memory and the 4D data on-chip memory, implement the calculation of 4D parameters and 4D data, or the calculation of 4D data alone, and output 4D results; The 1D computing unit is used to implement the operation of a linear operator or a non-linear operator, read 1D data and 1D parameters from the 1D data on-chip memory and the 4D computing unit, implement the calculation of 4D results and 1D data, or the calculation of 4D results alone, and output 1D results; The activation unit is used to implement the activation calculation of 1D results to obtain 4D data, 1D data, and output data. Among them, the 4D data and 1D data are used as temporary data of the neural network algorithm and are respectively stored in the 4D data on-chip memory and the 1D data on-chip memory during the calculation process of the neural network algorithm; the output data is used as the final result of the neural network algorithm and is finally sent to the data type conversion unit; The data type conversion unit is used to implement the conversion processing of data types, and the calculation result of the data type conversion unit is written into the off-chip memory as the final output data.

2. The data flow architecture neuromorphic processor according to claim 1, wherein In the spiking neural network computing mode, the data type of the 4D data is spiking data; the data type of the 1D data is membrane potential data, and the data type of the output data is numerical data.

3. The data stream architecture neuromorphic processor according to claim 1, characterized in that, In the artificial neural network computing mode, the data types of the 4D data, 1D data, and output data are all numerical data.

4. The data flow architecture neuromorphic processor according to claim 1, characterized in that, The configuration information includes the data path connection relationships between modules, the behavior modes of each module, and the data types read into or written out by each module, and is used to determine the behaviors of the parameter on-chip memory, the 4D data on-chip memory, the 1D data on-chip memory, the 4D computing unit, the 1D computing unit, the activation unit, and the data type conversion unit.

5. The data stream architecture neuromorphic processor according to claim 1, characterized in that, The 4D computing unit is used to implement the operation functions of tensor operators, and the tensor operators include convolution, depthwise separable convolution, deconvolution, pooling, and interpolation.

6. The data flow architecture neuromorphic processor according to claim 1, characterized in that The 1D computing unit is used to implement the operation functions of linear operators or non-linear operators, including addition, multiplication, maximum value selection, and minimum value selection.

7. The data flow architecture neuromorphic processor according to claim 1, characterized in that, The activation unit is used to implement the activation calculations of artificial neural network algorithms and spiking neural network algorithms.

Images