Data processing device and artificial intelligence processor

Abstract

The present disclosure relates to a data processing device and an artificial intelligence processor, wherein the data processing device is applied to the processing core of the artificial intelligence processor, and the data processing device is connected to a storage module and a computing module respectively, and includes: an address generation module, which is used to generate a first address of first data to be read according to a set address generation method; a data conversion module, which is used to read the first data from the storage module according to the first address, convert the read first data, and obtain the converted second data; send the second data to the computing module; a control module, which is used to determine the corresponding address generation method and control the address generation module to generate an address according to a preset processing category and data category. By generating the address of the corresponding data according to the processing category and data category and performing data conversion, the present disclosure can speed up the access speed of data and improve the computing efficiency of the neuromorphic chip.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing device and an artificial intelligence processor. Background Art

[0002] In recent years, the field of neuromorphic computing has developed rapidly. Neural networks can be directly constructed using hardware circuits to simulate the functions of the brain. Specifically, neuromorphic chips can be used to establish a large-scale, parallel, low-energy computing platform that can support complex pattern learning. On this basis, the types of neural networks represented by artificial neural networks (ANNs) and spiking neural networks (SNNs) are increasing, and the scale is getting larger and larger, which puts higher requirements on the computing power of neuromorphic chips. Therefore, neuromorphic chips should have higher data processing and sorting capabilities.

[0003] However, traditional neuromorphic chips can only process a limited number of data types, and some are limited to matrices or low-dimensional tensors. In addition, the data accuracy that traditional neuromorphic chips can process is limited, which makes it impossible to apply to more types of networks. For example, as a third-generation neural network, the pulse neural network, its data is encoded in the spatiotemporal information of neuron pulse signals, which can be described by three-value data (for example, 0, 1, and -1). However, most of the current neuromorphic chips cannot efficiently support multi-precision (for example, 8 bits or more) data calculations while supporting three-value data operations, nor can they convert with other types of data according to the computing requirements of neural networks. In addition, the module structure used by traditional neuromorphic chips for data processing is complex, and different processing methods are used for different computing tasks and data types (for example, vectors, matrices, and tensors), which greatly limits the computing efficiency of neuromorphic chips and the reasoning efficiency of neural networks, and has poor scalability. Summary of the invention

[0004] In view of this, the present disclosure proposes a data processing device, which improves the computing efficiency of the neuromorphic chip and the reasoning efficiency of the neural network by generating the address of the corresponding data according to the processing category and the data category and performing data conversion, and is easy to expand.

[0005] According to one aspect of the present disclosure, there is provided a data processing device, which is applied to the processing core of an artificial intelligence processor, wherein the artificial intelligence processor comprises a plurality of processing cores, each processing core comprises a storage module, a computing module and a data processing device, wherein the data processing device is connected to the storage module and the computing module respectively, and the data processing device comprises: an address generating module, which is used to generate a first address of first data to be read according to a set address generating method; a data conversion module, which is connected to the address generating module, and is used to read the first data from the storage module according to the first address; convert the read first data to obtain converted second data; and send the second data to the computing module so that the computing module performs data calculation; and a control module, which is connected to the address generating module and the data conversion module respectively, and is used to determine a corresponding address generating method according to a preset processing category and a data category of the first data, and control the address generating module to generate an address according to the address generating method.

[0006] According to another aspect of the present disclosure, an artificial intelligence processor is provided, the artificial intelligence processor comprising a plurality of processing cores, each processing core comprising a storage module, a computing module and a data processing device, the data processing device being connected to the storage module and the computing module respectively, the data processing device comprising: an address generating module, for generating a first address of first data to be read according to a set address generating method; a data conversion module, connected to the address generating module, for reading the first data from the storage module according to the first address; converting the read first data to obtain converted second data; sending the second data to the computing module so that the computing module performs data calculation; a control module, connected to the address generating module and the data conversion module respectively, for determining a corresponding address generating method according to a preset processing category and a data category of the first data, and controlling the address generating module to generate an address according to the address generating method.

[0007] According to the embodiments of the present disclosure, by generating the address of the corresponding data according to the processing category and the data category and performing data conversion, the access speed to the data can be accelerated, the computing efficiency of the neuromorphic chip and the reasoning efficiency of the neural network can be improved, and expansion is easy.

[0008] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0010] Figure 1 A block diagram showing a data processing device according to an embodiment of the present disclosure.

[0011] Figure 2 A schematic diagram showing a data processing device according to an embodiment of the present disclosure.

[0012] Figure 3 A schematic diagram showing an address generation module and a data processing process according to an embodiment of the present disclosure.

[0013] Figure 4 A schematic diagram showing the data processing process of an embodiment of the present disclosure.

[0014] Figure 5a A schematic diagram showing the data processing process of an embodiment of the present disclosure.

[0015] Figure 5b A schematic diagram showing the data processing process of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0017] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0018] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0019] Figure 1 A block diagram showing a data processing device according to an embodiment of the present disclosure.

[0020] like Figure 1 As shown, the data processing device 60 is applied to the processing core of the artificial intelligence processor, the artificial intelligence processor includes multiple processing cores, each processing core includes a storage module 40, a computing module 50 and a data processing device 60, the data processing device is connected to the storage module and the computing module respectively, and the data processing device includes:

[0021] The address generating module 10 is used to generate a first address of first data to be read according to a set address generating method;

[0022] The data conversion module 20 is connected to the address generation module and is used to: read the first data from the storage module according to the first address; convert the read first data to obtain converted second data; and send the second data to the calculation module so that the calculation module performs data calculation;

[0023] The control module 30 is connected to the address generation module and the data conversion module respectively, and is used to: determine the corresponding address generation method according to the preset processing category and the data category of the first data, and control the address generation module to generate an address according to the address generation method.

[0024] In one possible implementation, the data processing device, storage module and computing module may be located in a processing core of an artificial intelligence processor. The artificial processor may be a neuromorphic chip, and the processing core may be a functional core of the neuromorphic chip.

[0025] A neuromorphic chip is a new type of chip that can simulate the working mechanism of the human brain. The neuromorphic chip can be organized according to processing cores. For example, a neuromorphic chip can include 4096 (e.g., a 64×64 array) processing cores, each of which can simulate 256 biological neurons. In this way, a neuromorphic chip can simulate about 1 million neurons in total.

[0026] In a possible implementation, the processing core can organize neurons according to a two-dimensional mesh structure. For example, a neuron may include an axon, a dendrite, and a synapse, etc. The axon may be a row of a two-dimensional mesh structure, the dendrite may be a column of a two-dimensional mesh structure, and the synapse may be a cross point of the rows and columns. Among them, the axon can be used to send data, the dendrite can be used to receive data, and the synapse can be used to store weights. The weight can be the weighted value of the latter neuron to the previous neuron before the synapse, corresponding to the weight parameter of the neural network. For example, when a neuron is not used, the weight of the neuron can be set to 0. When a neuron needs to be used for calculation, a weight value can be assigned to the neuron.

[0027] In one possible implementation, the processing core may include a storage module and a computing module, that is, the neuromorphic chip may adopt a storage-computing integrated processing architecture. Among them, the storage module may be used to store data and instructions related to neural network calculations, and the computing module may be used to perform neural network calculations (e.g., multiplication-accumulation operations). In one example, the storage module may be a memory with a certain storage capacity, storing different types of data such as vectors, matrices, and tensors in neural network calculations, and the computing module may be a multiplication-accumulator array (MACs) composed of 128 multiplication-accumulators. The multiplication-accumulator array may be divided into 4 groups, each group including 32 multiplication-accumulators, which are used to perform multiplication-accumulation operations on the output data of the data processing device, and the operation results may be returned to the data processing device for further processing.

[0028] In a possible implementation, the processing core can perform artificial neural network calculations, pulse neural network calculations, and hybrid neural network calculations that mix artificial neural networks and pulse neural networks. Different neural network calculation tasks require processing data of different precisions. For example, for pulse neural networks, time information is an important consideration. A neuron in a pulse neural network is activated only when its membrane potential reaches a specific value. When the neuron is activated, it generates a signal to transmit to other neurons, increasing or decreasing the membrane potential of other neurons. Therefore, before sending the three-value data to the calculation module for calculation, the three-value data needs to be converted into data precision acceptable to the calculation module for calculation.

[0029] In one possible implementation, the neuromorphic chip disclosed in the present invention can adopt a storage-computing integrated chip architecture, integrating the processor, storage module and communication components together, and information processing is performed locally, which is conducive to better meeting the needs of large-scale parallel computing of neural networks. The neuromorphic chip disclosed in the present invention can support calculations of two computing paradigms, which can support artificial neural network calculations, pulse neural network calculations, and hybrid neural network calculations that mix artificial neural networks and pulse neural networks. In this way, the neuromorphic chip can calculate neural networks in a distributed, large-scale and parallel manner with extremely low energy consumption. It should be understood by those skilled in the art that the neuromorphic chip and processing core are exemplary. The present disclosure does not limit to which processor architecture the data processing device is applied and whether the processor is organized according to the processing core.

[0030] In one possible implementation, different processing cores can process different neural network computing tasks, and the same processing core can also perform different neural network computing tasks in different working cycles. For example, in a certain working cycle, processing core A can be used to process neural network computing tasks related to images, and processing core B can simultaneously process neural network computing tasks related to text. Data of different neural network computing tasks can be assigned to one processing core for processing, or to different processing cores for processing. Therefore, data of different types and different precisions are frequently exchanged between processing cores and within processing cores, and it is necessary to find a suitable solution to efficiently schedule data of different types and different precisions.

[0031] Traditional chips use the von Neumann architecture to separate the storage module and the processor. The processor repeatedly exchanges information with the storage module through the bus. The read and write efficiency of data of different types and different precisions is low, and there are bottlenecks in computing power and energy consumption. At the same time, the existing neuromorphic chips are decentralized in processing data of different types and different precisions, that is, different processing methods are used for different computing tasks and data types. Existing neuromorphic chips can only support one of the computing paradigms of artificial neural network computing or pulse neural network computing, such as neuromorphic chips using leaky-integrate-and-fire (LIF) neuron model. These neuromorphic chips use different processing methods for different computing tasks and data types, which increases the number of accesses to the storage module, the process of scheduling data is cumbersome, the access efficiency is not high, and information redundancy may be caused. Therefore, traditional neuromorphic chips cannot efficiently schedule data of different types and / or different precisions, and the access efficiency to the storage module is low, which greatly limits the computing efficiency of neuromorphic chips and the reasoning efficiency of neural networks, and has poor scalability.

[0032] When traditional neural network chips access different types of data, the storage areas of each data block of different types of data are quite different. Therefore, neurons need to frequently access each separate storage area, which requires a high ability to access storage, and the module structure used for data processing is complex. The scheduling efficiency of the entire neural network chip for data of different types and / or different precisions is relatively low.

[0033] The present disclosure proposes a data processing device, which can be located in the processing core of a neuromorphic chip and uniformly schedule data of different types and / or different precisions. By centralizing data of different types and / or different precisions to the data processing device for processing, not only can data of different types and / or different precisions be efficiently scheduled and access speed of the data be accelerated, but also different computing modes and storage methods can be adapted to improve the computing efficiency of the neuromorphic chip and the reasoning efficiency of the neural network, without generating information redundancy and easy to expand.

[0034] Figure 2 A schematic diagram showing a data processing device according to an embodiment of the present disclosure.

[0035] In one possible implementation, Figure 2 As shown, the data processing device may include an address generating module (AGU, Address Generator Unit), a data conversion module and a control module. It should be understood by those skilled in the art that although Figure 2 Although the storage module is not shown in the figure, the data processing device can also interact with the storage module. For example, the data processing device can perform read and write operations on the storage module.

[0036] In a possible implementation manner, the address generation module may be used to generate a first address of first data to be read according to a set address generation method.

[0037] In one possible implementation, the first data includes multiple categories, that is, the first data may be different types of data. For example, the first data may include vectors, matrices, tensors, and the like. A vector may be a row or column of data, a matrix may be a two-dimensional data set, and a tensor may be a multi-dimensional data set (for example, a color picture contains data in three dimensions of length, width, and color, which can be represented by a three-dimensional tensor). It is worth noting that the output pulse sequence of neurons in a pulse neural network can be encoded as three-valued data. For example, (0, 1, and -1) can be used to describe that the neurons of a pulse neural network are in a state of no action, activation, and inhibition, respectively, and the three-valued data may also have different types. It should be understood by those skilled in the art that the present disclosure is not limited to different types of data.

[0038] In a possible implementation, the first data may be stored in a storage module connected to the data processing device in a certain storage mode. For example, the matrix data may be stored in a storage area limited by a certain address range in the storage module in the form of a one-dimensional vector, and may be stored in a centralized manner or in a decentralized manner. The bias vector may be stored in a storage area different from the matrix data in the storage module. Different types of data have different characteristics, and therefore may have different storage modes. In addition, the data processing device may read and write the data in a certain bit width. In one example, the bit width of the data bus used to read and write data may be 128 bits.

[0039] In a possible implementation, the data category of the first data may include: at least one of input image data, parameter weight data, and bias data; the data category of the first calculation result includes a membrane potential value. It should be understood by those skilled in the art that the present disclosure does not limit the data category and the data category of the first calculation result.

[0040] In a possible implementation, during the entire data processing process, the processing of the data may also include: data loading, data caching, and data shifting, reordering, and reorganization. For example, the data processing device may read the data of a color image from a storage module, and the data of the color image may be a three-dimensional tensor. The data processing device may send the read tensor data to a data conversion module for conversion. In the process of reading the tensor data, the data processing device may load some parameters in the PI register to change the order of reading the tensor data. The register group unit may also cache some data to speed up data access. The data processing device may also move a certain bit of the data and change the bitwise arrangement order, or may perform reorganization operations such as splitting or merging the data, for example, splicing two 8-bit integer data into a 16-bit integer data or splitting a 16-bit integer data into two 8-bit integer data. It should be understood by those skilled in the art that the above examples of the processing are exemplary, and more processing methods may be added in other embodiments, and the present disclosure does not limit the processing.

[0041] In a possible implementation, the first address of the first data may be an address where different types of data are stored in a storage module. The first address may be a starting address for data storage, or an address of a certain storage area. For example, the starting address of data of a certain type being a matrix may be 0x20020000, and when reading the data, the data may be read according to the starting address of the data. The present disclosure does not limit how to read and write data in a storage module, and will not be described in detail herein.

[0042] Table 1 shows the loops required for typical computing modes and their meanings. In the present disclosure, the processing categories can be understood as the computing modes in Table 1. As shown in Table 1, the processing categories (i.e., computing modes) may include convolution, vector dot multiplication, vector accumulation, average pooling, and full connection, etc. In addition, the processing categories may also include: at least one of convolution, vector dot multiplication, tensor dot multiplication, vector accumulation, average pooling, data loading, data caching, data shifting, data reordering, and data reorganization. It should be understood by those skilled in the art that the description of the computing mode in the present disclosure is exemplary and not exhaustive, and the present disclosure does not limit the computing mode.

[0043] In a possible implementation, as shown in Table 1, for different computing modes, the required loop count levels may be different. For example, for a computing mode such as vector dot multiplication, only three loop levels, L3, L4, and L5, may be used. The present disclosure does not limit the loop levels used for the computing modes.

[0044] In one example, as shown in Table 1, for common convolutional neural network calculations, the calculation mode can be divided into 6 steps, and each step can be processed using loop counters of different levels. For example, the L0 level loop can process input channel grouping, the L1 level loop can process convolution kernel width Kx, the L2 level loop can process convolution kernel height Ky, the L3 level loop can process output channel grouping, the L4 level loop can process output image width Ox, and the L5 level loop can process output image height Oy. The loop order of the loop counting submodule can also be flexibly configured. For example, the L1 level loop can process convolution kernel height Ky, and the L2 level loop can process convolution kernel width Kx. It should be understood by those skilled in the art that the number of levels of loop counting can be changed, for example, the loop counting level is expanded to 8 levels. The order and number of loop counting disclosed in the present invention are not limited.

[0045] Table 1

[0046]

[0047] In a possible implementation, the set address generation method may be a plurality of methods for dynamically generating the first address of the first data to be read according to the processing category and data category of the first data. For example, the data categories processed by the convolution operation and the vector dot multiplication are different. The convolution operation may process tensors, and the vector dot multiplication may process vectors. The processing categories of the convolution operation and the vector dot multiplication are also different, that is, the convolution element and the vector dot multiplication are two different calculation modes with different calculation steps. Therefore, when processing the convolution operation and the vector dot multiplication separately, the address of the data to be read can be dynamically generated according to these differences between the two, so as to be more efficient and simple.

[0048] In one possible implementation, the address generation module includes: an address generation submodule, used to generate first addresses of various categories of first data respectively; and a loop calculation submodule, used to loop calculate the first address of the first data to be read next time according to the first address of the first data to be read currently and the address generation method.

[0049] In a possible implementation, the address generation submodule is used to generate first addresses of various types of first data. The address generation submodule includes a plurality of address generation units, each of which is used to generate first addresses of various types of first data. Figure 2 As shown, the address generation submodule can be composed of four identical address generation units (AGU_InA, AGU_InB, AGU_V and AGU_Bias). The four address generation units can process four parameters of the neural network calculation model. Since the calculation model of the neural network has certain commonalities, the use of four address generation units is sufficient to meet the calculation requirements of the neural network. Each address generation unit can generate addresses corresponding to different neural network parameters, pointing to the area where the corresponding neural network parameters are stored in the storage module, so that the data processing device can directly read and write the storage area corresponding to the data from the storage module. For convenience, the four address generation units AGU_InA, AGU_InB, AGU_V and AGU_Bias are referred to as the first address generation unit, the second address generation unit, the third address generation unit and the fourth address generation unit respectively. It should be understood by those skilled in the art that the present disclosure does not limit the number of the address generation units.

[0050] In a possible implementation, the address generating unit can be switched according to different processing categories (i.e., calculation modes) and the data categories of the first data. For example, a common convolutional neural network calculation mode can be expressed as: V=X×W+Bias, where V can represent the value of the membrane potential, and the neuron is activated when the membrane potential reaches a certain threshold. X can represent the input of the convolutional neural network calculation, W can represent the weight of the convolutional neural network calculation, and Bias can represent the bias of the convolutional neural network calculation. For this neural network calculation mode, the first address generating unit can generate the address of the input image X. If the input image of the convolutional neural network is a color picture, the input image X can be a three-dimensional tensor. The second address generating unit can generate the address of the weight parameter W, and the weight parameter W can be a two-dimensional matrix. The third address generating unit can generate the address of the membrane potential, and the fourth address generating unit can generate the address of the bias. The membrane potential can be a quantitative value, and the bias can be a vector. Among them, the two parameters of membrane potential and bias are commonly used parameters in neural network calculations. The corresponding values ​​can be configured in the control module so that when the addresses of these two parameters need to be generated for reading and writing, the address generation module can directly generate the required addresses without performing address conversion, so that the data processing device can perform read and write operations more quickly.

[0051] In one possible implementation, for different processing categories and data categories of the first data, the address generating unit can be flexibly switched to perform different functions. For example, the first address generating unit can generate the address of the weight parameter, and the second address generating unit can generate the address of the input tensor. When the neural network calculation does not require biasing, the fourth address generating unit may not work. The address generated by the address generating unit can be selected and output by a multiplexer so that only part of the four parameters are operated in a certain time period. It should be understood by those skilled in the art that the naming and combination of the address generating units disclosed in this disclosure are only exemplary and do not constitute a limitation.

[0052] In a possible implementation, the loop calculation submodule is used to loop calculate the first address of the first data to be read next time according to the first address of the first data to be read currently and the address generation method. Specifically, the loop counting submodule may include a 6-level loop counter. The loop counter can be implemented by hardware or software. Preferably, a hardware loop counter (HWL, Hardware Loop Counter) is used to implement loop counting in order to speed up the loop speed. Each level of loop can correspond to different meanings and different steps of neural network calculation. The data processing device can read different types of data stored in the corresponding storage area from the storage module according to the address generated by the address generation submodule, and send it to the calculation module (i.e., the multiplier accumulator array) for calculation. Different data have different addresses, and a 6-level loop counter can be used to transform the addresses of different data.

[0053] In a possible implementation, the address generation submodule and the cycle counting submodule can be switched according to the difference between the calculation mode and the storage mode. For the same calculation mode, the functions of the address generation submodule and the cycle counting submodule can also be different in different time periods. When some address generation submodules and cycle counting submodules do not receive relevant instructions, the address generation module can only work partially. The instructions can be sent by an external module through a control bus, or can be pre-stored in a PI register of a control module. The address generation module can also generate a read-write enable signal to control when the data processing device reads and writes data from a storage module. Multiple read-write enable signals can be output via an OR gate.

[0054] In one possible implementation, the data conversion module is connected to the address generation module, and is used to: read the first data from the storage module according to the first address; convert the read first data to obtain converted second data; and send the second data to the calculation module so that the calculation module performs data calculation.

[0055] In one possible implementation, the data conversion module includes a precision conversion unit, which is used to: perform precision conversion on the first data according to the data precision suitable for the calculation module to obtain the second data; the precision conversion unit is also used to: perform precision conversion on the second calculation result according to the data precision suitable for the storage module to obtain the first calculation result.

[0056] In one possible implementation, the precision conversion unit can be used to convert data of different precisions, and send the converted data to the calculation module for calculation. The output data of the calculation module can also be converted in precision to facilitate storage or send it to other modules for processing. In one example, the calculation module can be a multiplier accumulator array, divided into 4 groups, each group can include 32 multiplier accumulators. The multiplier accumulator can be implemented in any possible way, and the present disclosure is not limited to this. The multiplier accumulator array can only receive data with a precision of int9 (i.e., integer 9 bits). In this way, data of different precisions, including int16, int8, and three-value data, etc., need to be converted into int9 by the precision conversion module before being sent to the multiplier accumulator array for calculation. It is worth noting that in order to support more calculation modes (such as vector accumulation), in addition to the communication between the data conversion module and the multiplier accumulator array, there is a communication channel between the data conversion module and the multiplier accumulator array. Figure 2 The data line Data_InA_to_D shown in the figure also has a data line Data_InB_to_D. The data line Data_InA_to_D can only receive int9 data, while the data line Data_InB_to_D can receive data of various precisions as needed, including int9 and int32. This can speed up the conversion of data of different precisions and improve the efficiency of precision conversion.

[0057] In a possible implementation, the data conversion module further includes a rearrangement unit, and the rearrangement unit is used to: concatenate or split the first data of different precisions to obtain the second data.

[0058] In a possible implementation, the reorganization unit can reorganize data of different precisions, including splicing or splitting data of different precisions. For example, in the neural network calculation process, four 2-bit wide data can be directly spliced ​​into 8-bit wide data, or a 32-bit wide data can be directly split into 16 2-bit wide data. By reintegrating the data in this way instead of expanding on the basis of the original data, data of different precisions can be converted more efficiently.

[0059] In a possible implementation, the data conversion module may further include a register group unit. The register group unit may include a combination of multiple registers with different bit widths, such as a 16-bit register and an 8-bit register. Under different calculation modes and storage modes, multiple registers with the same function may be combined into a submodule for repeated use. The registers in the register group unit may include a shift register and a gating register. The shift register may change different types of data bit by bit (i.e., permute the order). For example, some bits of a binary data (consisting of 0 and 1) may be shifted left or right. The gating register may be used to turn on or off some registers in the register group. In addition, the register group may also cache the input data of the data conversion module, so that when the output result of the data conversion module is sent to the multiplication and accumulator array for calculation according to different cycles, the data conversion module may directly read or write the cached data without frequent access to the storage module.

[0060] In a possible implementation, the data conversion module can be used to access the data from the storage module according to the address, convert the data and interact the converted data with the calculation module. Specifically, the data conversion can be a precision conversion of the data. For example, in the process of neural network calculation, it is necessary to process data of various precisions, including 32-bit integer data, 8-bit integer data, and three-value data. Before being sent to the calculation module for calculation, data of different precisions need to be uniformly converted to adapt to the calculation module.

[0061] In a possible implementation, the address generation module can be used to generate a second address of the first calculation result of the first data. The data conversion module is also used to: upon receiving the second calculation result of the calculation module for the second data, convert the second calculation result to obtain the first calculation result; and send the first calculation result to the second address of the storage module.

[0062] In a possible implementation, the control module is respectively connected to the address generation module and the data conversion module, and is used to: determine the corresponding address generation method according to a preset processing category and a data category of the first data, and control the address generation module to generate an address according to the address generation method.

[0063] In a possible implementation, the control module may include a PI (Primitive Instruction) register for controlling the storage module and / or the computing module. A primitive may be a program segment consisting of a number of instructions, which is used to implement a specific function. Specifically, the PI register may be a 32-byte register. The PI register may control the number of groups of multiplier accumulator arrays and the number of multiplier accumulators used in each group. For example, for a multiplier accumulator array divided into 4 groups, each group including 32 multiplier accumulators, a certain storage space may be allocated in the PI register, and only part of the multiplier accumulators work when performing neural network calculations. This can reduce the energy consumption of the neuromorphic chip, shorten the calculation time of the neural network, and thus improve the calculation efficiency of the neural network.

[0064] In a possible implementation, the PI register can be used to control the storage module. For example, the PI register can store instructions that prohibit read and write operations on the storage area of ​​some storage modules, and control the read and write access to some data, so as to avoid misoperation of protected data during neural network calculations, resulting in data loss. The PI register can also be configured with a specific storage space to distinguish different calculation modes, such as convolution, vector accumulation, vector dot multiplication, tensor scaling, etc. For example, when the data processing device performs neural network calculations, certain bits in the PI register can be used to indicate the mode in which the multiplier accumulator array performs convolution operations, and can also indicate the mode in which the multiplier accumulator array performs dot multiplication operations. The PI register can also pre-store parameters such as the step size and convolution kernel size used in the calculation mode, so that the address generation module can directly take out these pre-stored parameters from the PI register for cyclic calculations without repeated reading and writing, and quickly generate addresses of different types of data.

[0065] In a possible implementation, the control module may be used to control the storage module and / or the computing module. For example, the control module may be used to control when to read and write to the storage module, and may also be used to control which multipliers and accumulators of the computing module to work for different computing modes and storage modes. In one example, the control module may include a 128-bit wide register. The computing modes may include neural network computing modes such as convolution, vector accumulation, average pooling, vector dot multiplication, and full connection.

[0066] In one possible implementation, the computing module includes a multiplier-accumulator array, and the control module is also used to control the storage module and / or the computing module; the control module controls the computing module, including: according to a preset processing category, controlling the number of groups of multiplier-accumulators involved in data calculation in the computing module and the number of multiplier-accumulators in each group, so that the computing module performs data calculation.

[0067] By generating a first address of first data to be read according to a set address generation method, then reading the first data from the storage module according to the first address, converting the read first data to obtain converted second data, and then sending the second data to the computing module, and determining the corresponding address generation method according to a preset processing category and data category and controlling the address generation module to generate an address, the present disclosure can overcome the problems of limitation, complexity and inefficiency of traditional neuromorphic chips when processing data of different processing categories and data categories, can reduce the number of memory accesses, improve data acquisition efficiency, and improve data access speed, while improving the computing efficiency of neuromorphic chips and the reasoning efficiency of neural networks, and is easy to expand.

[0068] Figure 3 A schematic diagram showing an address generation module and a data processing process according to an embodiment of the present disclosure.

[0069] In one possible implementation, Figure 3 As shown, the calculation process of the neural network can be mapped to the address generation module. Figure 3In the above, X can be the input image of the convolutional neural network calculation, and the output result Y is obtained by convolution operation with the matrix Y. X includes a pixel point with four rows and four columns, with a total of 16 pixels. The horizontal right direction can be specified as the x direction, and the vertical downward direction can be specified as the y direction. When the data processing device receives an instruction to perform convolutional neural network calculation, the first address generating unit can generate the address of the first pixel point value in the upper left corner of X, and increase the length of a step (L1_step) along the x direction on the basis of this address, so as to perform address transformation (i.e., address jump) and generate the address of the second pixel point. And so on, until the pixel points in the first row are processed, the address transformation starts in the new row, and the address of the first pixel point on the left side of the second row is generated. The step size can be pre-stored in the PI (Primitive Instruction) register of the control module. When the pixel points of X are subjected to address transformation, the number and order of the loop need to be determined. For example, the width of the convolution kernel H is 3 and the height is 4. When performing address transformation, the L1-level loop can process the width of the convolution kernel, that is, it takes 3 loops to read the pixels with a width of 3; the L2-level loop can process the height of the convolution kernel, that is, after reading the pixels of each row with a width of 3, it is necessary to loop 4 times to perform address transformation between rows.

[0070] It is worth noting that the 6-level loop can be executed in different orders, that is, the L2-level loop can be performed after the L1-level loop, and the L5-level loop can be performed after the L1-level loop. The 6-level loop can also be selected by a multiplexer, and the address output by the multiplexer is used as the starting address. After completing the first-level loop, the starting address can be added with a predetermined step size or directly added by 1 as the starting address of the new first-level loop. By adapting the address generation module to different calculation modes and storage methods, the data processing device can flexibly increase the address generation of different types of data to achieve more complex calculations. The present disclosure does not limit how to perform address transformation.

[0071] Figure 4 A schematic diagram showing the data processing process of an embodiment of the present disclosure.

[0072] In one possible implementation, Figure 4 As shown, the data processing device can be used to perform convolutional neural network calculations. Figure 4In the figure, the input feature map X1 of the data processing device is a four-dimensional tensor, which includes Px*Py pixels, a depth of m (m is 4 in the figure), a width of the convolution kernel of Kx, a height of the convolution kernel X2 of Ky, a depth of m, and a total of n such convolution kernels. Each convolution kernel performs a convolution operation with each pixel on the input feature map, and then adds them together to finally obtain an output image X3 with a width of Ox, a height of Oy, and a depth of n.

[0073] In the process of executing the entire convolutional neural network calculation, the address generation submodule can generate the source address of the input tensor, bias vector and weight parameter, quickly read these convolutional neural network calculation parameters from the storage module, send them to the data conversion module for processing, and then send them to the calculation module for calculation. The loop counting submodule with 6 levels of loops can correspond to the 6 steps of the convolutional neural network calculation, and each step can be looped according to a certain step length. The step length can be pre-stored in the PI register of the control module so that when the address generation submodule is used to read and write different neural network parameters and needs to perform address conversion, the address jump can be performed quickly. In the calculation process of the convolutional neural network, data of different precisions can be sent to the data conversion module for precision conversion, and then the converted data is sent to the calculation module for calculation. The output data (i.e., the calculation result) of the calculation module can be written into the storage module, and the written address can also be generated by the address module. In this way, the various parameters of the convolutional neural network can be quickly accessed through the data processing device, thereby improving the calculation efficiency of the convolutional neural network.

[0074] In a possible implementation, the convolutional neural network can have different calculation orders when calculating, and the data processing device can adapt to different calculation orders. For example, when reading the pixel point of X1, the first pixel point in the upper left corner can be read first. Assuming that the horizontal right direction is the x direction, the vertical downward direction is the y direction, and the direction perpendicular to the paper is the z direction, after reading the first pixel point, the second pixel point of the first row can be read along the x direction until all the pixels of the first row are read, and then jump to the first pixel point of the second row along the y direction for reading until all the pixels of the two-dimensional feature map are read. Then all the pixels of the four two-dimensional feature maps are read along the z direction. It is also possible to read the first pixel point of the two-dimensional feature map of each depth along the z direction after reading the first pixel point in the upper left corner, and then read the second pixel point in the x direction until all the pixels of the depth are read, and then read the third pixel point along the x direction. After reading the pixels of this row, read along the y direction until all the pixels are read. In the convolutional neural network calculation, pixels and lines are read in sequence according to the step size. Corresponding to the data processing device, it means that each time a different pixel is read, the corresponding data address must be changed. The address generation module is used to quickly generate and transform the addresses of different types of data. The data processing device can quickly read and write the corresponding data, thereby improving the calculation efficiency of the neural network.

[0075] Figure 5a and Figure 5b A schematic diagram showing the data processing process of an embodiment of the present disclosure.

[0076] In one possible implementation, Figure 5a and Figure 5b The processing process of the data processing device for tensors can be divided into three types, which are respectively shown in (1), (2) and (3) in the figure.

[0077] In one possible implementation, Figure 5bThe first tensor processing process and the second tensor processing process are shown. For the first tensor processing process, two tensors of the same dimension can be dot-multiplied. For example, X1 is a feature map of an input image with a width of Ox, a height of Oy, and a depth of n (n is 4), and X2 is a tensor with the same width, height, and depth. The two tensors are dot-multiplied to obtain an output result X3 with the same width, height, and depth. For the second tensor processing process, a tensor and a scaled vector can be dot-multiplied. For example, X1 is a feature map of an input image with a width of Ox, a height of Oy, and a depth of n (n is 4), and a is a scaled vector, that is, a can be a one-dimensional vector obtained by expanding the first pixel point in the upper left corner of the other tensor (or X1) in the direction perpendicular to the paper (i.e., the depth direction), and the two are dot-multiplied to obtain the output result X2.

[0078] Figure 5a The third tensor processing process is shown. Figure 5a The vector a in can be multidimensional. For example, a can be a multidimensional vector obtained by expanding a row of pixels of the tensor X1 in the direction perpendicular to the paper (i.e., the depth direction), that is, a can be a part of X1. The dot product of the two is used to obtain the output result X2. It is worth noting that, if Figure 5a and Figure 5b As shown in the tensor operations of X1′, X2′ and X3′, for the above three types of tensor calculations, the depth of the tensor for vector dot product can be 1, and the corresponding bias can be added after the dot product to get the output result. Figure 5a and Figure 5b The calculation process of X1′, X2′ and X3′ is similar to the calculation process of X1, X2 and X3. For example, Figure 5a X1′ in the figure can be a feature map of an input image with a width of Ox, a height of Oy, and a depth of 1, and X2′ can be a tensor with the same width, height, and depth as X1′, which will not be repeated here.

[0079] In one possible implementation, for the above three types of tensor calculations, only three levels of loops are needed to process, corresponding to processing the width of the feature map, the height of the feature map, and the depth of the feature map. Figure 5bIn the first tensor process, since X1, X2 and X3 are all of the same width, height and depth, the loop counting submodule only needs to select 3 levels of loops in the 6-level loop to complete the entire operation process, and the remaining 3 levels of loops can be used to process other neural network computing tasks. It should be understood by those skilled in the art that for different computing modes and storage methods, the data processing device can be flexibly used for calculations in accordance with the above method, and the computing logic and storage logic are closely coordinated, so that different types and / or different precision data can be processed more simply and conveniently, thereby improving the computing efficiency of the neuromorphic chip and the reasoning efficiency of the neural network, and having good scalability.

[0080] In a possible implementation, the data processing device can be used for calculations of artificial neural networks, calculations of spiking neural networks, and can also process calculations of hybrid neural networks including artificial neural networks and spiking neural networks.

[0081] The present disclosure also provides an artificial intelligence processor, the artificial intelligence processor includes a plurality of processing cores, each processing core includes a storage module, a computing module and a data processing device, the data processing device is connected to the storage module and the computing module respectively, and the data processing device includes:

[0082] An address generating module, used to: generate a first address of first data to be read according to a set address generating method;

[0083] a data conversion module, connected to the address generation module, and configured to: read the first data from the storage module according to the first address; convert the read first data to obtain converted second data; and send the second data to the calculation module so that the calculation module performs data calculation;

[0084] The control module is connected to the address generation module and the data conversion module respectively, and is used to: determine the corresponding address generation method according to the preset processing category and the data category of the first data, and control the address generation module to generate an address according to the address generation method.

[0085] By generating a first address of the first data to be read according to a set address generation method, then reading the first data from the storage module according to the first address, converting the read first data to obtain converted second data, and then sending the second data to the computing module, and determining the corresponding address generation method according to the preset processing category and data category and controlling the address generation module to generate an address, the data processing device and artificial intelligence processor disclosed in the present invention can overcome the problems of limitation, complexity and inefficiency of traditional neuromorphic chips in processing data of different processing categories and data categories, improve the computing efficiency of neuromorphic chips and the reasoning efficiency of neural networks, and are easy to expand.

[0086] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A data processing device, characterized in that: A processing core applied to an artificial intelligence processor, the artificial intelligence processor comprising a plurality of processing cores, each processing core comprising a storage module, a computing module and a data processing device, the data processing device being connected to the storage module and the computing module respectively, the artificial intelligence processor comprising a neuromorphic chip, the neuromorphic chip using hardware circuits to construct a neural network to simulate the function of the brain; the processing core is a functional core of the neuromorphic chip, different processing cores can process different neural network computing tasks, and the same processing core can perform different neural network computing tasks in different working cycles; the neural network computing tasks include computing tasks of artificial neural networks, computing tasks of spiking neural networks, and computing tasks of hybrid neural networks of artificial neural networks and spiking neural networks; The data processing device is used to uniformly schedule data of different types and / or different precisions, and centralize the data of different types and / or different precisions to the data processing device for processing, wherein the data of different types and / or different precisions are exchanged between processing cores and within the processor core; The data processing device comprises: An address generating module, used to: generate a first address of first data to be read according to a set address generating method; a data conversion module, connected to the address generation module, and configured to: read the first data from the storage module according to the first address; convert the read first data to obtain converted second data; send the second data to the calculation module so that the calculation module performs data calculation, and upon receiving a second calculation result of the calculation module for the second data, convert the second calculation result to obtain a first calculation result; and send the first calculation result to the second address of the storage module; a control module, connected to the address generation module and the data conversion module respectively, for dynamically determining a corresponding address generation method according to a preset processing category and a data category of the first data, and controlling the address generation module to generate an address according to the address generation method; The data conversion module includes a precision conversion unit, which is used to: perform precision conversion on the first data according to the data precision adapted to the calculation module to obtain the second data, and perform precision conversion on the second calculation result according to the data precision adapted to the storage module to obtain the first calculation result; The calculation module includes a multiplier-accumulator array, and a data line for receiving specific precision data and a data line for receiving multiple precision data are arranged between the data conversion module and the multiplier-accumulator array; The data category includes: at least one of: vector, matrix, tensor, and spatiotemporal information encoding of a pulse signal; the processing category includes: at least one of: convolution, vector dot multiplication, tensor dot multiplication, vector accumulation, average pooling, data loading, data caching, data shifting, data reordering, and data reorganization; The address generation module comprises: An address generation submodule, the address generation submodule comprising a plurality of address generation units, each used to generate a start address in a first address of first data of various categories; the address generation unit is switched according to different processing categories and data categories of the first data to perform different functions, and the address generated by the address generation unit is selected and outputted through a multiplexer; The address generation module comprises: A loop calculation submodule, wherein the loop calculation submodule includes a multi-level loop counter, which is used to cyclically calculate the first address of the first data to be read next time according to the dimension of the first data of various categories; for different steps in the neural network calculation, each level of loop in the multi-level loop counter corresponds to a different dimension; in response to the first data being a tensor, loops of some levels in the multi-level loop counter are selected, and loops of the remaining levels are used to process other neural network calculation tasks.

2. The device according to claim 1, characterized in that The address generating module is further used to: generate a second address of a first calculation result of the first data; The data conversion module is also used for: upon receiving a second calculation result of the calculation module for the second data, converting the second calculation result to obtain the first calculation result; and sending the first calculation result to the second address of the storage module.

3. The device according to claim 1, characterized in that The first data includes multiple categories, and the address generation module includes: An address generating submodule, used for respectively generating first addresses of various types of first data; The cyclic calculation submodule is used to cyclically calculate the first address of the first data to be read next time according to the first address of the first data to be read currently and the address generation method.

4. The device according to claim 2, characterized in that The data category of the first data includes: at least one of input image data, parameter weight data, and bias data; The data category of the first calculation result includes a membrane potential value.

5. The device according to claim 2, characterized in that The precision conversion unit is further used to: perform precision conversion on the second calculation result according to data precision suitable for the storage module to obtain the first calculation result.

6. The device according to claim 4, characterized in that The data conversion module also includes a reorganization unit, which is used to: concatenate or split the first data with different precisions to obtain the second data.

7. The device according to claim 1, characterized in that The control module is also used to control the storage module and / or the calculation module; The control module controls the calculation module, including: According to the preset processing category, the number of groups of multiplier-accumulators involved in data calculation in the calculation module and the number of each group of multiplier-accumulators are controlled so that the calculation module performs data calculation.

8. An artificial intelligence processor, characterized in that: The artificial intelligence processor includes a plurality of processing cores, each of which includes a storage module, a computing module and a data processing device, the data processing device being connected to the storage module and the computing module respectively, the artificial intelligence processor includes a neuromorphic chip, the neuromorphic chip uses a hardware circuit to construct a neural network to simulate the function of the brain; the processing core is a functional core of the neuromorphic chip, different processing cores can process different neural network computing tasks, and the same processing core can perform different neural network computing tasks in different working cycles; the neural network computing tasks include artificial neural network computing tasks, pulse neural network computing tasks, and hybrid neural network computing tasks of artificial neural network and pulse neural network; The data processing device is used to uniformly schedule data of different types and / or different precisions, and centralize the data of different types and / or different precisions to the data processing device for processing, wherein the data of different types and / or different precisions are exchanged between processing cores and within the processor core; The data processing device comprises: An address generating module, used to: generate a first address of first data to be read according to a set address generating method; a data conversion module, connected to the address generation module, and configured to: read the first data from the storage module according to the first address; convert the read first data to obtain converted second data; send the second data to the calculation module so that the calculation module performs data calculation, and upon receiving a second calculation result of the calculation module for the second data, convert the second calculation result to obtain a first calculation result; and send the first calculation result to the second address of the storage module; a control module, connected to the address generation module and the data conversion module respectively, for dynamically determining a corresponding address generation method according to a preset processing category and a data category of the first data, and controlling the address generation module to generate an address according to the address generation method; The data conversion module includes a precision conversion unit, which is used to: perform precision conversion on the first data according to the data precision adapted to the calculation module to obtain the second data, and perform precision conversion on the second calculation result according to the data precision adapted to the storage module to obtain the first calculation result; The calculation module includes a multiplier-accumulator array, and a data line for receiving specific precision data and a data line for receiving multiple precision data are arranged between the data conversion module and the multiplier-accumulator array; The data category includes: at least one of: vector, matrix, tensor, and spatiotemporal information encoding of a pulse signal; the processing category includes: at least one of: convolution, vector dot multiplication, tensor dot multiplication, vector accumulation, average pooling, data loading, data caching, data shifting, data reordering, and data reorganization; The address generation module comprises: An address generation submodule, the address generation submodule comprising a plurality of address generation units, each used to generate a start address in a first address of first data of various categories; the address generation unit is switched according to different processing categories and data categories of the first data to perform different functions, and the address generated by the address generation unit is selected and outputted through a multiplexer; The address generation module comprises: A loop calculation submodule, wherein the loop calculation submodule includes a multi-level loop counter, which is used to cyclically calculate the first address of the first data to be read next time according to the dimension of the first data of various categories; for different steps in the neural network calculation, each level of loop in the multi-level loop counter corresponds to a different dimension; in response to the first data being a tensor, loops of some levels in the multi-level loop counter are selected, and loops of the remaining levels are used to process other neural network calculation tasks.

Images