Data processing device and artificial intelligence processor

By designing data processing devices in the processing core of the neuromorphic chip, data integration is realized, data integration problems are solved, the efficiency and flexibility of neural network computing are improved, and data processing of different precisions and types are supported.

CN113703837BActive Publication Date: 2024-11-05TSINGHUA UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110995714.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-27
Publication Date
2024-11-05
Estimated Expiration
2041-08-27

AI Technical Summary

Technical Problem

In the processing core of neuromorphic chips, how to effectively achieve data integration has become an urgent problem to be solved.

Method used

A data processing device is designed, including an address generation module and a data conversion module. By generating input and output addresses, data merging, splitting, migrating and type conversion operations are performed to ensure that the data in the storage module can be efficiently sorted and processed.

Benefits of technology

It realizes a variety of operations on the data in the storage module, including data merging, splitting, migration and type conversion, improves the efficiency and flexibility of neural network computing, supports data processing of different precisions and types, and improves the computing efficiency and Scalability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113703837B_ABST
    Figure CN113703837B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a data processing device and an artificial intelligence processor. The data processing device is applied to the processing core of an artificial intelligence processor, and the artificial intelligence processor includes multiple processing cores, each processing core includes a storage module and a data processing device, and the data processing device is connected to the storage module. The data processing device includes: an address generation module, which is used to generate an input address and an output address according to a control instruction; a data conversion module, which is connected to the address generation module, and is used to read the first data from the storage module according to the input address, perform a data conversion operation on the first data, obtain the second data, and write the second data to the output address of the storage module. The data processing device of the embodiment of the present disclosure can perform data integration on the storage data in the storage module.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing device and an artificial intelligence processor. Background Art

[0002] In recent years, the field of neuromorphic computing has developed rapidly. Neural networks can be constructed using hardware circuits to simulate the functions of the brain. For example, neuromorphic chips can be used to build a large-scale, parallel, low-energy computing platform that can support complex pattern learning.

[0003] In related technologies, neuromorphic chips include multiple processing cores, and how to achieve data integration in the processing cores has become a problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the present disclosure proposes a data processing device and an artificial intelligence processor.

[0005] According to one aspect of the present disclosure, a data processing device is provided, which is applied to a processing core of an artificial intelligence processor, wherein the artificial intelligence processor includes a plurality of processing cores, each processing core includes a storage module and a data processing device, and the data processing device is connected to the storage module, and the data processing device includes:

[0006] An address generation module, used to generate input addresses and output addresses according to control instructions;

[0007] A data conversion module, connected to the address generation module, is used to read first data from the storage module according to the input address, perform a data conversion operation on the first data to obtain second data, and write the second data to the output address of the storage module, wherein the data conversion operation includes at least one of data merging, data splitting, data migration and data type conversion operations.

[0008] In a possible implementation, the data conversion operation includes a data merging operation.

[0009] The address generating module is used to generate a plurality of input addresses and the output address according to the control instruction;

[0010] The data conversion module is used to read a plurality of first data from the storage module according to a plurality of input addresses, and write the second data generated by merging the plurality of first data into the output address of the storage module.

[0011] In a possible implementation, the data conversion operation includes a data splitting operation.

[0012] The address generating module is used to generate the input address and multiple output addresses according to the control instruction;

[0013] The data conversion module is used to read the first data from the storage module according to the input address, and write the multiple second data obtained by splitting the first data into the multiple output addresses of the storage module respectively.

[0014] In a possible implementation, the data conversion operation includes a data migration operation.

[0015] The address generating module is used to generate the input address and the output address according to the control instruction;

[0016] The data conversion module is used to read the first data from the storage module according to the input address, and write the second data determined according to the first data into the output address of the storage module.

[0017] In a possible implementation, the data conversion operation includes a data type conversion operation.

[0018] The data conversion module is used to perform a data type conversion operation on the first data to obtain the second data when the data type of the first data is different from the data type of the second data.

[0019] The data type includes any one of a 32-bit integer int32, an 8-bit integer int8, and a three-value data type.

[0020] In a possible implementation, the device further includes a row counting module.

[0021] The row counting module is used to count according to the counting pulses sent by the control module of the processing core, and send the control instruction to the address generating module when the counted number is greater than or equal to the row number threshold.

[0022] In a possible implementation, the control instruction includes a primitive instruction sent by a control module of the processing core.

[0023] In a possible implementation, the address generation module includes multiple groups of counters, which are respectively used to generate the input address and the output address, and control the data conversion module to perform the data conversion operation through counter logic.

[0024] In a possible implementation, the address generation module includes a first group of counters, a second group of counters, and a third group of counters.

[0025] The first group of counters is used to generate a first address, where the first address is the address of the input data;

[0026] The second group of counters is used to generate a second address;

[0027] The third group of counters is used to generate a third address, where the third address is the address of the output data.

[0028] When performing a data merging operation, the second address is the input address; when performing a data splitting operation, the second address is the output address; when performing a data migration operation, the generation operation of the second address is skipped.

[0029] In a possible implementation, the output address includes an address of a data sending area of ​​a routing module located in the processing core.

[0030] The data conversion module is used to write the second data into the sending data area of ​​the routing module, so that the routing module sends the second data.

[0031] In a possible implementation, the data type includes at least one of a 32-bit integer int32, an 8-bit integer int8, and a ternary data type.

[0032] According to another aspect of the present disclosure, an artificial intelligence processor is provided, which includes the above-mentioned processing core.

[0033] According to an embodiment of the present disclosure, an address generation module generates an input address and an output address according to a control instruction; a data conversion module reads first data from the storage module according to the input address, performs a data conversion operation on the first data to obtain second data, and writes the second data to the output address of the storage module, thereby realizing data integration processing of the storage data in the storage module.

[0034] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0036] Figure 1 A block diagram of an artificial intelligence processor according to an embodiment of the present disclosure is shown.

[0037] Figure 2 A block diagram showing a data processing device according to an embodiment of the present disclosure.

[0038] Figure 3 A schematic diagram showing the storage of feature graph data in a storage module according to an embodiment of the present disclosure.

[0039] Figure 4 A schematic diagram showing a data processing device according to an embodiment of the present disclosure.

[0040] Figure 5 A schematic diagram showing a data conversion operation according to an embodiment of the present disclosure.

[0041] Figure 6 A schematic diagram showing a data merging operation according to an embodiment of the present disclosure.

[0042] Figure 7 A schematic diagram showing how a plurality of first data are merged into second data according to an embodiment of the present disclosure.

[0043] Figure 8 A schematic diagram showing a data splitting operation according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0045] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0046] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0047] A neuromorphic chip is a new type of chip that can simulate the working mechanism of the human brain. The neuromorphic chip can be organized according to processing cores. For example, a neuromorphic chip can include 4096 (e.g., a 64×64 array) processing cores, each of which can simulate 256 biological neurons. In this way, a neuromorphic chip can simulate about 1 million neurons in total.

[0048] In a possible implementation, the processing core can organize neurons according to a two-dimensional mesh structure. Among them, a neuron can include an axon, a dendrite, and a cell body, etc., and the axon, dendrite, and cell body of the neuron can all be implemented by hardware circuits. The axon can be a row of a two-dimensional mesh structure, the dendrite can be a column of a two-dimensional mesh structure, and the synapse can be a cross point of the row and column, which is used to store weights. Among them, the axon can be used to send data, the dendrite can be used to receive data, and the synapse can be used to store weights. The weight can be the weighted value of the latter neuron to the previous neuron before the synapse, corresponding to the weight parameter of the neural network. For example, when a certain neuron is not used, the weight of the neuron can be set to 0, and when a certain neuron is needed for calculation, a weight value can be assigned to the neuron. The cell body can be used to implement the data sorting in the storage module. The data sorting method includes: data migration Move, data merging Merge, and data splitting Split operations. The cell body can also plan the corresponding path to route the data and finally reach the destination processing core.

[0049] Figure 1 A block diagram of an artificial intelligence processor according to an embodiment of the present disclosure is shown. As shown in Figure 1, the artificial intelligence processor 100 includes multiple processing cores 101, and each processing core 101 may include a storage module, a computing module, a data processing device, etc.

[0050] Among them, the storage module can be used for various types of data, for example, to store pixel data of an image and weight data of a convolution kernel. The storage module can include a separate storage module or a storage module located inside other modules, for example, a storage module located inside a routing module; the operation module can include a multiplier accumulator MAC array for performing operations based on pixel data and weight data. The data processing device can be used to organize the storage data of the storage module, for example, data migration, data merging, data comparison and other operations.

[0051] The present disclosure provides a data processing device, which can be used to organize the storage data of a storage module.

[0052] Figure 2 A block diagram of a data processing device according to an embodiment of the present disclosure is shown. Figure 2 As shown, the data processing device 60 can be applied to the processing core of an artificial intelligence processor, the artificial intelligence processor includes multiple processing cores, each processing core includes a storage module 40, a control module 50 and a data processing device 60, the data processing device is connected to the storage module and the control module, and the data processing device includes:

[0053] An address generation module 10 is used to generate an input address and an output address according to a control instruction;

[0054] A data conversion module 20 is connected to the address generation module, and is used to read the first data from the storage module according to the input address, perform a data conversion operation on the first data to obtain second data, and write the second data to the output address of the storage module, wherein the data conversion operation includes at least one of data merging, data splitting, data migration and data type conversion operations.

[0055] For example, the data conversion operation may be a merging operation on multiple first data, a splitting operation on the first data, a migration operation on the first data, etc., wherein, during the above operations, a data type conversion operation may also be performed, for example, the data type of the first data or the data type of the intermediate conversion data which is a 32-bit integer int32 is converted to an 8-bit integer int8. The first data may be one or more, and the second data may be one or more. The present disclosure does not limit the type of data conversion operation, the number of the first data and the second data.

[0056] The data processing device of the embodiment of the present disclosure can organize the storage data of the storage module.

[0057] The storage module of the processing core can be used to store data and instructions related to neural network calculations. In one example, the storage module can be a memory with a certain storage capacity, storing different types of data such as vectors, matrices and tensors in neural network calculations.

[0058] In a possible implementation, the storage module stores feature map data, wherein the feature map data is stored in vector units, and each vector contains data in the channel direction, which can be called the depth of the vector. The channel data can be aligned according to the 16-Byte width, and the number of units occupied by the storage module can be called the length of the unit vector. The feature map data can be composed of the number of vectors and the length of each vector.

[0059] Figure 3 FIG. 2 is a schematic diagram showing the storage of feature graph data in a storage module according to an embodiment of the present disclosure. Figure 3 As shown in , the feature map data is width × height × channel, where the width and height are 3 for example, and the number of vectors included in the feature map is 9. The data in the channel direction of each vector is aligned according to the width of 16Byte, and the length of the vector is 3. Figure 3 As shown, the storage order of the feature map data in the memory can be: stored in sequence according to the channel direction, the row direction of the vector in the feature map data, and the column direction of the vector in the feature map data.

[0060] It should be understood that the storage order of the feature graph data in the storage module may include multiple forms, and the present disclosure does not limit the data type processed by the data processing device and the storage form of the data in the storage module. For ease of understanding, the present disclosure takes the data conversion operation performed on the feature graph data in the form of a vector stored in the storage module as an example for explanation.

[0061] In a possible implementation, the data conversion operation includes at least one of data merging, data splitting, data migration, and data type conversion operations.

[0062] In a possible implementation, the data conversion operation includes a data merging operation.

[0063] The address generating module is used to generate a plurality of input addresses and the output address according to the control instruction;

[0064] The data conversion module is used to read a plurality of first data from the storage module according to a plurality of input addresses, and write the second data generated by merging the plurality of first data into the output address of the storage module.

[0065] In a possible implementation, the data conversion operation includes a data splitting operation.

[0066] The address generating module is used to generate the input address and multiple output addresses according to the control instruction;

[0067] The data conversion module is used to read the first data from the storage module according to the input address, and write the multiple second data obtained by splitting the first data into the multiple output addresses of the storage module respectively.

[0068] In a possible implementation, the data conversion operation includes a data migration operation.

[0069] The address generating module is used to generate the input address and the output address according to the control instruction;

[0070] The data conversion module is used to read the first data from the storage module according to the input address, and write the second data determined according to the first data into the output address of the storage module.

[0071] In a possible implementation, the data conversion operation includes a data type conversion operation.

[0072] The data conversion module is used to perform a data type conversion operation on the first data to obtain the second data when the data type of the first data is different from the data type of the second data.

[0073] The data type includes any one of a 32-bit integer int32, an 8-bit integer int8, and a three-value data type.

[0074] For example, a data merge operation can be understood as generating second data based on a merge of multiple first data, for example, the merge of two vectors in the depth direction can be realized. A data split operation can be understood as splitting the first data into multiple second data, for example, the splitting of the vector in the depth direction can be realized. A data migration operation can be understood as converting the first data corresponding to the input address in the storage module into the second data, and writing the second data to the output address of the storage module, wherein the conversion of the first data into the second data can be determining the first data as the second data, or performing a data type conversion on the first data to obtain the converted second data. In this way, a vector loop migration operation of the storage space can be realized.

[0075] The data type conversion operation may refer to converting a data type. For example, if the target data type of the second data to be written into the memory is int8 and the data type of the first data is int32, the data type may be converted from int32 to int8, etc. When performing a data conversion operation, at least one of data merging, data splitting, data migration, and data type conversion operations may be included. For example, data merging and data splitting operations may be included at the same time, and data merging and data type conversion operations may also be included at the same time. The present disclosure does not limit this.

[0076] In this way, various operations such as data merging, data splitting, data migration, data type conversion, etc. can be performed on the data in the storage module, so that the artificial intelligence processor can realize various data transformations in the neural network, such as partial sums, image frame extraction, data space migration, and storage space clearing operations.

[0077] The control instruction can be used to determine the type of data conversion operation and the address generation method corresponding to the data conversion operation. For example, different data conversion operations may correspond to different numbers of input addresses and output addresses. The control instruction can also be used to determine the address start and end positions of the input address and the output address, or the storage area of ​​the input address and the output address.

[0078] For example, when the data conversion operation is a data merge operation, the input address may include a first address and a second address, and the output address may include a third address. When the data conversion operation is a data split operation, the input address may include a first address, and the output address may include a second address and a third address. When the data conversion operation is a data migration operation, the generation of the second address may be skipped, the input address may include the first address, and the output address may include the third address.

[0079] It should be understood that when the data conversion operation is a data merging operation, the input address may include more than two addresses, not limited to the first address and the second address. When the data conversion operation is a data splitting operation, the output address may also include more than two addresses. The present disclosure does not impose any restrictions on the number of input addresses and the number of output addresses generated in the data conversion operation.

[0080] For ease of understanding, the following examples are used to illustrate when the data conversion operation is a data merge operation, the input address includes the first address and the second address, and the output address includes the third address. When the data conversion operation is a data split operation, the input address includes the first address, and the output address includes the second address and the third address. When the data conversion operation is a data migration operation, the input address includes the first address, and the output address includes the third address.

[0081] The control instruction may be from a control module connected to the data processing device, or may be a control instruction generated by the data processing device when a preset condition is met, and the present disclosure does not impose any limitation on this.

[0082] In some optional embodiments, the control instruction may be a primitive instruction sent by the control module, for example, may be a primitive instruction from a primitive instruction (PI, Primitive Instructions) register in the control module.

[0083] In some optional embodiments, the control instruction may be a control instruction generated by a row counting module in a data processing device.

[0084] Figure 4 A schematic diagram of a data processing device according to an embodiment of the present disclosure is shown. Figure 4 As shown, in some optional embodiments, the data processing device further includes a row counting module,

[0085] The row counting module is used to count according to the counting pulses sent by the control module, and send the control instruction to the address generating module when the counted times is greater than or equal to the row number threshold.

[0086] Among them, Figure 4 As shown, the row counting module may be a row counter.

[0087] Figure 5 Schematic diagram showing the data conversion operation of the embodiment of the present disclosure. Figure 5 As shown, the primitive instruction register of the control module sends a counting pulse to the row counting module. The row counting module counts according to the received counting pulses, and when the count times are greater than the row number threshold, sends a start control instruction to the address generation module. The address generation module generates an input address and an output address in response to the control instruction.

[0088] The row counting module can be used to complete the row pipeline operation of data conversion. The row pipeline operation can be understood as follows: each time the input vector storage module receives a row of feature map data, the primitive instruction register sends a counting pulse. Each time the row counting module receives a counting pulse, the counter will be triggered to count. When the number of rows in the counter is greater than or equal to the preset row number threshold, the row counting module sends a control signal to start the address generation module, and the address generation module can run the data conversion operation according to the control signal.

[0089] In a possible implementation, the address generation module includes multiple groups of counters, which are respectively used to generate input addresses and output addresses, and the counters are used to control the data conversion module to complete the data conversion operation. The number of vectors and the length of the vectors can be used as counter cycles to generate input addresses and output addresses.

[0090] For example, the address generation module of the embodiment of the present disclosure may include three groups of counters: a first group of counters for generating a first address Addr_in, a second group of counters for generating a second address Addr_ciso, and a third group of counters for generating a third address Addr_out.

[0091] Among them, in various data conversion operations, the first address Addr_in can be used as the input address, and the third address Addr_out can be used for the output address. The second address Addr_ciso can be used as the input address or the output address according to the type of data transfer operation. For example, when performing a data merge operation, the second address Addr_ciso can be used as the input address; when performing a data split operation, the second address Addr_ciso can be used as the output address. When performing a data migration operation, the generation of the second address Addr_ciso can be skipped.

[0092] In this way, the three groups of counters of the address generation module facilitate the sorting of various types of data in the storage module. It should be understood that the embodiment of the present disclosure does not limit the form of the address generation module, and as long as at least one data conversion operation among data merging, data splitting, data migration and data type conversion can be implemented, the present disclosure does not limit the number of counters of the address generation module.

[0093] In a possible implementation, the data conversion module is further configured to perform a data type conversion operation during the data conversion operation when the data type of the first data is different from the data type of the second data to be output.

[0094] The data conversion module can more efficiently handle data merging, data splitting, and data migration operations with data type conversion.

[0095] In some optional embodiments, the data conversion module is used to perform data type conversion on the first data according to the data type of the second data, and perform a data conversion operation on the third data after the data type conversion to obtain the second data.

[0096] For example, the data conversion module can convert the first data into third data of the same type as the second data to be output according to the data type conversion rule, and perform operations such as data merging, data splitting or data migration on the third data to obtain the second data.

[0097] In some optional embodiments, the data conversion module is further used to perform a data conversion operation on the first data to obtain third data, and convert the data type of the third data into the type of the second data, which is not limited in the present disclosure.

[0098] In a possible implementation, the data conversion module can be used for conversion operations between three data types: int32, int8, and ternary data. For example, int32 data can be converted into ternary data, int8 data can be converted into ternary data, and int32 data can be converted into int8 data.

[0099] Among them, the output pulse sequence of the neuron in the pulse neural network can be encoded according to three-value data. For example, (0, 1 and -1) can be used to describe that the neurons of the pulse neural network are in the state of no action, activation and inhibition respectively, and the three-value data can also have different types. It should be understood that the data conversion module can also convert the data into all-0 data, etc., and the present disclosure does not limit the conversion type of the data conversion module.

[0100] like Figure 4 As shown, in a possible implementation, the data conversion module may include multiple paths and a data multiplexer. The multiple paths may be respectively int32 to three-value path, int8 to three-value path, int32 to int8 path, straight-through path, and 16B "0" path.

[0101] In this way, the data conversion module can realize support for three data type conversions when the data types of the input data and the output data are different. For example, when the data types of the input data and the output data are the same, a straight-through path can be selected, and when performing a clear operation for the output address, a 16B "0" path can be selected, so that the second data written to the output address is all 0 data. The present disclosure does not limit the number of paths and path types included in the data conversion module.

[0102] To facilitate understanding, the following text will take the three groups of counters in the address generation module, which can generate the first address, the second address and the third address respectively according to the control instructions, as an example to illustrate how the data processing device of the present embodiment performs data merging operations, data splitting operations and data migration operations when the data types of the input data and the output data are the same or the data types of the input data and the output data are different.

[0103] In a possible implementation, the data conversion operation is a data merging operation, the input address includes a first address and a second address, the output address includes a third address,

[0104] The data conversion module 20 is used to read a plurality of first data from the storage module according to the first address and the second address, and write the second data generated by merging the plurality of first data into the third address of the storage module.

[0105] When the data types of the first data and the second data are the same, as described above, the data selector of the data conversion module can select a through path and perform a merging operation on the plurality of first data.

[0106] For example, the first data corresponding to the first address and the first data corresponding to the second address can be understood as input data, and the data after the first data are combined can be understood as output data. For example, the first data corresponding to the first address is in, the first data corresponding to the second address is ciso, and the combined second data is out.

[0107] The first data in and the first data ciso may be input vectors containing the same number of vectors and the same vector data type. The vector lengths of the first data in and the first data ciso may be different.

[0108] In one possible implementation, the first data corresponding to the first address and the first data corresponding to the second address may be concatenated into an output vector, wherein, in the concatenated second data, the vector elements of the first data corresponding to the first address are located before the vector elements of the first data corresponding to the second address.

[0109] For example, in the data merging process, the vector of in can be integrated into the output vector, and then the ciso vector can be integrated. For example, in, ciso and out are three sets of vectors, and the smallest unit in the vector set is the vector. The mathematical representation can be as follows:

[0110] in={in00,in01,in02}; ciso={ciso00,ciso01,ciso02};

[0111] The second data obtained by performing a merging operation on multiple first data may be:

[0112] out={out0,out1,out2}

[0113] Where: out0 = {in00, ciso00}, out1 = {in01, ciso01}, out2 = {in02, ciso02};

[0114] Among them, the second data out={{in00,ciso00},{in01,ciso11},{in02,ciso12}}.

[0115] It should be noted that the above mathematical expressions are exemplary and the present disclosure does not limit the merging method.

[0116] In some optional embodiments, the data conversion module 20 is used to perform a zero-fill operation on the remaining storage space when the number of vectors of the second data is greater than the sum of the number of vectors of the multiple first data, and / or the vector length of the second data is greater than the sum of the vector lengths of the multiple first data.

[0117] Figure 6 The schematic diagram of the data merging operation of the embodiment of the present disclosure is shown. The number of the first data in is: num_in, the vector length is: Km_num_in; the number of the first data ciso is: num_ciso, the vector length is: Km_num_ciso; the number of the second data out is: num_out, the vector length is: Km_num_out.

[0118] like Figure 6 As shown, when the data merging operation is performed, the total vector length of the second data out is 15, and the sum of the total vector lengths of the first data in and the first data ciso is 12. The 0-filling operation can be performed on the remaining storage space, and the second data out is generated according to Y0 obtained by merging X0 of the first data in and X0 of the first data ciso, Y1 obtained by merging X1 of the first data in and X1 of the first data ciso, and X2 of the first data in and Y2 obtained by 0-filling, and written to the third address.

[0119] In some optional embodiments, the data types of the first data and the second data are different.

[0120] For example, different truncation bits can be set to achieve precision setting. For example, data type conversion can be achieved by setting the parameter in_cut_start bit. It should be understood that during the data type conversion process, the vector length will change. For example, when data with a data type of int32 and a vector length of 4 is converted to a vector with a data type of int8, the vector length will change from 4 to 1. Among them, multiple continuous bits intercepted forward or backward from the target bit of the first data of the data type int32 can be determined as the first data of int8, or multiple bits of data can be randomly selected and determined as the first data of int8. The present disclosure does not limit this.

[0121] Figure 7 A schematic diagram showing how a plurality of first data are merged into second data according to an embodiment of the present disclosure.

[0122] like Figure 7 As shown, X0 of the first data in is int32 type data, which can be converted into an int8 vector, and the vector length will change from 4 to 1. Similarly, X0 of the first data ciso is converted into an int8 vector, and the vector length will change from 4 to 1, and the vectors with vector lengths of 1 are merged to obtain Y0 in the second data out. As mentioned above, the 0-padding operation can be performed during the data conversion process, which will not be repeated here.

[0123] In a possible implementation, the data conversion operation is a data splitting operation, the input address includes a first address, the output address includes a second address and a third address,

[0124] The data conversion module 20 is used to read the first data from the storage module according to the first address, and write a plurality of second data obtained by splitting the first data into the second address and the third address of the storage module respectively.

[0125] As mentioned above, the first data in represents input data, the second data out represents first output data, and the first data ciso represents second output data. When splitting the data, the splitting can be performed in the order of the second data out first and then the first data ciso. If the number of vectors of the first data in is less than the sum of the number of vectors of the second data out and the first data ciso, and / or the vector length of the first data in is less than the sum of the vector lengths of the second data out and the first data ciso, it can be preferentially split into the second data out.

[0126] For example, in, ciso and out are three sets of vectors, and the smallest unit in the vector set is the vector length. The mathematical representation can be expressed as follows:

[0127] in={in00,in01,in02};

[0128] The vector splitting result can be expressed as:

[0129] out={out0, out1, out2}; ciso={ciso00, ciso01, ciso02};

[0130] Where: in00 = {out0, ciso00}, in01 = {out1, ciso01}, in02 = {out2, ciso02}

[0131] It should be noted that the above mathematical expressions are exemplary, and the present disclosure does not limit the manner and form of data splitting. As mentioned above, when it is determined that there is remaining storage space, a zero-filling operation can be performed, which will not be repeated here.

[0132] Figure 8 A schematic diagram showing a data splitting operation according to an embodiment of the present disclosure.

[0133] like Figure 8 As shown, during data splitting, a data splitting operation is performed on the first data in according to the vector quantity and vector length of the second data out and the vector length and vector data of the first data ciso0. Figure 8 As shown, the vector length of the second data out is 2, and the vector length of the first data ciso0 is 1. X0 of the first data in can be split into out0 (vector length is 2) and ciso0 (vector length is 1), and out0 is written to the third address, and ciso0 is written to the second address.

[0134] Among them, when the data types of the second data and the first data are different, the data type conversion operation of performing the data splitting operation can refer to the data type conversion operation of performing the data merging operation, and will not be repeated here.

[0135] In a possible implementation, the data conversion operation is a data migration operation, the input address includes a first address, the output address includes a third address,

[0136] The data conversion module 20 is used to read first data from the storage module according to the first address, and write second data determined according to the first data into a third address of the storage module.

[0137] The first data may be determined as the second data, or data obtained by performing data type conversion on the first data may be determined as the second data.

[0138] For example, when the data conversion operation is a data migration operation (for example, the number or length of the second address in the control instruction is 0), the address generation module skips the generation of the second address and generates the first address and the third address.

[0139] In this way, direct data migration operation can be implemented, and data migration operation with data type conversion can also be implemented. The first data is converted into a data type through a data conversion module and written into a third address as the second data to implement the data migration operation.

[0140] In one possible implementation, the data processing device, storage module and control module may be located in the processing core of an artificial intelligence processor. The artificial processor may be a neuromorphic chip, and the processing core may be a functional core of the neuromorphic chip. The processing core may also include other modules, such as a computing module, a routing module, etc.

[0141] In a possible implementation, the output address includes an address of a data sending area of ​​a routing module located in the processing core.

[0142] The data conversion module is used to write the second data into the sending data area of ​​the routing module, so that the routing module sends the second data.

[0143] In this way, the routing module can be started directly to send data.

[0144] In a possible implementation, the data processing device can be used for calculations of artificial neural networks, calculations of spiking neural networks, and can also process calculations of hybrid neural networks including artificial neural networks and spiking neural networks.

[0145] The present disclosure further provides a processing core, the processing core comprising the control module, the storage module and the data processing device, the data processing device comprising:

[0146] An address generation module, used to generate input addresses and output addresses according to control instructions;

[0147] A data conversion module is connected to the address generation module, and is used to read first data from the storage module according to the input address, perform a data conversion operation on the first data to obtain second data, and write the second data into the output address of the storage module.

[0148] In a possible implementation, the processing core further includes a routing module, the routing module includes a sending data area, and the routing module is used to send second data in the sending data area, wherein the second data may be second data written by the data processing module.

[0149] In a possible implementation, the control module may include a PI (Primitive Instruction) register for controlling the storage module and / or the computing module. A primitive may be a program segment consisting of a number of instructions, which is used to implement a specific function. Specifically, the PI register may be a 32-byte register. The PI register may control the number of groups of multiplier accumulator arrays and the number of multiplier accumulators used in each group. For example, for a multiplier accumulator array divided into 4 groups, each group including 32 multiplier accumulators, a certain storage space may be allocated in the PI register, and only part of the multiplier accumulators work when performing neural network calculations. This can reduce the energy consumption of the neuromorphic chip, shorten the calculation time of the neural network, and thus improve the calculation efficiency of the neural network.

[0150] In a possible implementation, the PI register can be used to control the storage module. For example, the PI register can store instructions that prohibit read and write operations on the storage area of ​​some storage modules, and control the read and write access to some data, so as to avoid misoperation of protected data during neural network calculations, resulting in data loss. The PI register can also be configured with a specific storage space to distinguish different calculation modes, such as convolution, vector accumulation, vector dot multiplication, tensor scaling, etc. For example, when the data processing device performs neural network calculations, certain bits in the PI register can be used to indicate the mode in which the multiplier accumulator array performs convolution operations, and can also indicate the mode in which the multiplier accumulator array performs dot multiplication operations. The PI register can also pre-store parameters such as the step size and convolution kernel size used in the calculation mode, so that the address generation module can directly take out these pre-stored parameters from the PI register for cyclic calculations without repeated reading and writing, and quickly generate addresses of different types of data.

[0151] In a possible implementation, the control module may be used to control the storage module and / or the computing module. For example, the control module may be used to control when to read and write to the storage module, and may also be used to control which multipliers and accumulators of the computing module to work for different computing modes and storage modes. In one example, the control module may include a 128-bit wide register. The computing modes may include neural network computing modes such as convolution, vector accumulation, average pooling, vector dot multiplication, and full connection.

[0152] In one possible implementation, the computing module includes a multiplier-accumulator array, and the control module is also used to control the storage module and / or the computing module; the control module controls the computing module, including: according to a preset processing category, controlling the number of groups of multiplier-accumulators involved in data calculation in the computing module and the number of multiplier-accumulators in each group, so that the computing module performs data calculation.

[0153] The present disclosure also provides an artificial intelligence processor, the artificial intelligence processor includes a plurality of processing cores, each processing core includes a storage module, a control module and a data processing device, the data processing device is connected to the storage module and the control module respectively, and the data processing device includes:

[0154] An address generation module, used to generate input addresses and output addresses according to control instructions;

[0155] A data conversion module is connected to the address generation module, and is used to read first data from the storage module according to the input address, perform a data conversion operation on the first data to obtain second data, and write the second data into the output address of the storage module.

[0156] In a possible implementation, the processing core may further include a storage module and a computing module, that is, the neuromorphic chip may adopt a storage-computing integrated processing architecture. Among them, the storage module may be used to store data and instructions related to neural network calculations, and the computing module may be used to perform neural network calculations (e.g., multiplication-accumulation operations). In one example, the storage module may be a memory with a certain storage capacity, storing different types of data such as vectors, matrices, and tensors in neural network calculations, and the computing module may be a multiplication-accumulator array (MACs) composed of 128 multiplication-accumulators. The multiplication-accumulator array may be divided into 4 groups, each group including 32 multiplication-accumulators, which are used to perform multiplication-accumulation operations on the output data of the data processing device, and the operation results may be returned to the data processing device for further processing.

[0157] In one possible implementation, the processing core can perform artificial neural network calculations, pulse neural network calculations, and hybrid neural network calculations that are a mixture of artificial neural networks and pulse neural networks. Different neural network calculation tasks require processing data of different precisions. For example, for pulse neural networks, time information is an important consideration. A neuron in a pulse neural network is activated only when its membrane potential reaches a specific value. When the neuron is activated, it generates a signal to transmit to other neurons, increasing or decreasing the membrane potential of other neurons. It should be understood that before the three-value data is sent to the calculation module for calculation, the three-value data needs to be converted into data precision acceptable to the calculation module for calculation.

[0158] In one possible implementation, the neuromorphic chip disclosed in the present invention can adopt a storage-computing integrated chip architecture, integrating the processor, storage module and communication components together, and information processing is performed locally, which is conducive to better meeting the needs of large-scale parallel computing of neural networks. The neuromorphic chip disclosed in the present invention can support calculations of two computing paradigms, which can support artificial neural network calculations, pulse neural network calculations, and hybrid neural network calculations that mix artificial neural networks and pulse neural networks. In this way, the neuromorphic chip can calculate neural networks in a distributed, large-scale and parallel manner with extremely low energy consumption. It should be understood by those skilled in the art that the neuromorphic chip and processing core are exemplary. The present disclosure does not limit to which processor architecture the data processing device is applied and whether the processor is organized according to the processing core.

[0159] In one possible implementation, different processing cores can process different neural network computing tasks, and the same processing core can also perform different neural network computing tasks in different working cycles. For example, in a certain working cycle, processing core A can be used to process neural network computing tasks related to images, and processing core B can simultaneously process neural network computing tasks related to text. Data of different neural network computing tasks can be assigned to one processing core for processing, or to different processing cores for processing. Therefore, data of different types and different precisions are frequently exchanged between processing cores and within processing cores, and it is necessary to find a suitable solution to efficiently schedule data of different types and different precisions.

[0160] Traditional chips use the von Neumann architecture to separate the storage module and the processor. The processor repeatedly exchanges information with the storage module through the bus. The read and write efficiency of data of different types and different precisions is low, and there are bottlenecks in computing power and energy consumption. At the same time, the existing neuromorphic chips are decentralized in processing data of different types and different precisions, that is, different processing methods are used for different computing tasks and data types. Existing neuromorphic chips can only support one of the computing paradigms of artificial neural network computing or pulse neural network computing, such as neuromorphic chips using leaky-integrate-and-fire (LIF) neuron model. These neuromorphic chips use different processing methods for different computing tasks and data types, which increases the number of accesses to the storage module, the process of scheduling data is cumbersome, the access efficiency is not high, and information redundancy may be caused. Therefore, traditional neuromorphic chips cannot efficiently schedule data of different types and / or different precisions, and the access efficiency to the storage module is low, which greatly limits the computing efficiency of neuromorphic chips and the reasoning efficiency of neural networks, and has poor scalability.

[0161] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A data processing device, characterized in that: A processing core applied to an artificial intelligence processor, the artificial intelligence processor comprising a plurality of processing cores, each processing core comprising a storage module and a data processing device, the data processing device being connected to the storage module, the data processing device comprising: An address generation module, used to generate input addresses and output addresses according to control instructions; a data conversion module, connected to the address generation module, configured to read first data from the storage module according to the input address, perform a data conversion operation on the first data to obtain second data, and write the second data into the output address of the storage module, The data conversion operation includes at least one of data merging, data splitting, data migration and data type conversion operations; The device also includes a row counting module, The row counting module is used to count according to the counting pulses sent by the control module of the processing core, and send the control instruction to the address generating module when the counted number is greater than or equal to the row number threshold; The address generation module includes multiple groups of counters. The multiple groups of counters are used to generate the input address and the output address, and control the data conversion module to perform the data conversion operation through counter logic; The address generation module includes a first group of counters, a second group of counters and a third group of counters. The first group of counters is used to generate a first address, where the first address is the input address; The second group of counters is used to generate a second address; The third group of counters is used to generate a third address, and the third address is the output address. When performing a data merging operation, the second address is the input address; when performing a data splitting operation, the second address is the output address; when performing a data migration operation, the generation operation of the second address is skipped.

2. The device according to claim 1, characterized in that The data conversion operation includes a data merging operation, The address generating module is used to generate a plurality of input addresses and the output address according to the control instruction; The data conversion module is used to read a plurality of first data from the storage module according to a plurality of input addresses, and write the second data generated by merging the plurality of first data into the output address of the storage module.

3. The device according to claim 1, characterized in that The data conversion operation includes a data splitting operation, The address generating module is used to generate the input address and multiple output addresses according to the control instruction; The data conversion module is used to read the first data from the storage module according to the input address, and write the multiple second data obtained by splitting the first data into the multiple output addresses of the storage module respectively.

4. The device according to claim 1, characterized in that The data conversion operation includes Data migration operations, The address generating module is used to generate the input address and the output address according to the control instruction; The data conversion module is used to read the first data from the storage module according to the input address, and write the second data determined according to the first data into the output address of the storage module.

5. The device according to claim 1, characterized in that The data conversion operation includes Data type conversion operations, The data conversion module is used to perform a data type conversion operation on the first data to obtain the second data when the data type of the first data is different from the data type of the second data. The data type includes any one of a 32-bit integer int32, an 8-bit integer int8, and a three-value data type.

6. The device according to claim 1, characterized in that The output address includes the address of the routing module sending data area located in the processing core, The data conversion module is used to write the second data into the sending data area of ​​the routing module, so that the routing module sends the second data.

7. An artificial intelligence processor, characterized in that: It comprises a plurality of processing cores, each of which comprises a storage module and the data processing device as claimed in any one of claims 1 to 6.