Data processing method, device and equipment based on convolutional neural network

Abstract

The present application relates to a data processing method, apparatus, and device based on a convolutional neural network. The method comprises: preprocessing at least two input data to obtain preprocessed data corresponding to each of the at least two input data, wherein at least one of the at least two input data has a length of M; processing the at least two preprocessed data through a convolutional computation layer and an activation function layer of at least M full adder operation units in the convolutional neural network to obtain operation data of the M full adder operation units; and post-processing the operation data to obtain output data. The solution provided by the present application can implement the processing logic of a full adder on a dedicated neural network chip, thereby improving hardware resource utilization, enriching chip functionality, and reducing costs.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device and equipment based on convolutional neural network. Background Art

[0002] Logic gates, also known as the "basic unit of digital logic circuits," are the fundamental components that perform logical operations such as "OR," "AND," "NOT," "NOR," and "NAND" on integrated circuits. These gates can form any complex logic circuit and are widely used in fields such as computers, communications, control, and digital instrumentation. Currently, specialized digital logic chips or general-purpose processors can perform a wide variety of logical operations, enabling the implementation of computer algorithms such as hashing and encryption algorithms.

[0003] With the rapid development of deep learning algorithms based on convolutional neural networks, convolutional neural networks have been widely used in different technical fields, and neural network-specific chips that perform convolutional neural network tasks (such as convolutional neural network inference chips, ASIC (Application Specific Integrated Circuit) chips, etc.) are also widely used in the market.

[0004] However, convolutional neural networks primarily rely on convolution operations, which calculate the product of two variables. Aside from convolution, all other computational operators in neural networks perform nonlinear changes on a single variable. Therefore, convolutional neural networks only have the "dot product" method for operating on two variables and cannot directly handle logical operations. This results in limited functionality, low hardware resource utilization, and increased development and usage costs. Summary of the Invention

[0005] In order to solve or partially solve the problems existing in the related art, the present application provides a data processing method, device and equipment based on convolutional neural network, which can realize addition processing based on logical operations on a neural network dedicated chip, improve hardware resource utilization, enrich chip functions and reduce costs.

[0006] In a first aspect, the present application provides a data processing method based on a convolutional neural network, which is applied to a processor. The method includes: preprocessing at least two input data respectively to obtain preprocessed data corresponding to the at least two input data, and the length of at least one of the at least two input data is M; processing the at least two preprocessed data through a convolution calculation layer and an activation function layer of at least M full adder operation units in the convolutional neural network to obtain operation data of the M full adder operation units; and post-processing the operation data to obtain output data.

[0007] In one embodiment, at least two input data are preprocessed to obtain preprocessed data corresponding to each of the at least two input data, including: converting at least two serial data streams into at least two matrix data blocks through a splicing operation; and the operation data are post-processed to obtain output data, including: converting the operation data in the matrix data block format into a serial data stream.

[0008] In one embodiment, M full adder operation units are connected in series, each full adder operation unit includes an addition branch and a carry branch; at least two preprocessed data are processed through the convolution calculation layer and the activation function layer of at least M full adder operation units in a convolutional neural network to obtain operation data of the M full adder operation units, including: for each full adder operation unit, performing a first convolution processing on the at least two-channel preprocessed data in the first convolution calculation layer to output double-channel convolution data; performing a logical operation on the first single-channel convolution data in the double-channel convolution data through the addition branch to obtain a first operator result of the first single-channel, and performing a logical operation on the second single-channel convolution data in the double-channel convolution data through the carry branch and then processing it through the activation function of the activation function layer to obtain a second operator result of the second single-channel; and using the first operator result and the second operator result as input data of the full adder operation unit of the next level of the current full adder operation unit.

[0009] In one embodiment, the logical operation performed by the addition branch includes a logical operation performed by an exclusive-OR operation unit, the exclusive-OR operation unit being used to perform a first convolution processing on at least two-channel preprocessed data in a first convolution calculation layer to output double-channel convolution data; performing an OR operation on the double-channel convolution data through the OR operation unit and a NAND operation through the NAND operation unit; and performing an AND operation on the results of the OR operation and the NAND operation respectively to obtain operation data of the exclusive-OR operation unit.

[0010] In one embodiment, performing an OR operation through an OR operation unit includes: after the first convolution calculation layer of the OR operation unit performs a first convolution processing on at least two-channel preprocessed data to output single-channel convolution data, processing it through the activation function of the activation function layer, and then performing a second convolution processing on the second convolution calculation layer to output single-channel convolution data, to obtain the operation data of the OR operation unit; and\or, performing an AND operation through an AND-NOT operation unit includes: after the first convolution calculation layer of the AND-NOT operation unit performs a first convolution processing on at least two-channel preprocessed data to output at least two-channel convolution data, processing it through the activation function of the first activation function layer, performing a second convolution processing on at least two-channel convolution data on the second convolution calculation layer to output single-channel convolution data, and processing it through the activation function of the second activation function layer to obtain the operation data of the AND-NOT operation unit.

[0011] In one embodiment, the carry branch includes: a cascaded AND operation unit and a logical left shift unit, wherein the logical left shift unit is used to convolve the preprocessed data of the first single channel in the first convolution calculation layer to output dual-channel convolution data, and then sequentially pass the convolution processing through the second convolution calculation layer, the third convolution calculation layer and the fourth convolution calculation layer, and then pass the convolution data channel merging processing through the fifth convolution calculation layer to output the convolution data of the second single channel as the left shift operation data of the logical left shift unit.

[0012] In one embodiment, the second convolution calculation layer, the third convolution calculation layer, and the fourth convolution calculation layer each include two different convolution kernels, where each convolution kernel includes two channel parameters.

[0013] In one embodiment, for a non-square serial data stream whose length is any positive integer, converting at least two serial data streams into at least two matrix data blocks through a concatenation operation includes: padding the ends of the serial data streams with zeros to obtain a zero-padded serial data stream whose length is any square positive integer, and then concatenating the zero-padded serial data streams into a matrix data block; or concatenating the data into matrix data blocks of unequal length and width. Furthermore, for a non-square serial data stream whose length is any positive integer, converting computational data in a matrix data block format into a serial data stream includes: after obtaining the left-shifted computational data, concatenating and merging the data based on the principle of right-end data segment priority.

[0014] In one embodiment, the method further includes: splitting the N-bit full adder operation unit into N\P-bit carry-lookahead addition operation units, so as to utilize the N\P-bit carry-lookahead addition operation units to process at least two preprocessed data in parallel.

[0015] In one embodiment, the above method further includes: increasing the number of bits of the full adder operation unit by increasing the depth and dimension of the convolution calculation layer.

[0016] In one embodiment, the input data is any one of binary data, octal data or hexadecimal data, wherein the convolution parameters of the carry branches corresponding to the binary data, octal data or hexadecimal data are different.

[0017] In one embodiment, M full adder operation units are connected in series, the input data is a decimal array, and each full adder operation unit includes an addition branch. Accordingly, in a convolutional neural network, at least two preprocessed data are processed through the convolution calculation layer and activation function layer of at least M full adder operation units to obtain operation data of the M full adder operation units, including: for each full adder operation unit, performing a first convolution processing on the at least two-channel preprocessed data in the first convolution calculation layer to output double channel convolution data; performing a logical operation on the at least two-channel preprocessed data through the addition branch to obtain a third operator result; and using the third operator result as input data of the next-level full adder operation unit of the current full adder operation unit.

[0018] In one embodiment, the above method further includes: determining the complement of each of the at least two input data before preprocessing the at least two input data respectively; and using the complement of each of the at least two input data as the at least two input data to obtain a subtraction result for the at least two input data.

[0019] In one embodiment, the method further includes: obtaining a multiplication result for at least two input data by superimposing full adder operation units.

[0020] A second aspect of the present application provides a data processing device based on a convolutional neural network, comprising: a preprocessing module, a calculation module, and a post-processing module;

[0021] The preprocessing module is used to preprocess the at least two input data respectively to obtain preprocessed data corresponding to the at least two input data, and the length of at least one of the at least two input data is M;

[0022] The operation module is used to process at least two preprocessed data through a convolution calculation layer and an activation function layer of at least M full adder operation units in a convolutional neural network to obtain operation data of the M full adder operation units;

[0023] The post-processing module is used to post-process the operation data obtained by the operation module to obtain output data.

[0024] A third aspect of the present application provides an artificial intelligence chip, comprising the data processing device based on the convolutional neural network as described above.

[0025] A fourth aspect of the present application provides a board, the board comprising: a storage device, an interface device, a control device, and the above-mentioned artificial intelligence chip;

[0026] The artificial intelligence chip is connected to the storage device, the control device and the interface device respectively;

[0027] The storage device is used to store data;

[0028] The interface device is used to realize data transmission between the artificial intelligence chip and external equipment;

[0029] The control device is used to monitor the status of the artificial intelligence chip.

[0030] A fifth aspect of the present application provides a computing device comprising: a processor; and a memory. The memory stores executable code, and when the executable code is executed by the processor, the processor executes the above method.

[0031] In a sixth aspect, the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor executes the above method.

[0032] The technical solution provided by this application may have the following beneficial effects:

[0033] The method provided in this application preprocesses input data to obtain preprocessed data. Then, in a convolutional neural network, at least two preprocessed data are processed through the convolutional computation layers and activation function layers of at least M full adder arithmetic units to obtain computational data from the M full adder arithmetic units. The computational data is then post-processed to obtain output data. This allows full adders to be constructed using convolutional computation layers and activation function layers, enabling direct execution of full adder arithmetic processing on a dedicated neural network chip. This improves hardware resource utilization, enriches chip functionality, and reduces costs.

[0034] Furthermore, the method provided in the present application can convert a serial data stream into a matrix data block, so that the input data can be received and processed by a convolutional neural network, and finally convert the operation data in the matrix data block format into a serial data stream.

[0035] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0037] Figure 1 1 is a flow chart of a data processing method based on a convolutional neural network according to an embodiment of the present application;

[0038] Figure 2 1 is another flow chart of a data processing method based on a convolutional neural network according to an embodiment of the present application;

[0039] Figure 3 Schematic diagram of converting a serial data stream into a matrix data block according to an embodiment of the present application;

[0040] Figure 4 Schematic diagram of converting a matrix data block into a serial data stream according to an embodiment of the present application;

[0041] Figure 5 Schematic diagram of the full adder processing process shown in an embodiment of the present application;

[0042] Figure 6 Schematic diagram of an XOR operation process according to an embodiment of the present application;

[0043] Figure 7 Schematic diagram of the OR operation processing process shown in an embodiment of the present application;

[0044] Figure 8 1 is a schematic diagram of a NOT operation processing process shown in an embodiment of the present application;

[0045] Figure 9 Schematic diagram of the AND operation processing process shown in an embodiment of the present application;

[0046] Figure 10 Schematic diagram of a logical left shift operation process shown in an embodiment of the present application;

[0047] Figure 11 1 is a schematic diagram of the structure of a data processing device based on a convolutional neural network shown in an embodiment of the present application;

[0048] Figure 12 is another structural schematic diagram of a data processing device based on a convolutional neural network shown in an embodiment of the present application;

[0049] Figure 13 This is a structural block diagram of an artificial intelligence chip shown in an embodiment of the present application;

[0050] Figure 14 This is a structural block diagram of a board shown in an embodiment of the present application;

[0051] Figure 15 It is a structural diagram of a computing device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although the accompanying drawings illustrate embodiments of the present application, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0053] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0054] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0055] In the related art, neural network-specific chips cannot directly process logical operations, resulting in limited functionality and low hardware resource utilization, which in turn increases development and usage costs. To address the above issues, the present invention provides a data processing method based on a convolutional neural network, which can perform logical operations on a neural network-specific chip, thereby improving hardware resource utilization, enriching chip functionality, and reducing costs.

[0056] The data processing method based on convolutional neural networks in the present application can be applied to a processor, which can be a general-purpose processor, such as a CPU (Central Processing Unit), or an artificial intelligence processor for performing artificial intelligence operations. Artificial intelligence operations may include machine learning operations, brain-like operations, etc. Among them, machine learning operations include neural network operations, k-means operations, support vector machine operations, etc. The artificial intelligence processor may, for example, include one or a combination of GPU (Graphics Processing Unit), NPU (Neural-Network Processing Unit), DSP (Digital Signal Process, Digital Signal Processing Unit), and Field Programmable Gate Array (Field-Programmable Gate Array, FPGA) chips.

[0057] The artificial intelligence processor may be a processor used in an artificial intelligence chip. The artificial intelligence chip may be, for example, a neural network chip or other chip. The neural network chip may be, for example, a convolutional neural network inference chip, an ASIC chip, etc. This application does not limit the specific type of processor.

[0058] In one possible implementation, the processor mentioned in this application may include multiple processing units, each of which can independently run various assigned tasks, such as convolution operations, pooling tasks, or fully connected tasks. This application does not limit the processing units and the tasks they run. Multiple processing units in a processor can share some storage space, such as some RAM storage space and register files, or have their own storage space.

[0059] The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0060] Figure 1 This is a flow chart of a data processing method based on a convolutional neural network according to an embodiment of the present application. This method can be applied to a processor, which can be a processor of an artificial intelligence chip.

[0061] See also Figure 1 , the method may include:

[0062] S110 , preprocessing at least two input data respectively to obtain preprocessed data corresponding to the at least two input data respectively, wherein the length of at least one of the at least two input data is M.

[0063] In this step, the input serial data stream can be converted into a matrix data block through a splicing operation. M is an integer greater than or equal to 4.

[0064] Among them, a serial data stream is a data stream composed of "0" and "1", such as 110010101, 101001110, etc.

[0065] In this step, a serial data stream of a set byte length can be received, for example, a serial data stream of a 32-bit byte length, a serial data stream of a 9-bit byte length, etc. It should be noted that the lengths of the multiple serial data streams input can be the same or different. The following is an exemplary description using two serial data streams, 110010101 and 101001110, each of which has a 9-bit byte length. In other embodiments, serial data streams of different byte lengths, such as 6 bits, 12 bits, or 16 bits, can also be received, and this application does not limit this.

[0066] In this step, the input serial data stream can be converted into a matrix data block (Matrix) in the form of matrix data. For example, a serial data stream with a length of 9 bits can be converted into a matrix data block in the form of a 3×3 matrix.

[0067] S120, processing the at least two preprocessed data through the convolution calculation layer and activation function layer of at least M full adder operation units in the convolutional neural network to obtain operation data of the M full adder operation units.

[0068] The logic operations performed by the full adder's arithmetic unit include, but are not limited to, "OR," "AND," "NOT," "NOR," "NAND," "XOR," and other logic operations. The logic operation units of a full adder include, but are not limited to, "OR unit," "AND unit," "NOT unit," "NOR unit," "NAND unit," "XOR unit," and other logic operation units. These logic operation units can also be referred to as logic operation circuits.

[0069] S130, post-processing the calculated data to obtain output data.

[0070] In this operation, operation data in the matrix data block format may be converted into a serial data stream.

[0071] As can be seen from this example, the method provided by this application, after pre-processing the input data to obtain pre-processed data, can process the pre-processed data through the convolution calculation layer and activation function layer of multiple logic operation units in the convolutional neural network to obtain the operation data of the full adder. Then, the operation data is post-processed to obtain the output data. In this way, the convolution calculation layer and the activation function layer can be used to construct logical operations, so that the logical operation function of the full adder can be directly executed on the neural network dedicated chip, thereby improving the utilization of hardware resources, enriching the chip functions, and reducing costs.

[0072] Figure 2 This is another flowchart of the data processing method based on convolutional neural network in an embodiment of the present application.

[0073] See also Figure 2 , the method comprising:

[0074] S210 , receiving at least two input serial data streams.

[0075] A serial data stream is a data stream consisting of "0"s and "1"s, such as 110010101, 101001110, etc. In this step, a serial data stream of a set byte length can be received, for example, a serial data stream of a 9-bit byte length, such as 110010101. In other embodiments, serial data streams of different byte lengths, such as 6 bits, 12 bits, 16 bits, or 32 bits, can also be received, and this application is not limited to this.

[0076] S220 , performing preprocessing to convert each of the at least two serial data streams into at least two matrix data blocks through a splicing operation.

[0077] In this step, the input serial data stream can be converted into a matrix data block in the form of matrix data. For example, a serial data stream with a length of 9 bits can be converted into a matrix data block in the form of a 3×3 matrix.

[0078] For example, in a serial data stream with a 9-bit length, the first, second, and ninth bits are sorted from right to left. First, the first three bytes are taken and placed in the top row of a 3×3 matrix from left to right. Then, the middle three bytes are taken and placed in the middle row of a 3×3 matrix from left to right. Finally, the last three bytes are taken and placed in the bottom row of a 3×3 matrix from left to right. It is understood that other sorting orders can also be used to obtain a matrix data block in the form of a 3×3 matrix.

[0079] refer to Figure 3Taking the serial data stream ...987654321 as an example, the splicing conversion method can be to put the serial data stream of 9 bytes in length into a matrix in a 3×3 form to obtain a matrix data block in the form of a 3×3 matrix. The process is as follows Figure 3 The "S / M" conversion process is identified.

[0080] Convert 987654321 to

[0081] In this way, the converted matrix data block can be adapted to the convolution operation in the convolutional neural network.

[0082] The above operation of converting the serial data stream into the matrix data block may be referred to as a data preprocessing operation, which may be represented by “S / M”.

[0083] In certain embodiments, for a non-square serial data stream having a length equal to any positive integer, converting at least two serial data streams into at least two matrix data blocks through a concatenation operation may include any of the following operations: padding the ends of the serial data streams with zeros to obtain a zero-padded serial data stream having a length equal to any square of a positive integer, and then concatenating the zero-padded serial data streams into a matrix data block. Alternatively, concatenating the data into matrix data blocks of unequal lengths and widths.

[0084] For example, for data segments whose length is not a square number, there are two data mapping (MAP) operations. One is to first add zeros to the end and then splice the data into a matrix data block. The other is to directly splice the data into a matrix data block of unequal length and width, but of similar length and width.

[0085] S230, performing different processing on the matrix data block obtained by preprocessing through the convolution calculation layer and activation function layer of at least M full adder operation units to obtain operation data of the M full adder operation units.

[0086] Among them, in the convolutional neural network, the convolution calculation layer and the activation function layer in the convolutional neural network can be constructed, the convolution kernel in the convolution calculation layer, the padding, step size and bias during the convolution processing and other convolution parameters are pre-set, and the activation function type in the activation function layer is pre-set, thereby realizing the functional requirements of the addition branch and the requirements of the logical shift function, and realizing the addition operation processing of at least two matrix data blocks.

[0087] Specifically, the full adder operation unit can be composed of a plurality of logical operation units through logical combination. The logical operation units include, but are not limited to, an AND operation unit, an OR operation unit, a NOT operation unit, a NAND-NAND operation unit, an XOR operation unit, and a shift operation unit. Among them, some logical operation units can be composed of other logical operations, such as an XOR operation unit can be composed of an OR operation unit, an AND operation unit, and a NAND-NAND operation unit. The shift operation unit can include a left shift operation unit and a right shift operation unit.

[0088] For example, after the first convolution calculation layer of the AND operation unit convolves the at least two-channel preprocessed data to output single-channel convolution data, the data is processed by the activation function of the activation function layer to obtain operation data of the AND operation unit. The first convolution calculation layer adds the at least two-channel preprocessed data to output the single-channel convolution data.

[0089] For example, after the first convolution calculation layer of the OR operation unit performs a first convolution process on the at least two-channel preprocessed data to output single-channel convolution data, the data is processed by the activation function of the activation function layer, and then a second convolution process is performed on the second convolution calculation layer to output single-channel convolution data, thereby obtaining the operation data of the OR operation unit. In the first convolution calculation layer, the at least two-channel preprocessed data are added and then the inverse of the data is taken bit by bit to output the single-channel convolution data; in the second convolution calculation layer, the data is inversely taken bit by bit to output the single-channel convolution data.

[0090] For example, after the first convolution calculation layer of the non-operation unit performs convolution processing on the single-channel pre-processed data and outputs the single-channel convolution data, it is processed by the activation function of the activation function layer to obtain the operation data of the non-operation unit. In the first convolution calculation layer, the single-channel pre-processed data is inversely bit by bit and the single-channel convolution data is output.

[0091] For example, after the first convolution calculation layer of the AND-NOT operation unit performs a first convolution process on the at least two-channel preprocessed data to output at least two-channel convolution data, the data is processed by the activation function of the first activation function layer, and the at least two-channel convolution data is processed by the second convolution calculation layer to output single-channel convolution data, and the data is processed by the activation function of the second activation function layer to obtain the operation data of the AND-NOT operation unit. In this case, the first convolution calculation layer adds the at least two-channel preprocessed data to output at least two-channel convolution data; and the second convolution calculation layer takes the inverse of the at least two-channel convolution data bit by bit to output single-channel convolution data.

[0092] For example, in the first convolution calculation layer of the XOR operation unit, the at least two-channel preprocessed data is subjected to a first convolution process to output double-channel convolution data; an OR operation and a NAND operation are respectively performed on the double-channel convolution data; and an AND operation (AND) is performed on the results of the OR and NAND operations to obtain operation data of the XOR operation unit. In the first convolution calculation layer, the at least two-channel preprocessed data is subjected to a first convolution process, the at least two-channel preprocessed data is copied, and then spliced ​​to produce double-channel convolution data of at least two channels.

[0093] S240 , performing post-processing to convert the operation data in the matrix data block format into a serial data stream.

[0094] In this step, the matrix data block can be converted into a serial data stream. For example, a 3×3 matrix data block can be converted into a 9-bit serial data stream. This operation process can be called a data post-processing operation, which is the inverse operation of the data pre-processing operation (i.e., converting the serial data stream into a matrix data block) and can be represented by "M / S". Figure 4 :

[0095] can be converted to 110110110.

[0096] It should be noted that for different logical operation requirements, such as "AND", "OR" and "NOT" logical operations, different types of convolutional neural networks can be constructed according to different logical operation rules and combined with the operating principles of convolutional neural networks to realize the processing of the above-mentioned "AND", "OR" and "NOT" logical operations, thereby realizing the execution of logical operation processing on neural network dedicated chips, so that there is no need to use digital logic dedicated chips or general-purpose processors to perform logical operations, thereby making full use of hardware computing resources, and can simultaneously execute neural network reasoning tasks and logic gate operation tasks to meet user needs.

[0097] The following is an exemplary description of the relevant contents of the full adder.

[0098] In some embodiments, M full adder operation units are connected in series, and each full adder operation unit includes an addition branch and a carry branch.

[0099] Accordingly, for each full adder operation unit, processing the at least two preprocessed data through the convolution calculation layer and activation function layer of at least M full adder operation units in the convolutional neural network to obtain operation data of the M full adder operation units may include the following operations:

[0100] First, the first convolution layer performs a first convolution on at least two channels of preprocessed data, outputting double-channel convolution data. Specifically, the convolution layer, with two channels as input and four channels as output, performs a "copy" operation on the input data, i.e., the original single-channel data block is copied into two identical dual-channel data blocks. The copies are then concatenated, forming two sets of data.

[0101] Then, the first single-channel convolution data in the double-channel convolution data is subjected to a logical operation through the addition branch to obtain the first operator result of the first single-channel. The second single-channel convolution data in the double-channel convolution data is subjected to a logical operation through the carry branch and then processed by the activation function of the activation function layer to obtain the second operator result of the second single-channel. Specifically, the above two sets of data are respectively sent to the addition branch and the carry branch. For example, the addition branch can be implemented by an exclusive-OR operation unit, and the carry branch can be implemented by cascading an AND operation unit and a logical left shift unit to determine the number of bits that have carried.

[0102] For example, for a non-square serial data stream whose length is any positive integer, converting the matrix data block format into a serial data stream can include: after obtaining the left-shifted data, concatenating and merging the data segments based on the principle of right-end data segment priority. Specifically, data post-processing is the reverse of pre-processing, i.e., converting the matrix data blocks into a serial data stream output (labeled M / S in the figure). For data padded with zeros during pre-processing, these zeros should be removed during post-processing to obtain the final result.

[0103] Then, the first operator result and the second operator result are used as input data of the full adder operation unit at the next stage of the current full adder operation unit. Specifically, the first operator result and the second operator result can be integrated into corresponding 0 / 1 bit data as the input of the next stage full adder.

[0104] Figure 5 Schematic diagram of the full adder processing process shown in an embodiment of the present application.

[0105] See also Figure 5 Taking a 256-bit adder as an example, the two serial data streams to be added are preprocessed to obtain at least two matrix data blocks, which are then input to a full-adder. The full-adder's first convolutional layer performs a first convolution on the at least two channel matrix data blocks, outputting double-channel convolution data.

[0106] Among them, the bias value of the first convolution calculation layer is zero (Bias=0), the step size is set to 1 (stride=1), and the padding is 1 (padding=1).

[0107] The first convolution calculation layer may include four convolution kernels, which may be respectively referred to as the zeroth convolution kernel, the first convolution kernel, the second convolution kernel, and the third convolution kernel. Among them, the four convolution kernels are all 3×3 two-channel convolution kernels.

[0108] The structure of the zeroth convolution kernel is the same as that of the second convolution kernel, and the corresponding two channels are:

[0109] and

[0110] The structure of the first convolution kernel is the same as that of the third convolution kernel, and the corresponding two channels are:

[0111] and

[0112] The two matrix data blocks are spliced ​​into a two-channel matrix data block, which is then convolved with the zeroth convolution kernel to extract data from one channel of the two-channel matrix data block. The data is then convolved with the first convolution kernel to extract data from the other channel of the two-channel matrix data block. Finally, the two extracted single-channel matrix data blocks are spliced ​​together to obtain a two-channel matrix data block, effectively copying the two matrix data blocks once and obtaining one copy of the input data.

[0113] Similarly, the second and third convolution kernels can be used to perform the convolution operation on the two matrix data blocks to obtain another set of input data. In this way, the two matrix data blocks are copied, resulting in two sets of input data, each of which includes two matrix data blocks.

[0114] That is to say, the first convolution calculation layer outputs a four-channel matrix data block, which can perform different processing on a channel-by-channel basis. That is, it is divided into two parts of input data, one of which is input into the addition branch for data processing, and the other is input into the carry branch for data processing. The two calculation results are then used as the input data of the next-level full adder.

[0115] See also Figure 5 The logical operation performed by the addition branch may include an exclusive-OR operation (XOR) performed by an exclusive-OR operation unit. It should be noted that no pre-processing or post-processing operations are required for the data block when performing the exclusive-OR operation. The logical operation performed by the carry branch may include a cascaded AND operation unit and a logical left shift unit (LS). The operation results of the addition branch and the carry branch serve as input data for the next full-adder.

[0116] It should be noted that for M-bit two's complement addition, at least M full adders are required to be cascaded to ensure that the carry bit at the end is passed to the first bit to output the correct operation result. For example, the addition of 1010 and 0101 is 1111, just press Figure 5 The data path is executed once to get the correct result. For example, the sum of 1111 and 1111 is 0000, so you need to press Figure 5 The data path must be executed 4 times to output the correct result.

[0117] The addition branch is schematically described below.

[0118] Figure 6 This is a schematic diagram of the XOR operation processing process shown in an embodiment of the present application.

[0119] See also Figure 6 The XOR operation unit is used to perform a first convolution process on at least two-channel preprocessed data in the first convolution calculation layer to output double-channel convolution data; perform an OR operation on the double-channel convolution data through the OR operation unit and a NAND operation through the NAND operation unit; and perform an AND operation on the results of the OR operation and the NAND operation respectively to obtain the operation data of the XOR operation unit. It can be understood that the logical operation processing of "XOR" can be performed by performing "OR" operation and "NAND" operation on the four-channel data block by channel, and then performing "AND" operation on the output result. It should be noted that when performing the XOR operation, there is no need to perform pre-processing and post-processing operations on the matrix data block.

[0120] For example, Figure 6 The S / M operation and the operation performed by the first convolutional calculation layer (Conv.Layer) are respectively Figure 5 The S / M operation is the same as that performed by the first convolutional computation layer and does not need to be repeated. For example, Figure 5 The matrix data blocks of two channels in the four channels obtained by the first convolution calculation layer can be used as Figure 6 The matrix data blocks of the two channels obtained by the first convolution calculation layer are transmitted to the OR operation unit for performing an OR operation (OR) and to the NAND operation unit for performing an NAND operation, so that the operation results of the OR operation unit and the NAND operation unit are transmitted to the AND operation unit for performing an AND operation to obtain the operation result of the XOR operation unit.

[0121] In order to facilitate understanding of the technical solution of the present application, the OR operation (or), AND operation (and) and NAND operation (nand) are exemplarily described below.

[0122] In some embodiments, performing an OR operation through an OR operation unit includes: after the first convolution calculation layer of the OR operation unit performs a first convolution processing on at least two-channel preprocessed data to output single-channel convolution data, processing it through the activation function of the activation function layer, and then performing a second convolution processing in the second convolution calculation layer to output single-channel convolution data to obtain the operation data of the OR operation unit.

[0123] Figure 7 It is a schematic diagram of the OR operation processing process shown in an embodiment of the present application.

[0124] See also Figure 7 After the first convolution calculation layer performs a first convolution on the at least two-channel preprocessed data to output single-channel convolution data, the data is processed by the activation function of the activation function layer, and then a second convolution is performed on the second convolution calculation layer to output single-channel convolution data, thereby obtaining the operation data of the OR operation unit. In the first convolution calculation layer, the at least two-channel preprocessed data are added and then bit-by-bit negated to output the single-channel convolution data. In the second convolution calculation layer, the data are bit-by-bit negated to output the single-channel convolution data.

[0125] Here, taking two-channel input as an example but not limited thereto, the serial data streams of the two-channel input are respectively converted into two-channel matrix data blocks, the matrix data blocks are in the form of 3×3 matrices, and the two matrix data blocks can be represented by A and B respectively.

[0126] That is to say, for the "OR" operation unit, after the data is pre-processed, it is processed by the first convolution calculation layer, the activation function calculation layer, and the second convolution calculation layer, and then the post-processing operation is performed to obtain the output result of the "OR" operation unit. The convolution kernel size of the first convolution calculation layer of the "OR" operation unit is 3x3, which is a two-channel input and a single-channel output for the first convolution calculation layer. The convolution kernel size of the second convolution calculation layer of the "OR" operation unit is 3x3, which is a single-channel input and a single-channel output for the second convolution calculation layer. Among them, the convolution operation of the first convolution calculation layer is equivalent to adding the two-channel data and then taking the inverse bit by bit, and the convolution of the second convolution calculation layer is equivalent to taking the inverse bit by bit of the data block. The convolution kernel parameters are as follows: Figure 7 As shown in .

[0127] In this embodiment, the logical OR operation expression may be: A∪B=1-relu(1-AB).

[0128] Among them, the convolution kernel of the first convolution calculation layer is 3×3 (ie kernel = 3x3), which is a two-channel convolution kernel, the step size is set to 1 (ie stride = 1), the padding is 1 (ie padding = 1), and the bias value is set to 1 (ie Bias = 1).

[0129] Among them, the convolution kernel of the second convolution calculation layer is 3×3 (ie kernel = 3x3), which is a single-channel convolution kernel, the step size is set to 1 (ie stride = 1), the padding is 1 (ie padding = 1), and the bias value is set to 1 (ie Bias = 1).

[0130] The activation function in the activation function layer may be a Relu activation function.

[0131] In one embodiment, the processing of the first convolutional calculation layer, the activation function layer, and the second convolutional calculation layer includes:

[0132] 1) In the first convolution calculation layer, the input two-channel matrix data block is convolved according to the convolution parameters.

[0133] In the convolution process, the convolution parameters include: padding = 1, stride = 1, Bias = 1,

[0134] The convolution kernel corresponding to the two channels is and

[0135] The input two-channel matrix data blocks are:

[0136] Matrix data block Matrix data block

[0137] After padding = 1, the two-channel matrix data blocks A and B are and

[0138] A' and B' are convolved with the convolution kernel of the first convolution calculation layer to obtain the single-channel matrix data: (That is, to implement -AB processing, which is equivalent to adding two matrix data blocks and then taking the opposite number bit by bit), and then undergo Bias=1 bias processing operation (add 1 to each data in the matrix data), and get the matrix data (i.e., achieving 1-AB processing).

[0139] 2) In the activation function layer, the Relu activation function is used to process the data obtained by convolution processing to obtain the output data of the activation function layer.

[0140] For example, the data obtained by convolution Use the Relu activation function to process and get the output data of the activation function layer (i.e., implementing relu(1-AB) processing).

[0141] Among them, the Relu activation function can be defined as: Relu(x)=max(0,x).

[0142] The Relu activation function is a linear correction function. If the calculated value is less than 0, it is equal to 0, otherwise the original value remains unchanged.

[0143] It can be seen that the matrix data block can represent the result of a logical AND operation on the two matrix data blocks A and B input above.

[0144] It should be noted that the processing process of step 1) can be understood as the processing operation of the convolution calculation layer on the two matrix data blocks, and the processing process of step 2) can be understood as the processing operation of the activation function layer on the output data of the convolution calculation layer, thereby obtaining the matrix data block of the "AND" operation result.

[0145] 3) In the second convolution calculation layer, the output data of the activation function layer is convolved according to the convolution parameters.

[0146] In the convolution process, the convolution parameters include: padding = 1, stride = 1, Bias = 1,

[0147] The convolution kernel corresponding to a single channel is

[0148] For example, the output data of the activation function layer After padding = 1, the convolution kernel of the second convolution calculation layer is used for convolution processing, and the obtained (equivalent to taking the opposite number bit by bit), and then after Bias=1 bias processing operation, the matrix data block is obtained (i.e., implementing 1-relu (1-AB) processing).

[0149] It can be seen that the matrix data block can represent the result of a logical OR operation on the two matrix data blocks A and B input above.

[0150] It should be noted that the processing process of step 1) can be understood as the processing operation of the first convolution calculation layer on the two matrix data blocks, the processing process of step 2) can be understood as the processing operation of the activation function layer on the output data of the first convolution calculation layer, and the processing process of step 3) can be understood as the processing operation of the second convolution calculation layer on the output data of the activation function layer, thereby obtaining the matrix data block of the "OR" operation result.

[0151] In some embodiments, performing a NAND operation through a NAND operation unit includes: after the first convolution calculation layer of the NAND operation unit performs a first convolution processing on at least two-channel preprocessed data to output at least two-channel convolution data, processing is performed through the activation function of the first activation function layer, performing a second convolution processing on the at least two-channel convolution data in the second convolution calculation layer to output single-channel convolution data, and processing is performed through the activation function of the second activation function layer to obtain the operation data of the NAND operation unit.

[0152] Figure 8 This is a schematic diagram of the NAND operation processing process shown in an embodiment of the present application.

[0153] See also Figure 8 , after the first convolution calculation layer performs a first convolution processing on the at least two-channel preprocessed data to output at least two-channel convolution data, it is processed by the activation function of the first activation function layer, and the at least two-channel convolution data is subjected to a second convolution processing in the second convolution calculation layer to output single-channel convolution data, and is processed by the activation function of the second activation function layer to obtain the operation data of the NAND operation unit; wherein, the at least two-channel preprocessed data are added in the first convolution calculation layer to output at least two-channel convolution data; in the second convolution calculation layer, the at least two-channel convolution data are inversely bit by bit to output single-channel convolution data.

[0154] Here, taking two-channel input as an example but not limited thereto, the serial data streams of the two-channel input are respectively converted into two-channel matrix data blocks, the matrix data blocks are in the form of 3×3 matrices, and the two matrix data blocks can be represented by A and B respectively.

[0155] That is to say, for the "AND NOT" operation unit, after the data is pre-processed, a two-channel data block is obtained. The data block will be processed by two convolution calculation layers, where the processing of each convolution calculation layer includes convolution calculation and activation function calculation (Relu). After that, after post-processing operations, the output result of the "AND NOT" operation unit can be obtained. The convolution kernel size of the first convolution calculation layer of the "AND NOT" operation unit is 3x3. For the first convolution calculation layer, it is two-channel input and two-channel output. The convolution calculation of this layer is equivalent to performing an "AND" operation on two serial data; the convolution kernel size of the second convolution calculation layer is 3x3. For the second convolution calculation layer, it is two-channel input and single-channel output. The convolution calculation of this layer is equivalent to taking the opposite bit of the data in the data block. The cascade of the two, that is, the two convolution calculation layers, can realize the "AND NOT" operation. Post-processing operations are performed on the output results of the convolution calculation layer to obtain the "AND NOT" calculation results of the two serial data. The parameters of each convolution calculation layer are as follows: Figure 8 shown.

[0156] Among them, the convolution kernel of the first convolution calculation layer is 3×3 (ie kernel = 3x3), which is a two-channel convolution kernel, the step size is set to 1 (ie stride = 1), the padding is 1 (ie padding = 1), and the bias value is set to -1 (ie Bias = -1).

[0157] Among them, the convolution kernel of the second convolution calculation layer is 3×3 (ie kernel = 3x3), which is a two-channel convolution kernel, the step size is set to 1 (ie stride = 1), the padding is 1 (ie padding = 1), and the bias value is set to 1 (ie Bias = 1).

[0158] The activation function in the activation function layer may be a Relu activation function.

[0159] In one embodiment, the processing of the first convolution calculation layer, the first activation function layer, the second convolution calculation layer, and the second activation function layer includes:

[0160] 1) In the first convolution calculation layer, the input two-channel matrix data block is subjected to the first convolution processing according to the convolution parameters.

[0161] In this convolution process, the convolution parameters include: padding = 1, stride = 1, Bias = -1,

[0162] The convolution kernel corresponding to the two channels is and

[0163] The input two-channel matrix data blocks are:

[0164] Matrix data block Matrix data block

[0165] The process of performing convolution processing according to the convolution parameters in this step can be referred to the processing process in the OR operation, which will not be repeated here.

[0166] It should be noted that at this time, the first convolution calculation layer outputs two-channel convolution data.

[0167] 2) In the first activation function layer, the data obtained by convolution processing is processed using the first ReLU activation function to obtain an output matrix data block.

[0168] 3) In the second convolution calculation layer, the two-channel matrix data blocks are subjected to a second convolution process according to the convolution parameters.

[0169] In the convolution process, the convolution parameters include: padding = 1, stride = 1, Bias = 1,

[0170] The convolution kernel corresponding to the two channels is and

[0171] The process of performing convolution processing according to the convolution parameters in this step can be referred to the processing process in the AND operation, which will not be repeated here.

[0172] 4) In the second activation function layer, the data obtained by the convolution processing is processed using the second ReLU activation function to obtain an output matrix data block.

[0173] After processing by the second activation function layer, the matrix data block is finally obtained

[0174] It can be seen that the matrix data block can represent the result of performing a logical AND operation on the two matrix data blocks A and B input above.

[0175] Figure 9 This is a schematic diagram of the AND operation processing process shown in an embodiment of the present application.

[0176] See also Figure 9 After the first convolution calculation layer convolves the at least two-channel preprocessed data to output single-channel convolution data, it is processed by the activation function of the activation function layer to obtain operation data of the operation unit; wherein, the first convolution calculation layer adds the at least two-channel preprocessed data to output single-channel convolution data.

[0177] Here, taking two-channel input as an example but not limited thereto, the serial data streams of the two-channel input are respectively converted into two-channel matrix data blocks, the matrix data blocks are in the form of 3×3 matrices, and the two matrix data blocks can be represented by A and B respectively.

[0178] That is to say, for the AND operation unit, after the data is pre-processed, it is processed by the convolution calculation layer and the activation function calculation layer respectively, and then the post-processing operation is performed to obtain the output result of the AND operation unit. The convolution kernel size of the AND operation unit is 3x3, which means that the convolution calculation layer has two channels of input and a single channel of output. The convolution operation of the convolution calculation layer is equivalent to the addition of two channels of data. The convolution kernel parameters are as follows: Figure 9 As shown in .

[0179] In this embodiment, the logical AND operation expression may be: A∩B=relu(A+B-1).

[0180] Among them, the convolution kernel of the convolution calculation layer is 3×3 (ie kernel = 3x3), the convolution kernel is a two-channel convolution kernel, the step size is set to 1 (ie stride = 1), the padding is 1 (ie padding = 1), and the bias value is set to -1 (ie Bias = -1).

[0181] The activation function in the activation function layer may be a Relu activation function.

[0182] In one embodiment, the processing of the convolutional calculation layer and the activation function layer includes:

[0183] 1) In the convolution calculation layer, the input two-channel matrix data block is convolved according to the convolution parameters.

[0184] In this convolution process, the convolution parameters include: padding = 1, stride = 1, Bias = -1, and the convolution kernel corresponding to the two channels is and

[0185] The input two-channel matrix data blocks are:

[0186] Matrix data block Matrix data block

[0187] Among them, the matrix data block A is obtained by converting the serial input data stream 110010101, and the matrix data block B is obtained by converting the serial input data stream 101001110.

[0188] After padding = 1, the two-channel matrix data blocks A and B are and

[0189] Then, the convolution kernel of the convolution calculation layer is used for convolution processing according to the convolution principle to obtain the single-channel matrix data. (i.e., A+B processing is implemented, which is equivalent to matrix addition of two matrix data blocks), and then Bias=-1 bias processing operation (adding -1 to each of the data in the matrix data) is performed to obtain the matrix data (i.e., achieving A+B-1 processing).

[0190] Among them, the convolution processing using the convolution kernel can be implemented using a related convolution function algorithm, which is not limited in this application.

[0191] 2) In the activation function layer, the Relu activation function is used to process the data obtained by convolution processing to obtain the output matrix data block.

[0192] For example, the data obtained by convolution Use the Relu activation function to process and get the matrix data block (i.e., implementing relu(A+B-1) processing).

[0193] It can be seen that the matrix data block can represent the result of a logical AND operation on the two matrix data blocks A and B input above.

[0194] It should be noted that the processing process of step 1) can be understood as the processing operation of the convolution calculation layer on the two matrix data blocks, and the processing process of step 2) can be understood as the processing operation of the activation function layer on the output data of the convolution calculation layer, thereby obtaining the matrix data block of the "AND" operation result.

[0195] The carry branch is schematically described below.

[0196] See also Figure 5 The carry branch includes: a cascaded AND operation unit and a logical left shift unit. The data processing process of the AND operation unit can refer to the relevant content shown above and will not be described in detail here.

[0197] Figure 10 This is a schematic diagram of the logical left shift operation processing process shown in an embodiment of the present application.

[0198] See also Figure 10 The logical left shift unit is used to convolve the preprocessed data of the first single channel in the first convolution calculation layer to output dual-channel convolution data, and then sequentially pass the convolution processing through the second convolution calculation layer, the third convolution calculation layer and the fourth convolution calculation layer, and then pass the convolution data channel merging processing through the fifth convolution calculation layer to output the convolution data of the second single channel as the left shift operation data of the logical left shift unit.

[0199] It can be seen from this embodiment that the method provided in the embodiment of the present application can construct a convolutional neural network to realize the processing of the above-mentioned logical left shift according to the required logical left shift rules and the operating principle of the convolutional neural network, thereby realizing the logical left shift task of data on an operating processing device that is adapted to the neural network structure (for example, a neural network dedicated chip, a general processor, and an image processor, etc.), and can give full play to the excellent computing acceleration function of the neural network dedicated chip, so that there is no need for an external ARM chip or the use of a DMA chip to realize logical shift, thereby making full use of hardware computing resources, and can execute neural network reasoning tasks and logical shift tasks at the same time, which can improve computing efficiency and reduce latency.

[0200] In some embodiments, the second convolution calculation layer, the third convolution calculation layer, and the fourth convolution calculation layer each include two different convolution kernels, where each convolution kernel includes two channel parameters.

[0201] See also Figure 10 After the processor convolves the preprocessed data in the first convolution calculation layer to output two-channel convolution data, it performs convolution processing through three different intermediate convolution calculation layers according to the logical left shift operation rule, and then merges the convolution data channels through the fifth convolution calculation layer to obtain the shift operation data of the logical left shift.

[0202] That is to say, in order to realize the "logical left shift" operation of the data, after the preprocessing operation, the data is processed by the first convolution calculation layer, the second convolution calculation layer, the third convolution calculation layer, the fourth convolution calculation layer and the fifth convolution calculation layer using different convolution kernels, and then through the post-processing operation, the output result after the "logical left shift" operation can be obtained.

[0203] In one embodiment, the processing of the first convolution calculation layer, the second convolution calculation layer, the third convolution calculation layer, the fourth convolution calculation layer, and the fifth convolution calculation layer includes:

[0204] 1) In the first convolution calculation layer, the input single-channel matrix data block is convolved according to the convolution parameters.

[0205] In this convolution process, the two convolution kernels are the same, and the channel parameters of each convolution kernel are

[0206] For example, the input single-channel matrix data block is: matrix data block A=101001110.

[0207] Among them, the matrix data block A is converted from the serial input data stream 101001110

[0208] After the matrix data block A is filled with padding = 1, it becomes

[0209] After the two convolution kernels in the first convolution calculation layer are convolved according to the convolution principle, the output two-channel matrix data is obtained. and

[0210] It can be found that the “copying” of the input single-channel matrix data block is achieved in this step.

[0211] 2) In the second convolution calculation layer, the data output by the first convolution calculation layer is convolved according to the convolution parameters.

[0212] In this convolution process, the channel parameter of one of the convolution kernels is and The channel parameter of another convolution kernel is and

[0213] For example, converting two-channel matrix data and After padding=1

[0214] The matrix data after filling are The two matrix data are convolved through the two convolution kernels in the second convolution calculation layer, and the following are obtained:

[0215] and

[0216] After fitting the matrix data, the output two-channel matrix data is obtained:

[0217] and

[0218] It can be found that in this step, the one-channel matrix data is realized The upward "movement" obtains For another channel matrix data Move left to get

[0219] 3) In the third convolution calculation layer, the data output by the second convolution calculation layer is convolved according to the convolution parameters.

[0220] In this convolution process, the channel parameter of one of the convolution kernels is and The channel parameter of another convolution kernel is and

[0221] For example, converting two-channel matrix data and After padding=1, we get and Convolution is performed on the two convolution kernels in the third convolution calculation layer, and the following are obtained:

[0222] and

[0223] After fitting the matrix data, the output two-channel matrix data is obtained: and

[0224] It can be found that in this step, the one-channel matrix data is realized Move rightward to get For another channel matrix data The "copy" gets

[0225] 4) In the fourth convolution calculation layer, the data output by the third convolution calculation layer is convolved according to the convolution parameters.

[0226] In this convolution process, the channel parameter of one of the convolution kernels is and The channel parameter of another convolution kernel is and

[0227] For example, converting two-channel matrix data and After padding=1, we get and Convolution is performed through the two convolution kernels in the fourth convolution calculation layer, and the following are obtained:

[0228] and

[0229] After fitting the matrix data, the output two-channel matrix data is obtained:

[0230] and

[0231] It can be found that in this step, the one-channel matrix data is realized Move rightward to get For another channel matrix data The "copy" gets

[0232] 5) In the fifth convolution calculation layer, the data output by the fourth convolution calculation layer is convolved according to the convolution parameters.

[0233] In this convolution process, the channel parameters of the convolution kernel in the fifth convolution calculation layer are:

[0234] and

[0235] For example, converting two-channel matrix data and After padding = 1, the convolution kernel in the fifth convolution layer is used to perform convolution processing, and the result is:

[0236] It can be found that in this step, by merging the two-channel matrix data, the effect of "rearrangement" of the data is achieved.

[0237] It can be seen that in the process of "logical left shift", the convolution operation characteristics of the convolutional neural network are utilized. By using different convolution kernels of the first convolution calculation layer, the second convolution calculation layer, the third convolution calculation layer, the fourth convolution calculation layer and the fifth convolution calculation layer, different operations are performed on the input data, thereby realizing the logical left shift operation of the input data and obtaining the matrix data block of the logical left shift operation result.

[0238] In the embodiments shown above, the full adder function is achieved by combining multiple logical operations. For example, the data is first copied through a convolutional layer with dual-channel input and four-channel output, that is, the original single-channel data block is copied into two identical dual-channel data blocks. The copied results are spliced ​​two by two to form two sets of data. Then, the two sets of data are sent to the addition branch and the carry branch respectively. Among them, the addition branch can be implemented by an exclusive-OR logical operation. When performing the exclusive-OR logical operation, there is no need to perform pre-processing and post-processing operations on the data block. The carry branch is implemented by cascading an AND operation unit and a logical left shift unit to determine the number of bits that have carried out and integrate them into corresponding 0 / 1 bit data as the input of the next level full adder.

[0239] It should be noted that in the above embodiment, the input data is matrixed into a batch size of 1 and a channel size of 2. In other embodiments, the efficiency of the convolution calculation can be improved by dividing the input data and matrixing it into multiple channels. In addition, when calculating the addition operation for multiple input data, it can be achieved by increasing the batch size. In the scenario of expanding the channel (Channel), the channel (Channel) of the corresponding convolution kernel also needs to be adjusted accordingly, and the convolution parameters are copied to the expanded channel.

[0240] In addition, the length of the serial data stream, the structure of each logic operation unit, and the dimensions of the rectangular data block shown above are only exemplary and should not be construed as limiting the present application. For example, the number of bits of the full adder operation unit can be increased by increasing the depth and dimension of the convolution calculation layer.

[0241] The embodiments of the present application can implement an M-bit two's complement adder. This adder only requires direct design of the adder network's convolution kernel parameters and network structure within a neural network training framework such as Caffe, TensorFlow, or Pytorch. This allows for adaptive deployment on neural network inference chips, general-purpose processors, and graphics processing units (GPUs) without changing the hardware architecture. This convenient implementation of an M-bit two's complement adder does not compromise the computational performance of existing neural network inference chips.

[0242] In certain embodiments, a parallel carry-lookahead adder can be similarly implemented by combining multiple logical operations. For example, by implementing carry-lookahead logic in conjunction with logical operations implemented using convolution operations (such as "OR," "AND," "NOT," "NAND-NOR," and "XOR"), a carry-lookahead adder based on convolution operations can be implemented. If a serial carry adder is used in conjunction with a carry-lookahead adder, a more efficient multi-bit adder can be implemented.

[0243] For example, an N-bit full adder operation unit is split into N\P-bit carry-lookahead addition operation units, so as to utilize the N\P-bit carry-lookahead addition operation units to process at least two pre-processed data in parallel.

[0244] In some embodiments, the input data is any one of binary data, octal data or hexadecimal data, wherein the convolution parameters of the carry branches corresponding to the binary data, octal data or hexadecimal data are different.

[0245] For example, M full adder operation units are connected in series, the input data is a decimal array, and each full adder operation unit includes an addition branch.

[0246] Accordingly, for each full adder operation unit, processing the at least two preprocessed data through the convolution calculation layer and activation function layer of at least M full adder operation units in the convolutional neural network to obtain operation data of the M full adder operation units may include the following operations:

[0247] First, at the first convolution calculation layer, at least two channels of preprocessed data are subjected to a first convolution process to output double channel convolution data.

[0248] Then, a logic operation is performed on the pre-processed data of at least two channels through an addition branch to obtain a third operator result.

[0249] Next, the third operator result is used as input data of a full adder operation unit at a next stage of the current full adder operation unit.

[0250] Specifically, in the decimal adder scenario, the decimal array is rearranged into matrix data. This is achieved by removing the carry branch in the binary full adder and modifying the convolution kernel parameters and convolution structure of the addition branch based on the structure of each full adder arithmetic unit.

[0251] In some embodiments, based on the binary adder, the M-bit binary subtractor can be implemented by directly performing a complement operation on the original code of the input data.

[0252] Specifically, before the at least two input data are preprocessed respectively, the above method may further include the following operations:

[0253] First, a complement code for each of at least two input data is determined.

[0254] Then, the complements of the at least two input data are used as the at least two input data to obtain a subtraction result of the at least two input data.

[0255] For example, the two's complement of a positive integer is its binary representation, which is the same as the original code. For example, the 8-bit two's complement of +4 is 00000100, which is the same as the original code.

[0256] The complement of a negative number is to invert all bits of the original code except the sign bit. Specifically, 0 is changed to 1, 1 is changed to 0, the sign bit remains unchanged at 1, and then 1 is added to the inverted bit.

[0257] For example, the 8-bit binary code of the positive number 1 corresponding to -1 is 00000001, and the 8-bit binary code after all bits are inverted is 11111110. After adding 1, the 8-bit binary complement of -1 is 111111111.

[0258] The result of the operation + 4-1 is 00000100+11111111, which is 0000011, which corresponds to 3 in decimal.

[0259] The binary subtractor can be realized by the above complement method.

[0260] In some embodiments, a multiplier may be implemented by combining the above logic operations.

[0261] Specifically, by superimposing and using full adder operation units, a multiplication result for at least two input data is obtained.

[0262] It should be noted that two-variable calculations of addition, subtraction, and multiplication in different bases can be combined and used in the same neural network, compared to the original neural network model which only has convolution calculations. The embodiments of the present application can greatly expand the versatility of neural network calculations and enrich the functions of neural networks. And it can encapsulate all neural network reasoning hardware (no longer just dedicated neural network computing hardware) into a general-purpose computing chip for use.

[0263] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a data processing device, computing equipment, chip, board and corresponding embodiments based on convolutional neural network.

[0264] Figure 11 It is a structural diagram of a data processing device based on a convolutional neural network shown in an embodiment of the present application.

[0265] See also Figure 11 A data processing device 1100 based on a convolutional neural network is applied to a processor. The data processing device 1100 based on a convolutional neural network includes: a pre-processing module 1110, a calculation module 1120, and a post-processing module 1130.

[0266] The preprocessing module 1110 is configured to preprocess at least two input data to obtain preprocessed data corresponding to each of the at least two input data. At least one of the at least two input data has a length of M. The preprocessing module 1110 can convert an input serial data stream into a matrix data block through a concatenation operation. In other words, the input data can be an input serial data stream, and the preprocessed data can be a matrix data block. A serial data stream is a data stream consisting of "0s" and "1s," such as 110010101, 101001110, etc.

[0267] The operation module 1120 is used to process at least two preprocessed data through the convolution calculation layer and activation function layer of at least M full adder operation units in the convolutional neural network to obtain operation data of the M full adder operation units. The operation module 1120 may include multiple full adders connected in series.

[0268] The post-processing module 1130 is configured to post-process the operation data obtained by the operation module 1120 to obtain output data. For example, the post-processing module 1130 may convert the shift operation data in the matrix data block format into a serial data stream.

[0269] The device provided in this application is applied to a processor. After the processor preprocesses input data to obtain preprocessed data, the preprocessed data is input into the convolution calculation layer and activation function layer of at least M full adder operation units for processing to obtain an addition operation result. The addition operation result is then converted into a serial data stream. In this way, the addition task is executed on an operating processing device adapted to the neural network structure, thereby improving hardware resource utilization, improving computing efficiency, and reducing latency.

[0270] Figure 12 This is another structural diagram of a data processing device based on a convolutional neural network shown in an embodiment of the present application.

[0271] See also Figure 12 A data processing device 1100 based on a convolutional neural network is applied to a processor. The data processing device 1100 based on a convolutional neural network includes a preprocessing module 1210, an operation module 1220, and a post-processing module 1230. The operation module 1220 includes an addition branch 1221 and a carry branch 1222.

[0272] Specifically, the addition branch 1221 includes an exclusive OR operation unit 12211, which is used to perform a first convolution processing on at least two-channel preprocessed data in the first convolution calculation layer to output double-channel convolution data; perform an OR operation on the double-channel convolution data through the OR operation unit and a NAND operation through the NAND operation unit; and perform an AND operation on the results of the OR operation and the NAND operation through the AND operation unit to obtain the operation data of the exclusive OR operation unit.

[0273] The carry branch 1222 includes a cascaded AND operation unit 12221 and a logic left shift unit 12222 .

[0274] Among them, the AND operation unit 12221 is used to perform a first convolution processing on at least two-channel preprocessed data in the first convolution calculation layer of the AND operation unit 12221 to output at least two-channel convolution data, and then process it through the activation function of the first activation function layer, perform a second convolution processing on the at least two-channel convolution data in the second convolution calculation layer to output single-channel convolution data, and process it through the activation function of the second activation function layer to obtain the operation data of the AND operation unit.

[0275] The logical left shift unit 12222 is used to convolve the preprocessed data of the first single channel in the first convolution calculation layer of the logical left shift unit 12222 to output dual-channel convolution data, and then sequentially pass the convolution data through the second convolution calculation layer, the third convolution calculation layer and the fourth convolution calculation layer for convolution processing, and then pass the convolution data channel merging processing through the fifth convolution calculation layer to output the convolution data of the second single channel as the left shift operation data of the logical left shift unit.

[0276] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0277] It should be understood that the above-described device embodiments are merely illustrative, and the device of the present application may also be implemented in other ways. For example, the division of units / modules in the above-described embodiments is merely a logical functional division, and actual implementations may employ other division methods. For example, multiple units, modules, or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0278] In addition, unless otherwise specified, the functional units / modules in the various embodiments of the present application may be integrated into a single unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The aforementioned integrated units / modules may be implemented in the form of hardware or software program modules.

[0279] If the integrated unit / module is implemented in hardware, the hardware may be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), and an ASIC (Application-Specific Integrated Circuit). Unless otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0280] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned memory includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0281] Figure 13 This is a structural block diagram of the artificial intelligence chip shown in an embodiment of the present application.

[0282] See also Figure 13 The present application also provides an artificial intelligence chip 1300, which includes the above-mentioned data processing device 1100 based on convolutional neural network. The structure of the data processing device 1100 based on convolutional neural network can be seen in Figure 11 and Figure 12 The artificial intelligence chip 1300 may be, for example, a neural network chip or other chip. The neural network chip may be, for example, a convolutional neural network inference chip, an ASIC chip, or the like.

[0283] The present application also provides a board card, which includes a storage device, an interface device, a control device and the above-mentioned artificial intelligence chip; wherein the artificial intelligence chip is connected to the storage device, the control device and the interface device respectively; the storage device is used to store data; the interface device is used to realize data transmission between the artificial intelligence chip and external equipment; the control device is used to monitor the status of the artificial intelligence chip.

[0284] Figure 14 This is a block diagram of the board shown in the embodiment of the present application, see Figure 14 In addition to the artificial intelligence chip 1489, the board may also include other supporting components, including but not limited to: a storage device 1490, an interface device 1491, and a control device 1492;

[0285] Memory device 1490 is connected to artificial intelligence chip 1489 via a bus for storing data. Memory device 1490 may include multiple groups of memory cells 1493. Each group of memory cells 1493 is connected to artificial intelligence chip 1489 via a bus. It is understood that each group of memory cells 1493 may be DDR SDRAM (Double Data Rate SDRAM).

[0286] DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read out on the rising and falling edges of the clock pulse. The speed of DDR is twice that of standard SDRAM. In one embodiment, the storage device may include 4 groups of storage units 1493. Each group of storage units 1493 may include multiple DDR4 particles (chips). In one embodiment, the artificial intelligence chip 1489 may include 4 72-bit DDR4 controllers, and 64 bits of the above 72-bit DDR4 controllers are used for data transmission and 8 bits are used for ECC verification. It can be understood that when DDR4-3200 particles are used in each group of storage units 1493, the theoretical bandwidth of data transmission can reach 25600MB / s.

[0287] In one embodiment, each group of memory cells 1493 includes multiple parallel-connected double-bit rate synchronous dynamic random access memories (DDRs). DDRs can transmit data twice within a single clock cycle. A DDR controller is provided within the chip to control data transmission and storage for each memory cell 1493.

[0288] The interface device 1491 is electrically connected to the artificial intelligence chip 1489. The interface device 1491 is used to realize data transmission between the artificial intelligence chip 1489 and an external device (such as a server or a computer). For example, in one embodiment, the interface device 1491 can be a standard PCIE interface. For example, the data to be processed is transmitted to the chip by the server through the standard PCIE interface to realize data transfer. Preferably, when the PCIE 3.0X 16 interface is used for transmission, the theoretical bandwidth can reach 16000MB / s. In another embodiment, the interface device 1491 can also be other interfaces. This application does not limit the specific form of expression of the above-mentioned other interfaces. The interface unit can realize the switching function. In addition, the calculation results of the artificial intelligence chip 1489 are still transmitted back to the external device (such as a server) by the interface device.

[0289] The control device 1492 is electrically connected to the artificial intelligence chip 1489. The control device 1492 is used to monitor the status of the artificial intelligence chip 1489. Specifically, the artificial intelligence chip 1489 and the control device 1492 can be electrically connected via an SPI interface. The control device 1492 may include a single-chip microcomputer (MCU). The artificial intelligence chip 1489 may include multiple processing chips, multiple processing cores, or multiple processing circuits, which can drive multiple loads. Therefore, the artificial intelligence chip 1489 can be in different working states such as multi-load and light load. The control device 1492 can realize the regulation of the working states of multiple processing chips, multiple processing and / or multiple processing circuits in the artificial intelligence chip.

[0290] In one possible implementation, the present application further provides a computing device comprising the aforementioned artificial intelligence chip. The computing device includes a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigation system, a sensor, a camera, a server, a cloud server, a camera, a video camera, a projector, a watch, headphones, mobile storage, a wearable device, a vehicle, a household appliance, and / or a medical device. Vehicles include aircraft, ships, and / or cars; household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; and medical devices include magnetic resonance imaging (MRI) machines, ultrasound machines, and / or electrocardiographs.

[0291] Figure 15It is a structural diagram of a computing device shown in an embodiment of the present application.

[0292] See also Figure 15 , the computing device 1500 includes a memory 1510 and a processor 1520 .

[0293] The processor 1520 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0294] Memory 1510 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. ROM may store static data or instructions required by processor 1520 or other modules of the computer. Permanent storage may be a readable and writable storage device. A permanent storage device may be a non-volatile storage device that retains stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (e.g., a magnetic or optical disk, flash memory) as the permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (e.g., a floppy disk, optical drive). System memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. System memory may store some or all instructions and data required by the processor during operation. In addition, memory 1510 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. In some embodiments, the memory 1510 may include a readable and / or writable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0295] The memory 1510 stores executable codes. When the executable codes are processed by the processor 1520 , the processor 1520 may execute part or all of the above-mentioned methods.

[0296] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0297] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0298] The embodiments of the present application have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein.

Claims

1. A data processing method based on convolutional neural network, characterized in that: Applied to a processor, the processor being applied to a neural network dedicated chip, the method comprising: Preprocessing at least two input data respectively to obtain preprocessed data corresponding to the at least two input data, wherein at least one of the at least two input data has a length of M; wherein the input data comprises a serial data stream, the preprocessing comprises a conversion process, and the preprocessed data comprises data after the conversion process; In a convolutional neural network, at least two preprocessed data are processed through a convolution calculation layer and an activation function layer of at least M full adder operation units to obtain operation data of the M full adder operation units; the M full adder operation units are connected in series, and each full adder operation unit includes an addition branch and a carry branch; wherein the preprocessed data are convolved by the convolution calculation layer to obtain convolution data, and the convolution data are subjected to a logical operation and processed through an activation function of the activation function layer to obtain operation data of the M full adder operation units; Post-processing the operation data to obtain output data; The step of preprocessing the at least two input data to obtain preprocessed data corresponding to the at least two input data comprises: converting the at least two serial data streams into at least two matrix data blocks through a splicing operation; The post-processing of the operation data to obtain output data includes: converting the operation data in a matrix data block format into a serial data stream.

2. The method according to claim 1, wherein: The method of processing at least two pre-processed data through a convolutional computing layer and an activation function layer of at least M full adder computing units in a convolutional neural network to obtain computing data of the M full adder computing units comprises: for each full adder computing unit, Performing a first convolution processing on at least two-channel preprocessed data in a first convolution calculation layer to output double-channel convolution data; Performing a logical operation on the first single-channel convolution data in the double-channel convolution data through the addition branch to obtain a first operator result of the first single-channel, and performing a logical operation on the second single-channel convolution data in the double-channel convolution data through the carry branch and then processing it through the activation function of the activation function layer to obtain a second operator result of the second single-channel; The first operator result and the second operator result are used as input data of a full adder operation unit at a next stage of the current full adder operation unit.

3. The method according to claim 2, wherein: The logic operation performed by the addition branch includes a logic operation performed by an exclusive OR operation unit, and the exclusive OR operation unit is used to perform a first convolution process on at least two channel preprocessed data in a first convolution calculation layer to output double channel convolution data; Performing an OR operation on the double channel convolution data through an OR operation unit and performing a NAND operation on the double channel convolution data through a NAND operation unit; The results of the OR operation and the NAND operation are respectively subjected to an AND operation by the AND operation unit to obtain operation data of the XOR operation unit.

4. The method according to claim 3, wherein: The performing of the OR operation by the OR operation unit includes: performing a first convolution operation on the at least two-channel pre-processed data in a first convolution operation layer of the OR operation unit to output single-channel convolution data, processing the data through an activation function of an activation function layer, and then performing a second convolution operation on the second convolution operation layer to output single-channel convolution data, thereby obtaining operation data of the OR operation unit; and\or The performing of the NAND operation by the NAND operation unit includes: after the first convolution calculation layer of the NAND operation unit performs a first convolution processing on at least two-channel preprocessed data to output at least two-channel convolution data, processing is performed through the activation function of the first activation function layer, performing a second convolution processing on the at least two-channel convolution data in the second convolution calculation layer to output single-channel convolution data, and processing is performed through the activation function of the second activation function layer to obtain the operation data of the NAND operation unit.

5. The method according to claim 2, wherein: The carry branch includes: a cascaded AND operation unit and a logic left shift unit, Among them, the logical left shift unit is used to convolve the preprocessed data of the first single channel in the first convolution calculation layer to output dual-channel convolution data, and then sequentially pass the convolution processing through the second convolution calculation layer, the third convolution calculation layer and the fourth convolution calculation layer, and then pass the convolution data channel merging processing through the fifth convolution calculation layer to output the convolution data of the second single channel as the left shift operation data of the logical left shift unit.

6. The method according to claim 5, characterized in that: The second convolution calculation layer, the third convolution calculation layer, and the fourth convolution calculation layer each include two different convolution kernels, wherein each convolution kernel includes two channel parameters.

7. The method according to claim 2, wherein: The input data is any one of binary data, octal data or hexadecimal data, wherein the convolution parameters of the carry branches corresponding to the binary data, the octal data or the hexadecimal data are different.

8. The method according to claim 1, wherein: For a serial data stream having a length of any positive integer and a non-square number, converting each of the at least two serial data streams into at least two matrix data blocks through a splicing operation includes: Padding zeros at the end of the serial data stream to obtain a zero-padded serial data stream having a length equal to the square of any positive integer, and then splicing the zero-padded serial data stream into a matrix data block; or Splice the data into matrix data blocks of varying lengths and widths; For a non-square serial data stream whose length is any positive integer, converting the operation data in the matrix data block format into the serial data stream includes: after obtaining the left-shift operation data, splicing and merging according to the principle of right-end data segment priority.

9. The method according to any one of claims 1 to 8, characterized in that Also includes: The number of bits of the full adder operation unit is increased by increasing the depth and dimension of the convolution calculation layer.

10. The method according to any one of claims 1 to 8, characterized in that: The M full adder operation units are connected in series, the input data is a decimal array, and each full adder operation unit includes an addition branch; The method of processing at least two pre-processed data through a convolutional computing layer and an activation function layer of at least M full adder computing units in a convolutional neural network to obtain computing data of the M full adder computing units comprises: for each full adder computing unit, Performing a first convolution processing on at least two-channel preprocessed data in a first convolution calculation layer to output double-channel convolution data; Performing a logic operation on the at least two-channel preprocessed data through an addition branch to obtain a third operator result; The third operator result is used as input data of the full adder operation unit at the next stage of the current full adder operation unit.

11. The method according to any one of claims 1 to 8, characterized in that Also includes: Before the at least two input data are respectively pre-processed, determining a complement for each of the at least two input data; The complements of the at least two input data are used as the at least two input data to obtain a subtraction result of the at least two input data.

12. The method according to any one of claims 1 to 8, characterized in that Also includes: By superimposing and using the full adder operation units, a multiplication result for the at least two input data is obtained.

13. A data processing device based on convolutional neural network, characterized in that: Applied to a processor, the processor being applied to a neural network dedicated chip, the device comprising: a preprocessing module, configured to preprocess at least two input data respectively to obtain preprocessed data corresponding to the at least two input data, wherein at least one of the at least two input data has a length of M; wherein the input data comprises a serial data stream, the preprocessing comprises a conversion process, and the preprocessed data comprises data after the conversion process; wherein the preprocessing module converts each of the at least two serial data streams into at least two matrix data blocks through a splicing operation; An operation module is used to process at least two preprocessed data through a convolution calculation layer and an activation function layer of at least M full adder operation units in a convolutional neural network to obtain operation data of the M full adder operation units; the M full adder operation units are connected in series, and each full adder operation unit includes an addition branch and a carry branch; wherein the preprocessed data is convolved by the convolution calculation layer to obtain convolution data, and the convolution data is subjected to a logical operation and processed by an activation function of the activation function layer to obtain operation data of the M full adder operation units; The post-processing module is used to post-process the operation data obtained by the operation module to obtain output data, including converting the operation data in the matrix data block format into a serial data stream.

14. An artificial intelligence chip, characterized in that: Including the data processing device based on convolutional neural network as described in claim 13.

15. A computing device, characterized in that include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to perform the method according to any one of claims 1 to 12. 16 . A computer-readable storage medium having executable codes stored thereon, wherein when the executable codes are executed by a processor of an electronic device, the processor is caused to execute the method according to claim 1 .

Images