A convolution operation circuit and a convolution operation method

Abstract

The present application provides a convolution operation circuit and a convolution operation method in the field of artificial intelligence, which are used to implement the convolution operation of the first matrix data and the second matrix data, including a splitting circuit and a matrix multiplication and accumulation circuit; wherein the splitting circuit is used to split the first matrix data to obtain the first sub-matrix data of the first N / 2 dimensions and the second sub-matrix data of the last N / 2 dimensions; split the second matrix data to obtain the third sub-matrix data of the first N / 2 dimensions and the fourth sub-matrix data of the last N / 2 dimensions; wherein the first matrix data and the second matrix data are both N-dimensional matrix data; N is a positive even number; the matrix multiplication and accumulation circuit is used to perform multiplication and accumulation operations according to the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data to obtain an operation result. The processor splits the convolution of two high-bit numbers into three low-bit number multiplication operations, which can reduce the area and power consumption of the processor.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a convolution operation circuit and a convolution operation method. Background Art

[0002] In computer technology, a convolutional neural network (CNN) is a multi-layered neural network. Currently, in CNNs, the convolution operation typically involves converting the convolution of input signal features with weights into a matrix multiplication between the signal matrix and the weight matrix, typically AxB([MxK]x[KxN]), where A represents the signal matrix (input matrix) and B represents the weight matrix. Matrix A is typically the input matrix extracted from the input data based on the kernel stride during convolution, i.e., the input matrix from which the input signal features are transformed.

[0003] Typically, both the input matrix and the weight matrix are high-dimensional. Implementing the convolution operation on these two high-dimensional matrices requires more processor space, which undoubtedly increases the processor's area and power consumption. Therefore, improving processor computational efficiency to reduce processor area and power consumption is a pressing technical challenge. Summary of the Invention

[0004] The present application provides a convolution operation circuit and a convolution operation method, which splits two high-digit convolution operations into three low-digit multiplications. Since two N / 2-dimensional multiplications are used to implement N-dimensional multiplication, the operation complexity is reduced by n / 2. 2 Reduce to n 1.585 , which can reduce the processor area and power consumption.

[0005] In a first aspect, a convolution operation circuit is provided, comprising a splitting circuit and a matrix multiplication and accumulation circuit; wherein the splitting circuit is used to symmetrically split the first matrix data to obtain first sub-matrix data of the first N / 2 dimensions and second sub-matrix data of the last N / 2 dimensions; here, the first sub-matrix data of the first N / 2 dimensions refers to the high-order portion of the first matrix data, and the second sub-matrix data of the last N / 2 dimensions refers to the low-order portion of the first matrix data; the second matrix data is symmetrically split to obtain third sub-matrix data of the first N / 2 dimensions and fourth sub-matrix data of the last N / 2 dimensions. Similarly, the third sub-matrix data of the first N / 2 dimensions refers to the high-order portion of the second matrix data, and the fourth sub-matrix data of the last N / 2 dimensions refers to the low-order portion of the second matrix data. wherein the first matrix data and the second matrix data are both N-dimensional matrix data; N is a positive even number; and here, the matrix data being N-dimensional means that the scale of the first matrix data and the second matrix data is relatively large. The matrix multiplication and accumulation circuit is used to perform multiplication and accumulation operations on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data to obtain an operation result.

[0006] In the embodiment of the present application, when implementing the convolution operation of two high-order digits, the operation circuit symmetrically splits the two N-dimensional high-order digits to obtain four corresponding N / 2-dimensional low-order digits. Then, three multiplication operations and accumulation operations are performed on these four N / 2-dimensional low-order digits. Since the N-dimensional multiplication operation is implemented by two N / 2-dimensional multiplications, the operation complexity is reduced by n 2 Reduce to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0007] In one possible implementation, the above-mentioned matrix multiplication and accumulation circuit may include a matrix multiplication circuit and an accumulation circuit; wherein the matrix multiplication circuit is used to perform a multiplication operation based on the first sub-matrix data and the third sub-matrix data to obtain the first intermediate data; perform a multiplication operation based on the second sub-matrix data and the fourth sub-matrix data to obtain the second intermediate data; perform a multiplication operation based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data to obtain the third intermediate data. Here, in the three multiplication operations, the N-dimensional multiplication operation is implemented by two N / 2-dimensional multiplications. The accumulation circuit is used to accumulate the first intermediate data, the second intermediate data and the third intermediate data to obtain the operation result. In implementing the embodiment of the present application, when implementing two high-bit convolution operations, the N-dimensional multiplication operation can be implemented by two N / 2-dimensional multiplications through the matrix multiplication circuit and the accumulation circuit, and its operation complexity is reduced by n. 2 Reduce to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0008] In one possible implementation, the matrix multiplication circuit includes a first matrix multiplication circuit, a second matrix multiplication circuit, and a third matrix multiplication circuit; wherein the first matrix multiplication circuit is used to perform a multiplication operation on the first sub-matrix data and the third sub-matrix data to obtain first sub-intermediate data, and to shift the first sub-intermediate data left by N bits to obtain first intermediate data; the second matrix multiplication circuit is used to perform a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; the third matrix multiplication circuit is used to perform an accumulation operation on the first sub-matrix data and the second sub-matrix data to obtain first sum matrix data, to perform an accumulation operation on the third sub-matrix data and the fourth sub-matrix data to obtain second sum matrix data, and to perform a multiplication operation on the first sum matrix data and the second sum matrix data to obtain fourth intermediate data; after accumulating the first sub-intermediate data, the second intermediate data, and the fourth intermediate data, the first sub-intermediate data, the second intermediate data, and the fourth intermediate data are shifted left by N / 2 bits to obtain third intermediate data. Here, the dimensions of the first intermediate data, the second intermediate data, and the third intermediate data can be N or N-1. In the three multiplication operations, two N / 2-dimensional multiplications are used to implement the N-dimensional multiplication operation. When implementing the embodiment of the present application, when implementing two high-bit convolution operations, three matrix multiplication circuits can be used to implement the N-dimensional multiplication operation using two N / 2-dimensional multiplications respectively. The operation complexity is reduced by n 2 Reduce to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0009] In one possible implementation, the splitting circuit is further configured to divide the first large matrix data and the second large matrix data into blocks, respectively obtaining m first matrix blocks and m second matrix blocks; wherein the i-th matrix block in the m first matrix blocks serves as the first matrix data, and the i-th matrix block in the m second matrix blocks serves as the second matrix data; wherein i is sequentially assigned values ​​from 1 to m to obtain m groups of first matrix data and second matrix data. Here, the first matrix block refers to a portion of the first large matrix data; similarly, the second matrix block refers to a portion of the second large matrix data. The matrix multiplication and accumulation circuit is configured to perform the multiplication and accumulation operations based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data for each group of the first matrix data and the second matrix data, to obtain m intermediate operation results. In other words, for each data block obtained by splitting the first large matrix data and the second large matrix data, an intermediate operation result corresponding to each group of two data blocks can be obtained, and then these m intermediate operation results are accumulated to obtain the operation result. In implementing the embodiment of the present application, when implementing the convolution operation of two large matrix data, the large matrix data can be divided into blocks, and then the intermediate operation results corresponding to each group of first matrix data and second matrix data are calculated by the matrix multiplication accumulation circuit. After that, the multiple intermediate operation results are accumulated to obtain the operation result. For any multiplication operation, since two N / 2-dimensional multiplications are used to implement the N-dimensional multiplication operation, its operation complexity is reduced by n / 2. 2 Reduce to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0010] In one possible implementation, the splitting circuit is further configured to divide the first large matrix data and the second large matrix data into blocks, respectively obtaining m first matrix blocks and m second matrix blocks. The i-th matrix block in the m first matrix blocks serves as the first matrix vector, and the i-th matrix block in the m second matrix blocks serves as the second matrix vector; wherein i is sequentially assigned values ​​from 1 to m to obtain m groups of first matrix vectors and second matrix vectors. Here, the first matrix block refers to a portion of the first large matrix data; similarly, the second matrix block refers to a portion of the second large matrix data. A matrix multiplication and accumulation circuit is configured to perform multiplication and accumulation operations based on the first sub-matrix vector, the second sub-matrix vector, the third sub-matrix vector, and the fourth sub-matrix vector for each group of the first matrix vector and the second matrix vector, to obtain m intermediate operation results. That is, for each data block obtained by splitting the first large matrix data and the second large matrix data, an intermediate operation result corresponding to each group of two data blocks can be obtained. These m intermediate operation results are then accumulated to obtain an operation result. In implementing the embodiment of the present application, when implementing the convolution operation of two large matrix data, the large matrix data can be divided into blocks, and then the vectors corresponding to each group of first matrix data and second matrix data are extracted, and then the intermediate operation results corresponding to each group of first matrix vectors and second matrix vectors are calculated by the matrix multiplication accumulation circuit, and then the multiple intermediate operation results are accumulated to obtain the operation result. For any of the multiplication operations, since two N / 2-dimensional multiplications are used to implement the N-dimensional multiplication operation, its operation complexity is reduced by n / 2. 2 Reduce to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0011] In one possible implementation, the first matrix data is a signed number; the first sub-matrix data and the second sub-matrix data both include a sign bit. That is, the first and second sub-matrix data obtained by splitting are composed of two parts: one part is the high-order data or the low-order data itself; the other part is the sign bit. The second matrix data is a signed number; the third and fourth sub-matrix data both include a sign bit. Similarly, the third and fourth sub-matrix data obtained by splitting are composed of two parts: one part is the high-order data or the low-order data itself; the other part is the sign bit. The operation circuit also includes an OR logic judgment circuit connected to the splitting circuit; wherein the OR logic judgment circuit is configured to perform an addition operation on the first sub-matrix data if the non-sign bits of the second sub-matrix data are not all 0. Here, the addition operation on the first sub-matrix data can produce a corresponding two's complement form. If the non-sign bits of the fourth sub-matrix data are not all 0, the addition operation on the third sub-matrix data can be performed. Similarly, the addition operation on the third sub-matrix data can produce a corresponding two's complement form. In the implementation of the embodiment of the present application, when the two high-order digits involved in the convolution operation are signed numbers, the operation circuit can convert the signed numbers into the corresponding binary complement form through the OR logic judgment circuit in the operation circuit, and then perform three multiplication operations and accumulation operations on the four N / 2-dimensional low-order digits obtained by splitting. By converting the signed numbers through the OR logic judgment circuit, errors in the convolution operation can be avoided. In addition, for any of the multiplication operations, since two N / 2-dimensional multiplications are used to implement the N-dimensional multiplication operation, its operation complexity is reduced by n. 2 Reduced to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0012] In a second aspect, an embodiment of the present application provides a convolution operation method, which is applied to a convolution operation circuit, the operation circuit including a splitting circuit and a matrix multiplication and accumulation circuit, and the method includes the following steps: splitting the first matrix data by the splitting circuit in the convolution operation circuit to obtain the first sub-matrix data of the first N / 2 dimensions and the second sub-matrix data of the last N / 2 dimensions; here, the first sub-matrix data of the first N / 2 dimensions refers to the high-order part of the first matrix data, and the second sub-matrix data of the last N / 2 dimensions refers to the low-order part of the first matrix data; splitting the second matrix data to obtain the third sub-matrix data of the first N / 2 dimensions and the fourth sub-matrix data of the last N / 2 dimensions. Similarly, the third sub-matrix data of the first N / 2 dimensions refers to the high-order part of the second matrix data, and the fourth sub-matrix data of the last N / 2 dimensions refers to the low-order part of the second matrix data. Wherein, the first matrix data and the second matrix data are both N-dimensional matrix data; N is a positive even number; here, the matrix data is N-dimensional, which means that the scale of the first matrix data and the second matrix data is relatively large. The matrix multiplication and accumulation circuit in the above operation circuit performs multiplication and accumulation operations on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data to obtain an operation result.

[0013] In the embodiment of the present application, when implementing the convolution operation of two high-order digits, the two N-dimensional high-order digits are symmetrically split to obtain four corresponding N / 2-dimensional low-order digits. After that, three multiplication operations and accumulation operations are performed on these four N / 2-dimensional low-order digits. Since the N-dimensional multiplication operation is implemented by two N / 2-dimensional multiplications, the operation complexity is reduced by n 2 Reduced to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0014] In one possible implementation, a matrix multiplication and accumulation circuit includes a matrix multiplication circuit and an accumulation circuit; the matrix multiplication and accumulation circuit in the operation circuit performs a multiplication and accumulation operation based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain an operation result, including: performing a multiplication operation based on the first sub-matrix data and the third sub-matrix data by the matrix multiplication circuit to obtain first intermediate data; performing a multiplication operation based on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; performing a multiplication operation based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain third intermediate data. Here, in each of the three multiplication operations, the N-dimensional multiplication operation is implemented by two N / 2-dimensional multiplications. The first intermediate data, the second intermediate data, and the third intermediate data are accumulated by the accumulation circuit to obtain the operation result.

[0015] In a possible implementation, the matrix multiplication circuit includes a first matrix multiplication circuit, a second matrix multiplication circuit, and a third matrix multiplication circuit; performing a multiplication operation on the first sub-matrix data and the third sub-matrix data through the matrix multiplication circuit to obtain first intermediate data; performing a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; performing a multiplication operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain third intermediate data, including: performing a multiplication operation on the first sub-matrix data and the third sub-matrix data through the first matrix multiplication circuit to obtain first intermediate data, The first sub-intermediate data is shifted left by N bits to obtain the first intermediate data; the second sub-matrix data and the fourth sub-matrix data are multiplied by the second matrix multiplication circuit to obtain the second intermediate data; the first sub-matrix data and the second sub-matrix data are accumulated by the third matrix multiplication circuit to obtain the first sum matrix data, the third sub-matrix data and the fourth sub-matrix data are accumulated to obtain the second sum matrix data, and the first sum matrix data and the second sum matrix data are multiplied to obtain the fourth intermediate data; after accumulating the above-mentioned first sub-intermediate data, the above-mentioned second intermediate data and the above-mentioned fourth intermediate data, they are shifted left by N / 2 bits to obtain the third intermediate data. Here, the dimensions of the first intermediate data, the second intermediate data and the third intermediate data can be N or N-1. In the three multiplication operations, the N-dimensional multiplication operation is realized by two N / 2-dimensional multiplications.

[0016] In one possible implementation, the method further includes: dividing the first large matrix data and the second large matrix data into blocks using a splitting circuit to obtain m first matrix blocks and m second matrix blocks, respectively; wherein the i-th matrix block in the m first matrix blocks serves as the first matrix data, and the i-th matrix block in the m second matrix blocks serves as the second matrix data; wherein i is sequentially taken from 1 to m to obtain m groups of first matrix data and second matrix data. Here, the first matrix block refers to a portion of the first large matrix data, and similarly, the second matrix block refers to a portion of the second large matrix data. A matrix multiplication and accumulation circuit in the operation circuit performs a multiplication and accumulation operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain an operation result, including: performing the multiplication and accumulation operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data for each group of the first matrix data and the second matrix data using the matrix multiplication and accumulation circuit to obtain m intermediate operation results. That is to say, for the data blocks obtained by splitting the first large matrix data and the second large matrix data, intermediate calculation results corresponding to each group of two data blocks can be obtained, and then these m intermediate calculation results are accumulated to obtain the calculation result.

[0017] In one possible implementation, the method further includes: dividing the first large matrix data and the second large matrix data into blocks by a splitting circuit to obtain m first matrix blocks and m second matrix blocks, respectively; wherein the i-th matrix block in the m first matrix blocks serves as the first matrix vector, and the i-th matrix block in the m second matrix blocks serves as the second matrix vector; wherein i takes values ​​from 1 to m in sequence to obtain m groups of first matrix vectors and second matrix vectors. Here, the first matrix block refers to a portion of the first large matrix data, and similarly, the second matrix block refers to a portion of the second large matrix data. A matrix multiplication and accumulation circuit in the operation circuit performs a multiplication and accumulation operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain an operation result, including: performing a multiplication and accumulation operation on each group of the first matrix vector and the second matrix vector by the matrix multiplication and accumulation circuit according to the first sub-matrix vector, the second sub-matrix vector, the third sub-matrix vector, and the fourth sub-matrix vector to obtain m intermediate operation results. That is to say, for the data blocks obtained by splitting the first large matrix data and the second large matrix data, intermediate calculation results corresponding to each group of two data blocks can be obtained, and then these m intermediate calculation results are accumulated to obtain the calculation result.

[0018] In one possible implementation, the first matrix data is a signed number; the first sub-matrix data and the second sub-matrix data both include a sign bit; that is, the first sub-matrix data and the second sub-matrix data obtained by splitting are composed of two parts, one of which is the high-order data or the low-order data itself; and the other is the sign bit. The second matrix data is a signed number; the third sub-matrix data and the fourth sub-matrix data both include a sign bit; similarly, the third sub-matrix data and the fourth sub-matrix data obtained by splitting are composed of two parts, one of which is the high-order data or the low-order data itself; and the other is the sign bit. The method also includes: using an OR logic judgment circuit to determine whether the non-sign bits of the second sub-matrix data are not all 0, adding 1 to the first sub-matrix data; and adding 1 to the third sub-matrix data when the non-sign bits of the fourth sub-matrix data are not all 0. Here, adding 1 to the first sub-matrix data can obtain the corresponding binary complement form.

[0019] In a third aspect, an embodiment of the present application provides a chip comprising the convolution operation circuit provided in the first aspect and at least one vector calculation circuit coupled to the operation circuit; the at least one vector calculation circuit is used for other layer network structures in the convolutional neural network to perform calculations based on the operation results to obtain recognition results.

[0020] In a fourth aspect, an embodiment of the present application provides a board card, which includes the chip provided in the third aspect and at least one storage device and a control device coupled to the above chip; the above at least one storage device is used to store calculation data of the convolutional neural network; the control device is used to communicate with the chip to realize the operation of the convolutional neural network.

[0021] In a fifth aspect, an embodiment of the present application provides an electronic device, which includes the board provided in the fourth aspect.

[0022] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of a convolutional neural network structure provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of a convolutional neural network structure provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of converting a three-dimensional convolution kernel into a two-dimensional convolution kernel using GEMM provided in an embodiment of the present application;

[0026] Figure 4a A specific implementation scenario of a convolutional neural network provided in an embodiment of the present application;

[0027] Figure 4b Another specific implementation scenario of a convolutional neural network provided in an embodiment of the present application;

[0028] Figure 5a A schematic diagram of the hardware structure of an artificial intelligence processor provided in an embodiment of the present application;

[0029] Figure 5b A schematic diagram of the hardware structure of an artificial intelligence processor provided in an embodiment of the present application;

[0030] Figure 5c A schematic diagram of the hardware structure of an arithmetic circuit provided in an embodiment of the present application;

[0031] Figure 5d A schematic diagram of the hardware structure of another artificial intelligence processor provided in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of the structure of a vector computing unit in an artificial intelligence processor provided by this application;

[0033] Figure 7a A schematic diagram of a convolution operation method provided in an embodiment of the present application;

[0034] Figure 7b A schematic diagram of a symmetrical splitting method provided in an embodiment of the present application;

[0035] Figure 7c A schematic diagram of a convolution operation provided in an embodiment of the present application;

[0036] Figure 7d A block diagram provided for an embodiment of the present application;

[0037] Figure 7e A schematic diagram of splitting matrix data of signed numbers provided in an embodiment of the present application;

[0038] Figure 8a A schematic diagram of the structure of a board provided in an embodiment of the present application;

[0039] Figure 8b A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0041] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, rather than to describe a specific order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0042] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0043] As used in this specification, the terms "component," "module," "system," and the like are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0044] To facilitate a better understanding of the technical solutions described in this application, the following explains the technical terms involved in the embodiments of this application:

[0045] (1) Convolutional neural network.

[0046] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. It is a deep learning architecture, which uses machine learning algorithms to perform multiple levels of learning at different levels of abstraction. As a deep learning architecture, a CNN is a feed-forward artificial neural network in which individual neurons respond to overlapping regions in the input image.

[0047] like Figure 1 As shown, a convolutional neural network (CNN) 100 may include an input layer 110, a convolutional layer / pooling layer 120, wherein the pooling layer is optional, and a neural network layer 130. The following is a detailed description of each layer:

[0048] Convolutional layer / pooling layer 120:

[0049] Convolutional layer:

[0050] like Figure 1The convolutional layer / pooling layer 120 shown may include layers 121-126, for example. In one implementation, layer 121 is a convolutional layer, layer 122 is a pooling layer, layer 123 is a convolutional layer, layer 124 is a pooling layer, layer 125 is a convolutional layer, and layer 126 is a pooling layer. In another implementation, layers 121 and 122 are convolutional layers, layer 123 is a pooling layer, layers 124 and 125 are convolutional layers, and layer 126 is a pooling layer. That is, the output of a convolutional layer can be used as the input of a subsequent pooling layer, or as the input of another convolutional layer to continue the convolution operation.

[0051] Taking convolution layer 121 as an example, convolution layer 121 can include multiple convolution operators, also known as kernels. In image processing, a convolution operator acts as a filter that extracts specific information from the input image matrix. A convolution operator is essentially a weight matrix, which is typically predefined. During the convolution operation, the weight matrix is ​​typically applied horizontally to the input image, pixel by pixel (or two pixels by two pixels, depending on the stride), to extract specific features from the image. The size of this weight matrix is ​​related to the size of the image. It is important to note that the depth dimension of the weight matrix is ​​the same as the depth dimension of the input image. During the convolution operation, the weight matrix extends across the entire depth of the input image. Therefore, convolution with a single weight matrix produces a convolution output with a single depth dimension. However, in most cases, a single weight matrix is ​​not used, and multiple weight matrices of the same dimension are applied. The outputs of each weight matrix are stacked to form the depth dimension of the convolved image. Different weight matrices can be used to extract different features from an image. For example, one weight matrix is ​​used to extract edge information, another weight matrix is ​​used to extract specific colors from an image, and another weight matrix is ​​used to blur unwanted noise in an image. These multiple weight matrices have the same dimensions, and the feature maps extracted using these weight matrices also have the same dimensions. The extracted feature maps are then merged to form the output of the convolution operation.

[0052] The weight values ​​in these weight matrices need to be obtained through a lot of training in practical applications. The weight matrices formed by the weight values ​​obtained through training can extract information from the input image, thereby helping the convolutional neural network 100 to make correct predictions.

[0053] When the convolutional neural network 100 has multiple convolutional layers, the initial convolutional layer (for example, 121) often extracts more general features, which can also be called low-level features. As the depth of the convolutional neural network 100 increases, the features extracted by the later convolutional layers (for example, 126) become more and more complex, such as high-level semantic features. Features with higher semantics are more suitable for the problem to be solved.

[0054] Pooling layer:

[0055] Since it is often necessary to reduce the number of training parameters, it is often necessary to periodically introduce a pooling layer after the convolution layer, such as Figure 1 The layers 121-126 illustrated in Figure 120 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. In the image processing process, the sole purpose of the pooling layer is to reduce the spatial size of the image. The pooling layer can include an average pooling operator and / or a maximum pooling operator to sample the input image to obtain a smaller image. The average pooling operator can calculate the average value of the pixel values ​​in the image within a specific range. The maximum pooling operator can take the pixel with the largest value within a specific range as the result of maximum pooling. In addition, just as the size of the weight matrix used in the convolutional layer should be related to the image size, the operators in the pooling layer should also be related to the image size. The size of the image output after processing by the pooling layer can be smaller than the size of the image input to the pooling layer. Each pixel in the image output by the pooling layer represents the average value or maximum value of the corresponding sub-region of the image input to the pooling layer.

[0056] Neural Network Layer 130:

[0057] After being processed by the convolution layer / pooling layer 120, the convolution neural network 100 is not sufficient to output the required output information. As mentioned above, the convolution layer / pooling layer 120 only extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (the required class information or other related information), the convolution neural network 100 needs to use the neural network layer 130 to generate one or a group of outputs of the required number of classes. Therefore, the neural network layer 130 may include multiple hidden layers (such as Figure 1 131, 132 to 13n) and the output layer 140 shown, the parameters contained in the multi-layer hidden layer can be pre-trained according to relevant training data of a specific task type, for example, the task type may include image recognition, image classification, image super-resolution reconstruction, etc.

[0058] After the multiple hidden layers in the neural network layer 130, that is, the last layer of the entire convolutional neural network 100 is the output layer 140, which has a loss function similar to the classification cross entropy, specifically for calculating the prediction error. Once the forward propagation of the entire convolutional neural network 100 (such as Figure 3 The propagation from 110 to 140 is forward propagation), and the reverse propagation (such as Figure 3 The propagation from 140 to 110 is back propagation) and then starts to update the weight values ​​and biases of the aforementioned layers to reduce the loss of the convolutional neural network 100 and the error between the result output by the convolutional neural network 100 through the output layer and the ideal result.

[0059] It should be noted that if Figure 1 The convolutional neural network 100 shown is only an example of a convolutional neural network. In specific applications, the convolutional neural network can also exist in the form of other network models, such as Figure 2 The multiple convolutional layers / pooling layers shown are operated in parallel, and the features extracted from each layer are input to the neural network layer 130 for processing.

[0060] As mentioned above, there are usually multiple convolution kernels in a convolutional neural network, and these multiple convolution kernels are often three-dimensional, containing three-dimensional data. The x and y directions are the length and width of the data, and the z direction can be considered as the depth of the data. In practical applications, the three-dimensional convolution kernel can be converted into a two-dimensional convolution kernel through matrix-matrix multiplication (GEMM). For details, see Figure 3 , is a schematic diagram of using GEMM to reduce the dimensionality of a three-dimensional convolution kernel provided in an embodiment of the present application.

[0061] The following describes by way of example the application scenarios to which the convolutional neural network 100 can be applied.

[0062] First application scenario:

[0063] In the embodiment of the present application, the convolutional neural network 100 can be applied to various electronic devices. In a specific implementation scenario, such as Figure 4aAs shown, smartphones 302 and 304 have built-in processors associated with convolutional neural network 100. Mobile smartphone client 301 initiates a voice call to mobile smartphone client 305. The voice signal is transmitted by smartphone 302 and forwarded to smartphone 304 via base station 303. Due to a sudden downpour accompanied by strong lightning and thunder when the voice call is initiated, input signal 306 is severely weakened and contains significant noise. This input signal can be a one-dimensional digital voice signal. Smartphone 304 is equipped with convolutional neural network 100, which can be implemented in the form of a dedicated circuit on a chip or as program instructions running on a central processing unit (CPU) or other processor. Input signal 306 is processed by the convolutional neural network in smartphone 304, including noise removal and effective signal enhancement, to produce output signal 307. This output signal fully preserves the voice information transmitted by the caller and avoids interference from the harsh natural environment.

[0064] Second application scenario:

[0065] The embodiment of the present application provides another specific implementation scenario of the convolutional neural network 100, such as Figure 4b As shown, a car 403 is traveling at high speed on a road. A passerby 401 uses a digital camera 402 to capture the license plate of car 403. However, due to the high speed v of car 403, the input signal 404 of the digital camera is motion-blurred. This input signal is a two-dimensional digital image signal. Digital camera 402 is equipped with a convolutional neural network 100, which can be implemented in the form of a dedicated circuit on a chip or as a software module running in an image signal processor. After input signal 404 is processed by the convolutional neural network in digital camera 402, including car motion model estimation and motion blur removal, an output signal 405 is generated. The license plate information contained in the output signal is more clear, allowing for accurate recognition.

[0066] As previously mentioned, convolutional neural networks, which are widely used in fields such as image recognition and audio recognition, often require a large number of matrix multiplication operations. Executing matrix multiplication operations requires very high memory bandwidth and a large amount of computation. To fully utilize the processing power of the hardware, in the embodiments of this application, the convolution operation is optimized based on the Karatsuba algorithm to increase the computation speed.

[0067] In the embodiment of the present application, the Karatsuba algorithm is a fast multiplication algorithm. It reduces the complexity of the multiplication operation of two n-digit numbers from n to n. 2 Reduce to Therefore, it is computationally faster than traditional algorithms.

[0068] In the embodiment of the present application, the theoretical basis for implementing the convolution operation of two high-bit numbers based on the Karatsuba algorithm is reflected in the following: the convolution operation can be understood as multiplication of two numbers and then accumulating the products one by one.

[0069] In the embodiment of the present application, the implementation process of performing a convolution operation on two high-order digits can be described as follows: Based on the Karatsuba algorithm, the convolution of the two high-order digits is represented as three low-order digit multiplications. The first low-order digit multiplication is reflected in: performing a multiplication operation on the first N / 2 dimensional data obtained by splitting the two high-order digits; the second low-order digit multiplication is reflected in: performing a multiplication operation on the second N / 2 dimensional data obtained by splitting the two high-order digits; and the third low-order digit multiplication is reflected in: performing a multiplication operation on the sum of the two high-order digits.

[0070] In the embodiment of the present application, a high-order bit refers to a type of data whose bit width is greater than a first preset threshold. For example, the bit width of the first matrix data is N (N=100). When N is greater than the first preset threshold (for example, the first preset threshold is 70), the first matrix data can be considered to be a high-order bit. Then, in contrast, a low-order bit refers to a type of data whose bit width is less than the first preset threshold. For example, the bit width of the first sub-matrix data is N / 2 (N=100). When N / 2 is less than the first preset threshold (for example, the first preset threshold is 70), the first sub-matrix data can be considered to be a low-order bit.

[0071] In some implementations, the convolution operation based on the Karatsuba algorithm can be shown as formula (1):

[0072]

[0073] Among them, "*" represents a convolution operation, and "×" represents a multiplication operation.

[0074] It can be seen from formula (1) that the result of the convolution operation based on the Karatsuba algorithm consists of three parts, among which the first part of the result can be described as: after the two high-order digits are split, the first N / 2-dimensional data are multiplied, and then the left shift N bits are performed; the second part of the result can be described as: after the two high-order digits are split, the second N / 2-dimensional data are multiplied; the third part of the result can be described as: after the sum of the high-order digit 1 and the sum of the high-order digit 2 are multiplied, the first N / 2 dimensions are multiplied, and the second N / 2 dimensions are multiplied, and then the sum and accumulation operations are performed between the three.

[0075] In some implementations, the convolution operation based on the Karatsuba algorithm can be shown as formula (2):

[0076] ∑a i *b i =∑{(a i1 ×b i1 )<<N+(a i2 ×b i2 )+[(a i1 +a i2 )×(b i1 +b i2 )-(a i1 ×b i1 )-(a i2 ×b i2 )]<<N / 2}

[0077] =∑(a i1 ×b i1 )<<N+∑(a i2 ×b i2 )+∑[(a i1 +a i2 )×(b i1 +b i2 )-(a i1 ×b i1 )-(a i2 ×b i2 )]<<N / 2

[0078] =∑ 16-1 (a i1 ×b i1 )<<N+∑ 16-1 (a i2 ×b i2 )+∑ 16-1 [(a i1 +a i2 )×(b i1 +b i2 )-(a i1 ×b i1 )-(a i2 ×b i2 )]<<N / 2

[0079] +∑ 16-2 (a i1 ×b i1 )<<N+∑ 16-2 (a i2 ×b i2 )+∑ 16-2 [(a i1 +a i2 )×(b i1 +b i2 )-(a i1 ×b i1 )-(a i2 ×bi2 )]< <N / 2

[0080] ……+∑ 16-m (a i1 ×b i1 )< <N+∑ 16-m (a i2 ×b i2 )+∑ 16-m [(a i1 +a i2 )×(b i1 +b i2 )-(a i1 ×b i1 )-(a i2 ×b i2 )]< <N / 2 (2)

[0081] Among them, "*" represents a convolution operation, and "×" represents a multiplication operation.

[0082] From formula (2), we can know that the two high-order digits (high-order digit 1 and high-order digit 2) are divided into blocks, and m blocks of sub-high-order digit 1 and m blocks of sub-high-order digit 2 are obtained respectively. The first block of high-order digits in the m-block high-order digits 1 and the first block of high-order digits in the m-block high-order digits 2 are convolved, and the result of the convolution operation based on the Karatsuba algorithm consists of three parts, wherein the first part of the result can be described as: after the first N / 2-dimensional data obtained by splitting the two high-order digits (for example, the first sub-high-order digit and the second sub-high-order digit) are respectively split, a multiplication operation is performed, and then a left shift N-bit operation is performed; the second part of the result can be described as: after the second N / 2-dimensional data obtained by splitting the two high-order digits (for example, the first sub-high-order digit and the second sub-high-order digit) are respectively split, a multiplication operation is performed; the third part of the result can be described as: after the sum corresponding to the first sub-high-order digit and the sum corresponding to the second sub-high-order digit are multiplied, the first N / 2-dimensional data obtained by splitting the two high-order digits (the first sub-high-order digit and the second sub-high-order digit) are respectively split, and the second N / 2-dimensional data obtained by splitting the two high-order digits (the first sub-high-order digit and the second sub-high-order digit) are respectively split, a summation and accumulation operation is performed between the three, to obtain a set of intermediate results of the first block of high-order digits;

[0083] It can be understood that convolving the second block of sub-high-order bits in m-block sub-high-order bits 1 with the second block of sub-high-order bits in m-block sub-high-order bits 2 can produce a set of intermediate calculation results for the second block of sub-high-order bits. In this case, the calculation result can be obtained by accumulating the intermediate calculation results corresponding to each of the m groups of sub-high-order bits.

[0084] In some implementations, based on the theoretical research formula shown in formula (2), the vector corresponding to each sub-high-order digit can be extracted, and then the operation result can be obtained through vector calculation. For example, the two high-order digits (high-order digit 1 and high-order digit 2) are divided into blocks to obtain m blocks of sub-high-order digits 1 and m blocks of sub-high-order digits 2 respectively. Among them, the first block of sub-high-order digits in the m blocks of sub-high-order digits 1 and the first block of sub-high-order digits in the m blocks of sub-high-order digits 2 are convolved. The operation result of the convolution operation based on the Karatsuba algorithm consists of three parts, among which the first part of the operation result can be described as: the first N / 2 dimensional data obtained by splitting the vectors corresponding to the two high-order digits (for example, the first sub-high-order digit vector and the second sub-high-order digit vector) are multiplied, and then the left shift N bits are performed; the second part of the operation result can be described as: the vectors corresponding to the two high-order digits (for example, the first sub-high-order digit vector and the second sub-high-order digit vector) are split into The multiplication operation is performed on the last N / 2 dimensional data obtained by splitting; the result of the third part of the operation can be described as: the sum corresponding to the first sub-high-order digit vector and the sum corresponding to the second sub-high-order digit vector are multiplied, the first N / 2 dimensional data obtained by splitting the two high-order digit vectors (the first sub-high-order digit vector and the second sub-high-order digit vector) are multiplied, and the last N / 2 dimensional data obtained by splitting the two high-order digit vectors (the first sub-high-order digit vector and the second sub-high-order digit vector) are multiplied, and then summation and accumulation operations are performed between the three to obtain a set of intermediate results of the operation of the first block of high-order digit vectors;

[0085] It can be understood that convolving the vector of the second block of high-order sub-bits in m-block high-order sub-bits 1 with the vector of the second block of high-order sub-bits in m-block high-order sub-bits 2 can obtain the intermediate result of the operation of the second block of high-order sub-bits. In this case, the operation result can be obtained by accumulating the intermediate results corresponding to the vectors of the m groups of high-order sub-bits.

[0086] Based on the mathematical calculation theory shown in formula (1) or formula (2), as shown in Figure 5a , which is a hardware structure diagram of a chip provided in an embodiment of the present application, can include an artificial intelligence processor (Neural Network Processing Unit, NPU) 50 for implementing the computing function of a convolutional neural network 100. The neural network processor in the embodiment of the present application can be applied to various devices that can perform matrix multiplication operations, such as mobile phones, tablet computers, servers, and wearable devices.

[0087] In an embodiment of the present application, the artificial intelligence processor 50 can be an NPU, Google's tensor processing unit (TPU), or a graphics processing unit (GPU), or any other processor suitable for large-scale XOR operation processing. Take the NPU as an example: the NPU can be mounted on the main CPU (HostCPU) as a coprocessor, and the main CPU assigns tasks to it. The core part of the NPU is the operation circuit 503, which extracts the first matrix data and the second matrix data in the memory through the operation circuit 503, wherein the first matrix data and the second matrix data are both N-dimensional matrix data; the two N-dimensional matrix data are symmetrically split into corresponding 4 low-bit multiplication operations and accumulation operations of N / 2-dimensional matrix data to obtain the operation result.

[0088] In some implementations, the arithmetic circuit 503 includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 503 may be a two-dimensional systolic array. The arithmetic circuit 503 may also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.

[0089] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. In the embodiment of the present application, the input buffer 501 can be expressed in the form of an input memory 5011 and a weight memory 5012. The operation circuit 503 obtains the weight data of the matrix B from the weight memory 5012 and caches it on each PE in the operation circuit 503. The operation circuit 503 obtains the input data of the matrix A from the input memory 5011, performs a matrix operation based on the input data of the matrix A and the weight data of the matrix B, and stores the partial result or the final result of the matrix in the output buffer 505.

[0090] In the embodiment of the present application, the input buffer 501 and the output buffer 505 can be random access memory (Random Access Memory, RAM) or power-off volatile storage devices, such as static random access memory (Static Random Access Memory, SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM) or synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate SDRAM (Dual Data Rate SDRAM, DDR SDRAM), etc., and the embodiment of the present application does not make specific limitations.

[0091] In an embodiment of the present application, weight data is directly transferred to the weight memory 5012 through the storage unit access controller (DMAC, Direct Memory Acccss Controllcr) 501.

[0092] BIU is Bus Interface Unit, i.e., bus interface 507, which is used for interaction between AXI bus, DMAC and instruction fetch buffer 509.

[0093] The bus interface unit 507 (BIU) is used for the instruction fetch memory 506 to obtain instructions from the external memory, and is also used for the storage unit access controller 501 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0094] DMAC is mainly used to move weight data to the weight memory 5012 or to move input data to the input memory 5011.

[0095] The vector calculation circuit 504 includes multiple processing circuits. When necessary, it further processes the output of the calculation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / FC layer network calculations in neural networks, such as pooling, batch normalization, local response normalization, etc.

[0096] In some implementations, the vector calculation circuit 504 stores the processed output vector in the output buffer 505. For example, the vector calculation circuit 504 can apply a nonlinear function to the output of the operation circuit 503, such as a vector of accumulated values, to generate an activation value. In some implementations, the vector calculation circuit 504 generates a normalized value, a merged value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 503, for example, for use in a subsequent layer in a neural network.

[0097] in, Figure 1 and Figure 2 The operations of each layer in the convolutional neural network shown can be performed by the vector calculation circuit 504.

[0098] like Figure 5b FIG. 5 is a schematic diagram of the structure of an artificial intelligence processor 50 provided in an embodiment of the present application. The operation circuit 503 may include a splitting circuit 5031 and a matrix multiplication and accumulation circuit 5032.

[0099] In an embodiment of the present application, the splitting circuit 5031 is used to split the first matrix data to obtain first sub-matrix data of the first N / 2 dimensions and second sub-matrix data of the last N / 2 dimensions; and to split the second matrix data to obtain third sub-matrix data of the first N / 2 dimensions and fourth sub-matrix data of the last N / 2 dimensions; wherein the first matrix data and the second matrix data are both N-dimensional matrix data; and N is a positive even number.

[0100] In the embodiment of the present application, the first matrix data may be input matrix data, for example, the input matrix data may be image matrix data. The second matrix data may be weight matrix data.

[0101] In the embodiments of the present application, the first matrix data and the second matrix data can be unsigned or signed, and this application does not impose any specific restrictions. Taking the first matrix data as image matrix data as an example, if the image matrix data is the matrix data corresponding to the original image without any processing, the image matrix data can be unsigned; if the image matrix data is the matrix data corresponding to the residual image, the image matrix data can be signed or unsigned, and the determination should be based on the specific actual situation.

[0102] In an embodiment of the present application, the first matrix data and the second matrix data are symmetrically split, and two N / 2-dimensional multiplications are used to implement the N-dimensional multiplication operation, which can improve the processing speed of the operation circuit. Generally speaking, when the bit width N of the multiplication operation is very large, the optimized N / 2-dimensional symmetrical split multiplication operation is faster than the N-dimensional multiplication operation. In terms of area, the area of ​​the N-dimensional multiplication operation is not twice that of the simple N / 2-dimensional multiplication, but three times or even four times, and the larger N is, the higher the multiple is.

[0103] In the embodiment of the present application, the first matrix data and the second matrix data involved in the convolution operation can be stored in the input buffer 501. Specifically, the first matrix data can be stored in the input memory 5011, and the second matrix data can be stored in the weight memory 5012. The splitting circuit 5031 can obtain the first matrix data from the input memory 5011 and the second matrix data from the weight memory, and then split the first matrix data and the second matrix data.

[0104] The matrix multiplication and accumulation circuit 5032 is used to perform multiplication and accumulation operations based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data to obtain an operation result. In the embodiment of the present application, when the operation circuit implements the convolution operation of two high-order digits, the two N-dimensional high-order digits are symmetrically split to obtain four corresponding N / 2-dimensional low-order digits. After that, three multiplication operations and accumulation operations are performed on these four N / 2-dimensional low-order digits. Since two N / 2-dimensional multiplications are used to implement the N-dimensional multiplication operation, its operation complexity is reduced by n 2 Reduced to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0105] In an embodiment of the present application, the matrix multiplication and accumulation circuit 5032 may include a matrix multiplication circuit and an accumulation circuit; wherein the matrix multiplication circuit is used to perform a multiplication operation based on the first sub-matrix data and the third sub-matrix data to obtain first intermediate data; perform a multiplication operation based on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; perform a multiplication operation based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data to obtain third intermediate data; and the accumulation circuit is used to accumulate the first intermediate data, the second intermediate data and the third intermediate data to obtain an operation result. In the present application, when implementing two high-bit convolution operations, the N-dimensional multiplication operation can be implemented by using two N / 2-dimensional multiplications through the matrix multiplication circuit and the accumulation circuit, and its operation complexity is reduced by n. 2 Reduced to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0106] In the embodiment of the present application, the matrix multiplication and accumulation circuit 5032 may include a plurality of matrix multiplication circuits 5032a (eg, 3 and integer multiples of 3) and one or more Karatsuba accumulation circuits 5032b.

[0107] In some possible implementations, the matrix multiplication and accumulation circuit 5032 may include three matrix multiplication circuits 5032a and one Karatsuba accumulation circuit 5032b; wherein the three matrix multiplication circuits 5032a may be a first matrix multiplication circuit, a second matrix multiplication circuit, and a third matrix multiplication circuit, respectively; and one Karatsuba accumulation circuit 5032b may perform a Karatsuba accumulation operation on the operation results generated by each of the three matrix multiplication circuits 5032a.

[0108] In the embodiment of the present application, the first matrix multiplication circuit 5032a is configured to perform a multiplication operation on the first sub-matrix data and the third sub-matrix data to obtain first sub-intermediate data, and shift the first sub-intermediate data left by N bits to obtain first intermediate data;

[0109] The second matrix multiplication circuit 5032a is configured to perform a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data;

[0110] The third matrix multiplication circuit 5032a is used to perform an accumulation operation on the first sub-matrix data and the second sub-matrix data to obtain first sum matrix data, perform an accumulation operation on the third sub-matrix data and the fourth sub-matrix data to obtain second sum matrix data, and perform a multiplication operation on the first sum matrix data and the second sum matrix data to obtain fourth intermediate data; after accumulating the first sub-intermediate data, the second intermediate data, and the fourth intermediate data, and shifting them left by N / 2 bits to obtain third intermediate data.

[0111] In a specific implementation, the multiple matrix multiplication circuits 5032a included in the matrix multiplication and accumulation circuit 5032 can each independently perform a matrix multiplication operation. Figure 5c , Figure 5c In the figure, a matrix multiplication and accumulation circuit 5032 including three matrix multiplication circuits 5032a is used as an example. Matrix multiplication circuit 5032a includes M operation groups consisting of operation blocks. Each operation group includes K operation blocks. Each operation block includes N sub-operation units. Each sub-operation unit has two inputs, one for receiving data from different memories (such as input memory 5011 and weight memory 5012) and multiplying the two data paths. Karatsuba accumulation circuit 5032b is used to perform Karatsuba accumulation on the calculation results generated by each of the three matrix multiplication circuits 5032a to obtain the final calculation result.

[0112] Considering that in some practical applications, taking the first matrix data as image matrix data and the second matrix data as weight matrix data as an example, the image matrix data and the weight matrix data may contain negative numbers, such as Figure 5d, which is a schematic diagram of the structure of another artificial intelligence processor 50 provided in an embodiment of the present application. The operation circuit 503 may include one or more splitting circuits 5031, one or more OR logic judgment circuits 5033, and multiple matrix multiplication and accumulation circuits 5032. The splitting circuit 5031 is connected to the OR logic judgment circuit 5033, and the OR logic judgment circuit 5033 is connected to the matrix multiplication and accumulation circuit 5032. Specifically, the splitting circuit 5031 is used to split the first matrix data to obtain first sub-matrix data and second sub-matrix data; split the second matrix data to obtain third sub-matrix data and fourth sub-matrix data; the OR logic judgment unit 5033 is used to process the signed number to obtain the binary complement of the signed number. After that, the matrix multiplication and accumulation circuit 5032 obtains the binary complement corresponding to the signed number and performs a matrix multiplication and accumulation operation, thereby obtaining an operation result.

[0113] In the embodiment of the present application, the first matrix data is a signed number; the first sub-matrix data and the second sub-matrix data obtained by splitting each include a sign bit; the second matrix data is a signed number; the third sub-matrix data and the fourth sub-matrix data obtained by splitting each include a sign bit. It can be understood that the split matrix data consists of two parts: one part is the matrix data itself, and the other part is the sign bit.

[0114] Then, in this case, the OR logic judgment circuit 5033 is used to: when the non-sign bits of the second sub-matrix data are not all 0, perform an addition operation on the first sub-matrix data; when the non-sign bits of the fourth sub-matrix data are not all 0, perform an addition operation on the third sub-matrix data.

[0115] It can be understood that the OR logic judgment circuit 5033 is also used for: when the non-sign bits of the second sub-matrix data are all 0, not performing the addition operation on the first sub-matrix data; when the non-sign bits of the fourth sub-matrix data are all 0, not performing the addition operation on the third sub-matrix data.

[0116] In the embodiment of the present application, taking the first sub-matrix data as a signed number as an example, performing an addition operation on the first sub-matrix data can obtain the binary complement form corresponding to the first sub-matrix data, so that the matrix multiplication accumulation circuit 5032 can then perform a matrix multiplication accumulation operation on it.

[0117] In the embodiments of this application, Figure 6 The structure of the vector calculation unit 504 is used to generate normalized values, merged values, or both. The processed output vector can be used as an activation input to the operation circuit 503, for example, for use in subsequent layers in the neural network, which mainly includes activation circuits, normalization circuits, and pooling circuits.

[0118] an activation circuit 5041 that applies a nonlinear function to each accumulated value to generate an activation value, such as a nonlinear hyperbolic function tanh(x);

[0119] A normalization circuit 5042 generates a normalized value according to the activation value;

[0120] The pooling circuit 5043 applies an aggregation function to the normalized value to generate a pooled value. For example, in some implementations, the aggregation function is a function that returns the maximum value, minimum value, or average value in a set of normalized values.

[0121] In the embodiment of this application, Figure 1 Taking the convolutional neural network shown in FIG. 5 as an example, the operation circuit 503 is used to perform a convolution operation in the convolution layer based on the first matrix data and the second matrix data to extract image feature data of the image / video to be processed. Subsequently, the vector calculation circuit 504 is used to perform operations in the neural network layer based on the image feature data to obtain a recognition result. For example, the recognition result can be image recognition or speech recognition, which is not specifically limited in this embodiment of the application.

[0122] based on Figure 5a or Figure 5d The hardware structure diagram shown below is combined with Figure 7a The flowchart of a convolution operation method provided in the embodiment of the present application is shown, which specifically explains how the convolution operation is implemented in the embodiment of the present application.

[0123] In an embodiment of the present application, the convolution operation method is applied to a convolution operation circuit, wherein the operation circuit includes a splitting circuit and a matrix multiplication and accumulation circuit. The method may include but is not limited to the following steps:

[0124] Step S700: Split the first matrix data by a splitting circuit in a convolution operation circuit to obtain first sub-matrix data of the first N / 2 dimensions and second sub-matrix data of the last N / 2 dimensions; split the second matrix data to obtain third sub-matrix data of the first N / 2 dimensions and fourth sub-matrix data of the last N / 2 dimensions; wherein the first matrix data and the second matrix data are both N-dimensional matrix data; and N is a positive even number.

[0125] In some implementations, as described above, the first matrix data may be stored in the input memory 5011, and the second matrix data may be stored in the weight memory 5012. In this case, the splitting circuit may obtain the first matrix data from the input memory 5011 and the second matrix data from the weight memory 5012.

[0126] In the embodiment of the present application, if a is the first matrix data (eg, image matrix data) and is an M*K matrix, then the element in the i-th row and j-th column of the first matrix data a can be recorded as a ij , i=(1, 2, 3,...M), j=(1, 2, 3,...K).

[0127] In the embodiment of the present application, if b is the second matrix data (for example, weight matrix data) which is an N*K matrix, then the element in the i-th row and j-th column of the second matrix data b can be recorded as b ij , i=(1, 2, 3,...N), j=(1, 2, 3,...K).

[0128] In an embodiment of the present application, the image matrix data and the weight matrix data can be unsigned numbers or signed numbers, and this application does not make specific limitations. Taking image matrix data as an example, if the image matrix data is the matrix data corresponding to the original image that has not been processed in any way, the image matrix data can be an unsigned number; if the image matrix data is the matrix data corresponding to the residual image, the image matrix data can be a signed number or an unsigned number, which should be determined in combination with specific actual conditions. In an embodiment of the present application, the first matrix data can be split in a symmetrical splitting manner to obtain the first sub-matrix data of the front N / 2 dimensions and the second sub-matrix data of the rear N / 2 dimensions, that is, the first sub-matrix data is the high-bit image matrix data in the image matrix data, and the second sub-matrix data is the low-bit image matrix data in the image matrix data.

[0129] Similarly, in the embodiment of the present application, the second matrix data can be split in a symmetrical splitting manner to obtain the third sub-matrix data of the first N / 2 dimensions and the fourth sub-matrix data of the last N / 2 dimensions, that is, the third sub-matrix data is the high-order weight matrix data in the weight matrix data, and the fourth sub-matrix data is the low-order weight matrix data in the weight matrix data. Specifically, Figure 7b As shown, the size of the image matrix data is 12*12, the size of the weight matrix data is 12*12, and the symmetrical splitting method is used to split the data, and the sizes of the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data are all 12*6.

[0130] Step S702: Perform a multiplication and accumulation operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data by the matrix multiplication and accumulation circuit in the operation circuit to obtain an operation result.

[0131] In some implementations, the matrix multiplication and accumulation circuit may include a matrix multiplication circuit and an accumulation circuit. In this case, the matrix multiplication and accumulation circuit in the operation circuit performs multiplication and accumulation operations based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data, and the implementation steps of obtaining the operation result may include: performing a multiplication operation based on the first sub-matrix data and the third sub-matrix data through the matrix multiplication circuit to obtain first intermediate data; performing a multiplication operation based on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; performing a multiplication operation based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data to obtain third intermediate data; and accumulating the first intermediate data, the second intermediate data and the third intermediate data through the accumulation circuit to obtain the operation result.

[0132] In some implementations, the matrix multiplication circuit includes a first matrix multiplication circuit, a second matrix multiplication circuit, and a third matrix multiplication circuit; then, in this case, the implementation steps of performing a multiplication and accumulation operation based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data by the matrix multiplication and accumulation circuit can be as follows: Figure 7c As shown, it may specifically include: performing a multiplication operation on the first sub-matrix data and the third sub-matrix data through a first matrix multiplication circuit to obtain first sub-intermediate data, and shifting the first sub-intermediate data left by N bits to obtain first intermediate data; performing a multiplication operation on the second sub-matrix data and the fourth sub-matrix data through a second matrix multiplication circuit to obtain second intermediate data; performing an accumulation operation on the first sub-matrix data and the third sub-matrix data through a third matrix multiplication circuit to obtain first sum matrix data, performing an accumulation operation on the third sub-matrix data and the fourth sub-matrix data to obtain second sum matrix data, and performing a multiplication operation on the first sum matrix data and the second sum matrix data to obtain fourth intermediate data; accumulating the first sub-intermediate data, the second intermediate data, and the fourth intermediate data, and shifting them left by N / 2 bits to obtain third intermediate data; and accumulating the first intermediate data, the second intermediate data, and the third intermediate data to obtain an operation result.

[0133] In an embodiment of the present application, the dimensions of the first sub-intermediate data, the second intermediate data and the fourth intermediate data can be N dimensions or N-1 dimensions, so that N-dimensional multiplication operations can be achieved by multiplying two N / 2-dimensional matrix data.

[0134] It should be noted that, in an embodiment of the present application, taking the first matrix data and the second matrix data as an example, the first matrix data can be split into first sub-matrix data and second sub-matrix data, and the second matrix data can be split into third sub-matrix data and fourth sub-matrix data. For ease of explanation, the three multiplication operations corresponding to the above four sub-matrix data are considered to be the multiplication operation process of the minimum calculation unit. For example, the first matrix data and the second matrix data are both 12*12 matrix data. In this case, the minimum calculation unit can be the multiplication operation of two 12*6 matrices. For another example, the first matrix data and the second matrix data are both 24*24 matrix data. In this case, the minimum calculation unit can be the multiplication operation of two 24*12 matrices.

[0135] In some implementations, considering that in actual applications, both the image matrix data and the weight matrix data are large matrix data, in this case, the first large matrix data and the second large matrix data can be divided into blocks to obtain m first matrix blocks and m second matrix blocks; wherein, the i-th matrix block in the m first matrix blocks is used as the first matrix data, and the i-th matrix block in the m second matrix blocks is used as the second matrix data; wherein, i is taken from 1 to m in sequence to obtain m groups of first matrix data and second matrix data. That is to say, the convolution operation of these multiple matrix blocks can be based on the above-mentioned minimum calculation unit, and its specific implementation process can include: performing an operation on each group of first matrix data and second matrix data in turn through a matrix multiplication and accumulation circuit, and m intermediate operation results can be obtained, and then the m intermediate operation results are accumulated to obtain the operation result. For example, Figure 7d As shown, before being divided into blocks, the size of the large image matrix data is 24*24, and the size of the large weight matrix data is 24*24. Now it is divided into blocks, for example, m=2, and two groups of first matrix data and second matrix data can be obtained. For example, the first group includes image matrix data 1 and weight matrix data 1; the second group includes image matrix data 2 and weight matrix data 2. According to the above description, the intermediate result of the operation corresponding to the first matrix block in the image matrix data and the first matrix block in the weight matrix data can be obtained. At this time, the intermediate result of the operation is stored in the output buffer; the intermediate result of the operation corresponding to the second matrix block in the image matrix data and the second matrix block in the weight matrix data can also be obtained. Afterwards, the intermediate result of the operation corresponding to the first group of first matrix data and second matrix data stored in the output buffer is accumulated with the intermediate result of the operation of the second group of first matrix data and second matrix data, so as to obtain the operation result.

[0136] In some implementations, considering that in practical applications, both image matrix data and weight matrix data are large matrix data, the first large matrix data and the second large matrix data can be divided into blocks to obtain m first matrix blocks and m second matrix blocks; wherein the i-th matrix block in the m first matrix blocks serves as the first matrix data, and the i-th matrix block in the m second matrix blocks serves as the second matrix data; wherein i is sequentially taken from 1 to m, to obtain m groups of first matrix data and second matrix data. In other words, the convolution operation on these multiple matrix blocks can be based on the above-mentioned minimum computational unit. In addition, the operation can also be performed by obtaining the vector of each matrix data. The specific implementation process can include: performing an operation on each group of first matrix vectors and second matrix vectors in sequence using a matrix multiplication and accumulation circuit element to obtain m intermediate operation results, and then accumulating the m intermediate operation results to obtain the operation result. For example, if m = 2, two groups of first matrix data and second matrix data can be obtained. Then, the vector corresponding to the first matrix data and the vector corresponding to the second matrix data are obtained respectively to obtain the first matrix vector and the second matrix vector. According to the above description, an operation is performed on each set of the first matrix vector and the second matrix vector to obtain two intermediate operation results. Thereafter, the two intermediate operation results are accumulated to obtain the operation result.

[0137] In some implementations, the operation result may be stored in an output buffer.

[0138] In the embodiment of the present application, after the arithmetic circuit performs the convolution operation, the output buffer temporarily stores the operation result so as to facilitate the subsequent operation of the neural network layer (e.g., pooling operation). Specifically, the convolution operation can be used to complete the feature extraction operation for the image, and then the extracted features can be input into the neural network layer for processing.

[0139] In the embodiment of the present application, when implementing the convolution operation of two high-order digits, the operation circuit symmetrically splits the two N-dimensional high-order digits to obtain four corresponding N / 2-dimensional low-order digits. Then, three multiplication operations and accumulation operations are performed on these four N / 2-dimensional low-order digits. Since the N-dimensional multiplication operation is implemented by two N / 2-dimensional multiplications, the operation complexity is reduced by n 2 Reduced to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0140] In some practical applications, the image matrix data and the weight matrix data may be signed or unsigned. In the case of signed numbers, the signed numbers may be processed to obtain the corresponding binary complement for ease of calculation.

[0141] In the embodiments of this application, Figure 7e As shown in the figure, taking the second matrix data as an example, and the second matrix data is a signed number, when the second matrix data is split, the third sub-matrix data and the fourth sub-matrix data obtained by splitting both contain sign bits. In this case, it can be considered that the third sub-matrix data consists of two parts, one of which is the sign bit and the other is the non-sign bit.

[0142] In an embodiment of the present application, the implementation process of determining the binary complement corresponding to the third sub-matrix data and the fourth sub-matrix data by the OR logic judgment circuit may include: the OR logic judgment circuit performs an addition operation on the third sub-matrix data when the non-sign bits of the fourth sub-matrix data are not all 0, so as to represent the third sub-matrix data in the form of a binary complement, so that the calculation result can be input into the matrix multiplication and accumulation circuit to implement the multiplication and accumulation operation of the two matrices. Then, correspondingly, when the non-sign bits of the fourth sub-matrix data are all 0, the OR logic judgment circuit does not perform the addition operation on the third sub-matrix.

[0143] In the implementation of the embodiment of the present application, when the two high-order digits involved in the convolution operation are signed numbers, the operation circuit can convert the signed numbers into the corresponding binary complement form through the OR logic judgment circuit in the operation circuit, and then perform three multiplication operations and accumulation operations on the four N / 2-dimensional low-order digits obtained by splitting. By converting the signed numbers through the OR logic judgment circuit, errors in the convolution operation can be avoided. In addition, since two N / 2-dimensional multiplications are used to implement the N-dimensional multiplication operation, the operation complexity is reduced by n. 2 Reduced to n 1.585 , which can improve the computing speed of the processor and thus reduce the area and power consumption of the processor.

[0144] The embodiment of the present application also provides a board card, which includes the packaging structure of the chip. Figure 8a , Figure 8a A board is provided. In addition to the chip 889, the board may also include other supporting components, including but not limited to: at least one storage device 890, an interface device 891, and a control device 892.

[0145] The memory device 890 is connected to the chip in the chip packaging structure via a bus for storing data.

[0146] In an embodiment of the present application, the at least one storage component 890 may store various intermediate calculation result data or configuration data generated by the chip running algorithm (e.g., convolutional neural network algorithm) or process.

[0147] In a specific implementation, the memory device may include multiple memory groups 893. Each memory group is connected to the chip via a bus. It is understood that each memory group may be a random access memory (RAM) or a power-off volatile memory device, such as a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate SDRAM (DDR SDRAM), etc.

[0148] Taking DDR as an example, DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read out on the rising and falling edges of the clock pulse. The speed of DDR is twice that of standard SDRAM. In one embodiment, the storage device may include 4 groups of the memories. Each group of the memories may include multiple DDR4 particles (chips). In one embodiment, the chip may include 4 72-bit DDR4 controllers, and 64 bits of the above 72-bit DDR4 controllers are used for data transmission and 8 bits are used for ECC verification. It can be understood that when DDR4-3200 particles are used in each group of the memories, the theoretical bandwidth of data transmission can reach 25600MB / s.

[0149] In one embodiment, each memory group includes a plurality of double data rate synchronous dynamic random access memories (DDRs) connected in parallel. DDRs can transmit data twice in one clock cycle. A controller for controlling the DDRs is provided in the chip to control data transmission and data storage of each memory.

[0150] The interface device is electrically connected to the chip in the chip packaging structure. The interface device is used to realize data transmission between the chip and an external device (such as a server or a computer). For example, in one embodiment, the interface device can be a standard PCIE interface. For example, the data to be processed is transferred to the chip by the server through the standard PCIE interface to realize data transfer. Preferably, when the PCIE 3.0X16 interface is used for transmission, the theoretical bandwidth can reach 16000MB / s. In another embodiment, the interface device can also be other interfaces. This application does not limit the specific forms of expression of the above-mentioned other interfaces. The interface unit can realize the switching function. In addition, the calculation results of the chip are still transmitted back to the external device (such as a server) by the interface device.

[0151] Wherein, the control device is electrically connected to the chip. The control device is used to communicate with the chip, call the chip to implement the operation of the convolutional neural network, and can also be used to monitor the status of the chip. In some implementations, the chip and the control device are electrically connected through an SPI interface. The control device may include a single-chip microcomputer (Micro Controller Unit, MCU). For example, the chip may include multiple processing chips, multiple processing cores or multiple processing circuits, which can drive multiple loads. Therefore, the chip can be in different working states such as multi-load and light load.

[0152] An embodiment of the present application further provides an electronic device, which includes a board.

[0153] An embodiment of the present application also provides an electronic device, which may include a data processing device, a robot, a computer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a cloud server, a camera, a camcorder, a projector, a watch, headphones, a mobile storage, and a wearable device.

[0154] like Figure 8b The electronic device 800 shown may include: a processor 801, a memory 802, a communication bus 803 and a communication interface 804, and the processor 801 is connected to the memory 802 and the communication interface 803 through the communication bus.

[0155] The processor 801 may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor 801 may be any conventional processor, etc.

[0156] The processor 801 may also be an integrated circuit chip with signal processing capabilities. During implementation, each step of the data processing method of the present application may be performed by hardware integrated logic circuits or software instructions in the processor 801. The processor 801 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The storage medium is located in the memory 801, and the processor 801 reads the information in the memory 802 and combines its hardware to complete the convolution operation in the convolutional neural network to extract image features.

[0157] The memory 802 may be a read-only memory (ROM), a random access memory (RAM), or other memory. In the embodiment of the present application, the memory 802 is used to store data and various software programs, such as the program of the convolution operation method involved in the embodiment of the present application.

[0158] The communication interface 804 uses a transceiver, such as but not limited to a transceiver, to enable communication between the electronic device 800 and other devices or a communication network. For example, the communication interface 903 can be used to obtain the original data set, the first data set, etc., to enable information exchange with a training device, a client device, a user device, or a terminal device.

[0159] Optionally, the electronic device may further include an artificial intelligence processor 805, which may be a neural network processor (NPU), a tensor processing unit (TPU), or a graphics processing unit (GPU), or any other processor suitable for large-scale XOR processing. The artificial intelligence processor 805 can be mounted on the host CPU (Host CPU) as a coprocessor, and the host CPU assigns tasks to it. The artificial intelligence processor 805 can implement one or more operations involved in the above-mentioned data processing method. For example, taking the NPU as an example, the core part of the NPU is the operation circuit, which controls the operation circuit through the controller to extract the matrix data in the memory 802 and perform multiplication and addition operations.

[0160] The processor 801 is configured to call the data and program codes in the memory and execute:

[0161] Splitting the first matrix data to obtain first sub-matrix data of the first N / 2 dimensions and second sub-matrix data of the last N / 2 dimensions; splitting the second matrix data to obtain third sub-matrix data of the first N / 2 dimensions and fourth sub-matrix data of the last N / 2 dimensions; wherein both the first matrix data and the second matrix data are N-dimensional matrix data; and N is a positive even number;

[0162] A multiplication and accumulation operation is performed on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain an operation result.

[0163] The processor 801 performs a multiplication-accumulation operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain an operation result, including:

[0164] Performing a multiplication operation on the first sub-matrix data and the third sub-matrix data to obtain first intermediate data; performing a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; performing a multiplication operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain third intermediate data;

[0165] The first intermediate data, the second intermediate data, and the third intermediate data are accumulated to obtain a calculation result.

[0166] The processor 801 performs a multiplication operation on the first sub-matrix data and the third sub-matrix data to obtain first intermediate data; performs a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; and performs a multiplication operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain third intermediate data, including:

[0167] Performing a multiplication operation on the first sub-matrix data and the third sub-matrix data to obtain first sub-intermediate data, and shifting the first sub-intermediate data left by N bits to obtain the first intermediate data;

[0168] Performing a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain the second intermediate data;

[0169] Performing an accumulation operation on the first sub-matrix data and the second sub-matrix data to obtain first sum matrix data, performing an accumulation operation on the third sub-matrix data and the fourth sub-matrix data to obtain second sum matrix data, performing a multiplication operation on the first sum matrix data and the second sum matrix data to obtain fourth intermediate data; after accumulating the first sub-intermediate data, the second intermediate data and the fourth intermediate data, shifting them left by N / 2 bits to obtain the third intermediate data.

[0170] The processor 801 may also be configured to:

[0171] Dividing the first large matrix data and the second large matrix data into blocks to obtain m first matrix blocks and m second matrix blocks, respectively; wherein the i-th matrix block among the m first matrix blocks serves as the first matrix data, and the i-th matrix block among the m second matrix blocks serves as the second matrix data; wherein i is sequentially taken from 1 to m to obtain m groups of first matrix data and second matrix data;

[0172] The processor 801 performs a multiplication-accumulation operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain an operation result, including:

[0173] For each group of the first matrix data and the second matrix data, perform the multiplication and accumulation operation based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data to obtain m intermediate operation results; and accumulate the m intermediate operation results to obtain an operation result.

[0174] The processor 801 may also be configured to:

[0175] Divide the first large matrix data and the second large matrix data into blocks to obtain m first matrix blocks and m second matrix blocks, respectively; wherein the i-th matrix block in the m first matrix blocks is used as the first matrix vector, and the i-th matrix block in the m second matrix blocks is used as the second matrix vector; wherein i is sequentially taken from 1 to m to obtain m groups of first matrix vectors and second matrix vectors;

[0176] The processor 801 performs a multiplication-accumulation operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain an operation result, including:

[0177] For each group of the first matrix vector and the second matrix vector, perform multiplication and accumulation operations based on the first sub-matrix vector, the second sub-matrix vector, the third sub-matrix vector and the fourth sub-matrix vector to obtain m intermediate operation results; accumulate the m intermediate operation results to obtain an operation result.

[0178] Wherein, the first matrix data is a signed number; the first sub-matrix data and the second sub-matrix data both include a sign bit; the second matrix data is a signed number; the third sub-matrix data and the fourth sub-matrix data both include a sign bit; the processor 801 may further include:

[0179] When the non-sign bits of the second sub-matrix data are not all 0, the first sub-matrix data is added by 1; when the non-sign bits of the fourth sub-matrix data are not all 0, the third sub-matrix data is added by 1.

[0180] It should be understood that the implementation of each device can also refer to the corresponding description in the above method embodiment, and the embodiments of this application will not be repeated.

[0181] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to implement a computer program for two high-bit convolutions based on the Karatsuba algorithm, and the computer program enables an electronic device to perform part or all of the steps of any convolution operation method described in the above method embodiments.

[0182] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable an electronic device to perform part or all of the steps of any convolution operation method recorded in the above method embodiments.

[0183] It is understood that those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the various embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0184] Those skilled in the art will appreciate that the functions described in the various illustrative logic blocks, modules, and algorithm steps disclosed in the various embodiments of this application can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions described in the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media can include computer-readable storage media, which corresponds to tangible media, such as data storage media, or communication media including any media that facilitates the transfer of computer programs from one place to another (e.g., according to a communication protocol). In this way, computer-readable media can generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the technology described in this application. A computer program product can include computer-readable media.

[0185] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0187] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0188] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0189] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0190] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A convolution operation circuit for implementing a convolution operation of a first matrix data and a second matrix data, characterized in that: It includes a splitting circuit and a matrix multiplication and accumulation circuit; wherein, The splitting circuit is configured to split the first matrix data to obtain first sub-matrix data of the first N / 2 dimensions and second sub-matrix data of the last N / 2 dimensions; and split the second matrix data to obtain third sub-matrix data of the first N / 2 dimensions and fourth sub-matrix data of the last N / 2 dimensions; wherein both the first matrix data and the second matrix data are N-dimensional matrix data; and N is a positive even number; The matrix multiplication and accumulation circuit includes a matrix multiplication circuit and an accumulation circuit; wherein, The matrix multiplication circuit is configured to perform a multiplication operation on the first sub-matrix data and the third sub-matrix data to obtain first intermediate data; perform a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; and perform a multiplication operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain third intermediate data; The accumulating circuit is configured to accumulate the first intermediate data, the second intermediate data, and the third intermediate data to obtain an operation result; The matrix multiplication circuit includes a first matrix multiplication circuit, a second matrix multiplication circuit and a third matrix multiplication circuit; wherein, The first matrix multiplication circuit is configured to perform a multiplication operation on the first sub-matrix data and the third sub-matrix data to obtain first sub-intermediate data, and to shift the first sub-intermediate data left by N bits to obtain the first intermediate data; The second matrix multiplication circuit is configured to perform a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain the second intermediate data; The third matrix multiplication circuit is used to perform an accumulation operation on the first sub-matrix data and the second sub-matrix data to obtain first sum matrix data, perform an accumulation operation on the third sub-matrix data and the fourth sub-matrix data to obtain second sum matrix data, and perform a multiplication operation on the first sum matrix data and the second sum matrix data to obtain fourth intermediate data; after accumulating the first sub-intermediate data, the second intermediate data and the fourth intermediate data, left-shift them by N / 2 bits to obtain the third intermediate data.

2. The arithmetic circuit according to claim 1, wherein: The splitting circuit is further configured to divide the first large matrix data and the second large matrix data into blocks to obtain m first matrix blocks and m second matrix blocks, respectively; wherein the i-th matrix block among the m first matrix blocks serves as the first matrix data, and the i-th matrix block among the m second matrix blocks serves as the second matrix data; wherein i is sequentially taken from 1 to m to obtain m groups of first matrix data and second matrix data; The matrix multiplication and accumulation circuit is used to perform the multiplication and accumulation operation based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data for each group of the first matrix data and the second matrix data, to obtain m intermediate operation results; and accumulate the m intermediate operation results to obtain an operation result.

3. The operation circuit according to claim 1, wherein: The splitting circuit is further configured to divide the first large matrix data and the second large matrix data into blocks to obtain m first matrix blocks and m second matrix blocks, respectively; wherein the i-th matrix block in the m first matrix blocks is used as the first matrix vector, and the i-th matrix block in the m second matrix blocks is used as the second matrix vector; wherein i is sequentially taken from 1 to m to obtain m groups of first matrix vectors and second matrix vectors; The matrix multiplication and accumulation circuit is used to perform multiplication and accumulation operations based on the first sub-matrix vector, the second sub-matrix vector, the third sub-matrix vector and the fourth sub-matrix vector for each group of the first matrix vector and the second matrix vector, to obtain m intermediate operation results; and accumulate the m intermediate operation results to obtain an operation result.

4. The arithmetic circuit according to any one of claims 1 to 3, wherein: The first matrix data is a signed number; the first sub-matrix data and the second sub-matrix data both include a sign bit; the second matrix data is a signed number; the third sub-matrix data and the fourth sub-matrix data both include a sign bit; the operation circuit further includes an OR logic judgment circuit connected to the splitting circuit; wherein, The OR logic judgment circuit is configured to perform an addition operation on the first sub-matrix data when the non-sign bits of the second sub-matrix data are not all 0; and perform an addition operation on the third sub-matrix data when the non-sign bits of the fourth sub-matrix data are not all 0.

5. A convolution operation method for implementing a convolution operation of a first matrix data and a second matrix data, characterized in that: include: Splitting the first matrix data by a splitting circuit in a convolution operation circuit to obtain first sub-matrix data of the first N / 2 dimensions and second sub-matrix data of the last N / 2 dimensions; Splitting the second matrix data to obtain third sub-matrix data of the first N / 2 dimensions and fourth sub-matrix data of the last N / 2 dimensions; wherein both the first matrix data and the second matrix data are N-dimensional matrix data; and N is a positive even number; The matrix multiplication and accumulation circuit includes a matrix multiplication circuit and an accumulation circuit; performing a multiplication operation on the first sub-matrix data and the third sub-matrix data by the matrix multiplication circuit to obtain first intermediate data; performing a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; and performing a multiplication operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain third intermediate data; Accumulating the first intermediate data, the second intermediate data, and the third intermediate data by the accumulating circuit to obtain an operation result; The matrix multiplication circuit includes a first matrix multiplication circuit, a second matrix multiplication circuit, and a third matrix multiplication circuit; the matrix multiplication circuit performs a multiplication operation on the first sub-matrix data and the third sub-matrix data to obtain first intermediate data; performs a multiplication operation on the second sub-matrix data and the fourth sub-matrix data to obtain second intermediate data; and performs a multiplication operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain third intermediate data, including: performing a multiplication operation on the first sub-matrix data and the third sub-matrix data by the first matrix multiplication circuit to obtain first sub-intermediate data, and shifting the first sub-intermediate data left by N bits to obtain the first intermediate data; performing a multiplication operation on the second sub-matrix data and the fourth sub-matrix data by the second matrix multiplication circuit to obtain the second intermediate data; The first sub-matrix data and the second sub-matrix data are added together by the third matrix multiplication circuit to obtain first sum matrix data, the third sub-matrix data and the fourth sub-matrix data are added together to obtain second sum matrix data, and the first sum matrix data and the second sum matrix data are multiplied together to obtain fourth intermediate data; the first sub-intermediate data, the second intermediate data, and the fourth intermediate data are added together and shifted left by N / 2 bits to obtain the third intermediate data.

6. The method according to claim 5, wherein The method further comprises: The first large matrix data and the second large matrix data are divided into blocks by the splitting circuit to obtain m first matrix blocks and m second matrix blocks respectively; wherein the i-th matrix block among the m first matrix blocks is used as the first matrix data, and the i-th matrix block among the m second matrix blocks is used as the second matrix data; wherein i is sequentially taken from 1 to m to obtain m groups of first matrix data and second matrix data; The matrix multiplication and accumulation circuit in the operation circuit performs a multiplication and accumulation operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain an operation result, including: The matrix multiplication and accumulation circuit performs the multiplication and accumulation operation based on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data and the fourth sub-matrix data for each group of the first matrix data and the second matrix data to obtain m intermediate operation results; and accumulates the m intermediate operation results to obtain an operation result.

7. The method according to claim 5, wherein The method further comprises: The first large matrix data and the second large matrix data are divided into blocks by the splitting circuit to obtain m first matrix blocks and m second matrix blocks respectively; wherein the i-th matrix block in the m first matrix blocks is used as the first matrix vector, and the i-th matrix block in the m second matrix blocks is used as the second matrix vector; wherein i is sequentially taken from 1 to m to obtain m groups of first matrix vectors and second matrix vectors; The matrix multiplication and accumulation circuit in the operation circuit performs a multiplication and accumulation operation on the first sub-matrix data, the second sub-matrix data, the third sub-matrix data, and the fourth sub-matrix data to obtain an operation result, including: The matrix multiplication and accumulation circuit performs multiplication and accumulation operations on each group of the first matrix vector and the second matrix vector according to the first sub-matrix vector, the second sub-matrix vector, the third sub-matrix vector and the fourth sub-matrix vector to obtain m intermediate operation results; and accumulates the m intermediate operation results to obtain an operation result.

8. The method according to any one of claims 5 to 7, wherein: The first matrix data is a signed number; the first sub-matrix data and the second sub-matrix data both include a sign bit; the second matrix data is a signed number; The third sub-matrix data and the fourth sub-matrix data both include a sign bit; and the method further includes: Through the OR logic judgment circuit, if the non-sign bits of the second sub-matrix data are not all 0, the first sub-matrix data is added by 1; if the non-sign bits of the fourth sub-matrix data are not all 0, the third sub-matrix data is added by 1.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 5 to 8.

10. A chip, characterized in that: The chip comprises a convolution operation circuit according to any one of claims 1 to 4 and at least one vector calculation circuit coupled to the operation circuit; The at least one vector calculation circuit is used for other layer network structures in the convolutional neural network to perform calculations based on the operation results to obtain recognition results.

11. A board, characterized in that: The board comprises the chip according to claim 10 and at least one memory device and a control device coupled to the chip; The at least one storage device is used to store calculation data of the convolutional neural network; The control device is used to communicate with the chip to implement the operation of the convolutional neural network.

12. An electronic device, characterized in that: The electronic device comprises the board according to claim 11.

Images