A data processing method, apparatus, neural network accelerator, and storage medium.

Abstract

This application discloses a data processing method, apparatus, neural network accelerator, and storage medium. The apparatus includes: a conversion processing module, configured to perform a first matrix transformation operation in the Winograd algorithm on the corresponding image data for each channel in a feature image, and perform a second matrix transformation operation in the Winograd algorithm on the corresponding convolution kernel, to obtain the corresponding converted image data and convolution kernel; at least one convolution processing unit, including a number of calculation groups matching the number of channels in the feature image, each calculation group supporting the execution of a dot product operation in the Winograd algorithm on the converted image data and convolution kernel corresponding to one channel in the feature image, to obtain the corresponding dot product data; the conversion processing module is further configured to accumulate the dot product data corresponding to different channels in the feature image, and perform a third matrix transformation operation in the Winograd algorithm on the obtained accumulated data, to obtain the convolution processing result corresponding to the feature image.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a data processing method, apparatus, neural network accelerator, and storage medium. Background Technology

[0002] The architecture of an artificial intelligence accelerator is mainly divided into convolution processing acceleration units and vector processing acceleration units. Among them, convolution operation accounts for the majority of the computation in the artificial intelligence algorithm network, and the core operation is multiplication and accumulation.

[0003] Currently, the convolution operation method in hardware architecture is generally direct convolution operation, which includes a large number of multiplication and addition operations. A large number of multiplication and addition operations mean more hardware resources and power consumption, and the operation efficiency is relatively low. Summary of the Invention

[0004] This application provides a data processing method, apparatus, neural network accelerator, and storage medium, which not only improves the efficiency of convolution operations but also has high flexibility and scalability.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a data processing apparatus, including: at least one convolution processing unit, and a conversion processing module connected to the at least one convolution processing unit;

[0007] The conversion processing module is used to perform the first matrix transformation operation in the Winograd algorithm on the corresponding image data for each channel in the feature image, and to perform the second matrix transformation operation in the Winograd algorithm on the corresponding convolution kernel, so as to obtain the corresponding converted image data and convolution kernel.

[0008] The at least one convolution processing unit includes a number of computation groups matching the number of channels in the feature image. Each computation group supports performing the dot product operation in the Winograd algorithm on the transformed image data corresponding to one channel in the feature image and the convolution kernel to obtain the corresponding dot product data.

[0009] The conversion processing module is further configured to accumulate the dot product data corresponding to different channels in the feature image, and perform the third matrix transformation operation in the Winograd algorithm on the accumulated data to obtain the convolution processing result corresponding to the feature image.

[0010] In the above-described device, the conversion processing module includes: an accumulation unit;

[0011] The accumulation unit is connected to the at least one convolution processing module and is used to accumulate the dot product data corresponding to different channels in the feature image to obtain the accumulated data.

[0012] In the above-described device, the conversion processing module further includes:

[0013] A first conversion unit is deployed in the at least one convolutional processing unit;

[0014] The first conversion unit is used to perform the first matrix conversion operation on the corresponding image data for each channel in the feature image, and to perform the second matrix conversion operation on the corresponding convolution kernel, so as to obtain the corresponding converted image data and convolution kernel;

[0015] The accumulation unit is also used to perform the third matrix transformation operation on the accumulated data to obtain the convolution processing result.

[0016] In the above-described device, the conversion processing module further includes: a second conversion unit;

[0017] The second conversion unit is connected to the at least one convolution processing module and the accumulation unit;

[0018] The second conversion unit is used for:

[0019] For each channel in the feature image, the first matrix transformation operation is performed on the corresponding image data, and the second matrix transformation operation is performed on the corresponding convolution kernel to obtain the corresponding transformed image data and convolution kernel; and,

[0020] The third matrix transformation operation is performed on the accumulated data to obtain the convolution processing result.

[0021] In the above-described apparatus, each of the at least one convolution processing units includes at least one multiply-accumulate tree, and each multiply-accumulate tree includes at least one dot multiplication operator.

[0022] Each computation group includes a dot multiplication operator in each multiply-accumulate tree within a convolution processing unit;

[0023] Each computation group supports multiplying the transformed image data corresponding to one channel in the feature image with the data at the same position in the convolution kernel using a dot product operator within the group to obtain the corresponding dot product data.

[0024] In the above-described apparatus, the at least one convolution processing unit is arranged in columns.

[0025] This application provides a data processing method, including:

[0026] Obtain the number of channels in the feature image;

[0027] Based on the number of channels, at least one convolutional processing unit is selected from the convolutional processing unit array; wherein, the at least one convolutional processing unit includes a number of computation groups matching the number of channels;

[0028] Using the transformation processing module, for each channel in the feature image, the first matrix transformation operation in the Winograd algorithm is performed on the corresponding image data, and the second matrix transformation operation in the Winograd algorithm is performed on the corresponding convolution kernel to obtain the corresponding transformed image data and convolution kernel;

[0029] By utilizing different computation groups in at least one convolution processing unit, the dot product operation in the Winograd algorithm is performed on the transformed image data and convolution kernel corresponding to different channels in the feature image to obtain the corresponding dot product data.

[0030] Using the conversion processing module, the dot product data corresponding to different channels in the feature image are accumulated, and the third matrix transformation operation in the Winograd algorithm is performed on the accumulated data to obtain the convolution processing result corresponding to the feature image.

[0031] This application provides a neural network accelerator, including the data processing device described above.

[0032] This application provides a data processing apparatus, including: a processor, a memory, and a communication bus;

[0033] The communication bus is used to realize the communication connection between the processor and the memory;

[0034] The processor is configured to execute one or more programs stored in the memory to implement the above-described data processing method.

[0035] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described data processing method.

[0036] This application provides a data processing method, apparatus, neural network accelerator, and storage medium. The data processing apparatus includes: at least one convolution processing unit and a conversion processing module connected to the at least one convolution processing unit; the conversion processing module is used to perform a first matrix transformation operation in the Winograd algorithm on the corresponding image data for each channel in the feature image, and to perform a second matrix transformation operation in the Winograd algorithm on the corresponding convolution kernel, to obtain the corresponding converted image data and convolution kernel; the at least one convolution processing unit includes a number of calculation groups matching the number of channels in the feature image, each calculation group supporting the execution of a dot product operation in the Winograd algorithm on the converted image data and convolution kernel corresponding to one channel in the feature image, to obtain the corresponding dot product data; the conversion processing module is also used to accumulate the dot product data corresponding to different channels in the feature image, and to perform a third matrix transformation operation in the Winograd algorithm on the obtained accumulated data, to obtain the convolution processing result corresponding to the feature image. The technical solution provided in this application, firstly, from an algorithmic perspective, uses the Winograd algorithm to perform convolution operations, mapping data to another space and replacing some multiplication operations with addition and subtraction operations. Under the premise that "addition and subtraction operations are much faster than multiplication operations", it achieves a significant acceleration effect. For the deployment of artificial intelligence network terminals, its operation speed is higher than that of traditional convolution operations. From an architectural perspective, in this solution, a single hardware architecture can simultaneously complete traditional convolution operations and the Winograd fast convolution algorithm, and the hardware architecture has high flexibility and scalability. Attached Figure Description

[0037] Figure 1 A schematic diagram of an exemplary convolution operation provided for an embodiment of this application;

[0038] Figure 2 This is a schematic diagram illustrating an exemplary conversion of convolution operations into matrix operations, provided for an embodiment of this application.

[0039] Figure 3 A schematic diagram illustrating an exemplary simplified matrix operation provided in this application embodiment;

[0040] Figure 4 A schematic diagram of an exemplary multichannel convolution operation provided in this application embodiment;

[0041] Figure 5 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 1 ;

[0042] Figure 6 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 2 ;

[0043] Figure 7 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 3 ;

[0044] Figure 8 This is a schematic diagram of the structure of a convolutional processing unit array provided in an embodiment of this application;

[0045] Figure 9 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0046] Figure 10 An exemplary grouping diagram provided for an embodiment of this application;

[0047] Figure 11 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 4 . Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] The technical solutions of this application and how they solve the aforementioned technical problems will be described in detail below through embodiments and in conjunction with the accompanying drawings. The embodiments below can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0050] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0051] This application provides a data processing method, apparatus, and storage medium. The specific data processing involves the Winograd algorithm, which performs matrix transformations on convolution operations, effectively reducing the number of multiplication operations in convolution. The Winograd algorithm will be explained below.

[0052] For convolution operations, the input feature image is convolved with a convolution kernel to obtain the output result. For example... Figure 1As shown, a 4×4 feature image convolved with a 3×3 kernel yields a 2×2 image. During the entire convolution process, each convolution window requires 9 multiplications, and the results of these 9 multiplications need to be summed, requiring an additional 8 additions. For a 4×4 feature image, there are a total of 4 convolution windows. Therefore, completing the convolution requires a total of 36 multiplications and 32 additions.

[0053] against Figure 1 The convolution operation between the 4×4 feature image and the 3×3 convolution kernel shown can be converted into the following: Figure 2 The matrix operations shown can be further divided into blocks to obtain... Figure 3 The simplified diagram shown below contains the following specific formulas:

[0054]

[0055] Where g = [W0, W1, W2], d = [F0, F1, F2, F3]

[0056] And W0 = [w0, w1, w2] T W1 = [w3, w4, w5] T W2 = [w6, w7, w8] T

[0057] F0 = [f0, f1, f2, f3] T

[0058] F1 = [f4, f5, f6, f7] T

[0059] F2 = [f8, f9, f10, f11] T

[0060] F3 = [f12, f13, f14, f15] T

[0061] Therefore, formula (1) can be simplified to the following formula (2):

[0062] F(2×2, 3×3)=A T [U⊙V]A (2)

[0063] Where, U = GgG T V = B T dB

[0064]

[0065]

[0066]

[0067] V=B T dB requires 32 additions, M = U⊙V requires 16 multiplications, and ATMA requires 16 additions, so a total of 16 multiplications and 48 additions are needed, which greatly reduces the number of multiplications that need to be calculated for convolution.

[0068] It should be noted that the above formula derivation is for convolution operations with a single-channel 3*3 sliding window. For multi-channel convolution window operations, the results obtained from each channel need to be added together.

[0069] like Figure 4 As shown, for the multi-channel Winograd algorithm, where d represents the tensor of the feature image, its single-channel window size is 4×4, after V=B T After the dB transformation, the size remains unchanged, g is the convolution kernel tensor, and after U = GgG T After the transformation, the single-channel size becomes 4×4. After dot multiplication, the result is added to obtain a final 4×4 result, which is then processed by A. T The MA transformation yields the final result, which is a 4×4 feature image that, after being convolved by a 3×3 convolution kernel, produces a 2×2 output.

[0070] The following section details the data processing device based on the winograd algorithm.

[0071] Figure 5 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 1 .like Figure 5 As shown, the data processing apparatus includes: at least one convolution processing unit 10, and a conversion processing module 11 connected to the at least one convolution processing unit 10;

[0072] The transformation processing module 11 is used to perform the first matrix transformation operation in the Winograd algorithm on the corresponding image data for each channel in the feature image, and to perform the second matrix transformation operation in the Winograd algorithm on the corresponding convolution kernel, so as to obtain the corresponding transformed image data and convolution kernel.

[0073] At least one convolution processing unit 10 includes a number of computation groups matching the number of channels in the feature image. Each computation group supports performing a dot product operation in the Winograd algorithm on the transformed image data corresponding to one channel in the feature image and the convolution kernel to obtain the corresponding dot product data.

[0074] The conversion processing module 11 is also used to accumulate the dot product data corresponding to different channels in the feature image, and perform the third matrix transformation operation in the Winograd algorithm on the accumulated data to obtain the convolution processing result corresponding to the feature image.

[0075] It should be noted that, in the embodiments of this application, the data processing device includes at least one convolution processing unit 10, which may be a subset of units selected from a preset convolution processing unit matrix.

[0076] It should be noted that, in the embodiments of this application, the data processing device includes at least one convolution processing unit 10 and a conversion processing module 11 connected to at least one convolution processing unit 10, wherein the conversion processing module 11 can perform corresponding matrix conversion operations on the image data and convolution kernel corresponding to each channel in the feature image respectively.

[0077] It should be noted that, in the embodiments of this application, the feature image can be any feature image, which has at least one channel. The specific feature image and the number of channels in the feature image are not limited in the embodiments of this application.

[0078] It is understood that, in the embodiments of this application, the first matrix transformation operation performed by the conversion processing module 11 for the image data d of each channel, i.e., the above-mentioned V=B T The dB matrix transformation process involves a second matrix transformation operation performed on the convolution kernel g corresponding to each channel, i.e., the above U = GgG. T Matrix transformation processing.

[0079] It should be noted that in the embodiments of this application, in at least one convolution processing unit 10, the number of computation groups matches the number of channels in the feature image. The number of computation groups can be equal to the number of channels in the feature image, so that the computing resources can be fully utilized. Of course, the number of computation groups can also be slightly greater than the number of channels. This application embodiment does not limit this.

[0080] It should be noted that, in the embodiments of this application, a computing group can perform a dot product between the transformed image data corresponding to one channel in the feature image and the convolution kernel. Therefore, by using different computing groups, the dot product between the transformed image data corresponding to different channels in the feature image and the convolution kernel can be achieved, thereby obtaining the dot product data corresponding to different channels in the feature image.

[0081] It should be noted that, in the embodiments of this application, the conversion processing module 11 can also accumulate the dot product data corresponding to different channels in the feature image, thereby performing the third matrix transformation operation in the Winograd algorithm on the accumulated data M, i.e., the aforementioned A TThe matrix transformation process of MA yields the convolution result corresponding to the feature image.

[0082] In the embodiments of this application, such as Figure 6 and Figure 7 As shown, the conversion processing module 11 includes: an accumulation unit 110;

[0083] The accumulation unit 110 is connected to at least one convolution processing module 10 and is used to accumulate the dot product data corresponding to different channels in the feature image to obtain accumulated data.

[0084] It should be noted that, in the embodiments of this application, the conversion processing module 11 may include an accumulation unit 110, the function of which is to accumulate the dot product data corresponding to different channels in the feature image.

[0085] like Figure 6 As shown in the embodiments of this application, the conversion processing module 11 further includes: a first conversion unit 111, which is deployed in at least one convolution processing unit 10;

[0086] The first conversion unit 111 is used to perform a first matrix conversion operation on the corresponding image data for each channel in the feature image, and to perform a second matrix conversion operation on the corresponding convolution kernel, so as to obtain the corresponding converted image data and convolution kernel.

[0087] The accumulator unit 110 is also used to perform a third matrix transformation operation on the accumulated data to obtain the convolution processing result.

[0088] It should be noted that, in the embodiments of this application, as... Figure 6 As shown, the conversion processing module 11 includes not only the above-mentioned accumulation unit 110, but also a first conversion unit 111. The first conversion unit 111 can actually include at least one. A first conversion unit 111 can be deployed in a convolution processing unit 10 to obtain the converted image data and convolution kernels corresponding to some channels in the feature images that each calculation group in the convolution processing unit 10 needs to process.

[0089] It is understood that, in the embodiments of this application, as Figure 6 The structure shown actually implements matrix transformations within the convolution processing unit 10.

[0090] like Figure 7 As shown, in the embodiments of this application, the conversion processing module 11 includes not only the above-mentioned accumulation unit 110, but also: a second conversion unit 112;

[0091] The second conversion unit 112 is connected to at least one convolution processing module and an accumulation unit 110;

[0092] The second conversion unit 112 is used for:

[0093] For each channel in the feature image, a first matrix transformation operation is performed on the corresponding image data, and a second matrix transformation operation is performed on the corresponding convolution kernel to obtain the corresponding transformed image data and convolution kernel; and,

[0094] Perform a third matrix transformation operation on the accumulated data to obtain the convolution result.

[0095] It should be noted that, in the embodiments of this application, as... Figure 7 As shown, the conversion processing module 11 includes not only an accumulation unit 110, but also a second conversion unit 112. The second conversion unit 112 is connected to at least one convolution processing module and the accumulation unit 110. Thus, the second conversion unit 112 performs the first matrix conversion operation and the second matrix conversion operation. Furthermore, since the second conversion unit 112 is also connected to the accumulation unit 110, when the accumulation unit 110 accumulates the dot product data corresponding to different channels in the feature image to obtain accumulated data, the second conversion unit 112 can perform the third matrix conversion operation.

[0096] It is understood that, in the embodiments of this application, as Figure 7 The structure shown actually implements matrix transformations outside of the accumulation unit 110 and the convolution processing unit 10, and different types of matrix transformations are all performed in the same unit, namely the second transformation unit 112.

[0097] It should be noted that, in the embodiments of this application, as... Figure 6 and Figure 7 As shown, the feature image is actually stored in the memory. Both the first conversion unit 111 and the second conversion unit 112 can obtain the feature image from the memory for further processing.

[0098] Specifically, in the embodiments of this application, such as Figure 6 and Figure 7 As shown, in at least one convolution processing unit 10, each convolution processing unit 10 includes at least one multiply-accumulate tree, and each multiply-accumulate tree includes at least one dot multiplication operator;

[0099] Each computation group includes a dot multiplication operator in each multiply-accumulate tree within a convolution processing unit 10;

[0100] Each computation group supports multiplying the transformed image data corresponding to one channel in the feature image with the data at the same position in the convolution kernel using a dot product operator within the group to obtain the corresponding dot product data.

[0101] It should be noted that, in the embodiments of this application, as... Figure 6 and Figure 7 As shown, each convolution processing unit 10 can include at least one multiply-accumulate tree. The data at the same position in the transformed image data corresponding to a channel in the feature image and the convolution kernel are multiplied using a dot multiplication operator on the multiply-accumulate tree. In fact, for the transformed image data and the convolution kernel corresponding to a channel, there is a corresponding computation group. The part enclosed by the dashed line in the convolution processing unit 10 is a computation group. The data corresponding to the position is multiplied using a dot multiplication operator in this computation group. In this way, all dot multiplication operations for the data corresponding to a channel can be achieved through this computation group.

[0102] For example, in the embodiments of this application, there are 4 convolution processing units 10, each convolution processing unit 10 includes 16 multiply-accumulate trees, and each multiply-accumulate tree includes 9 dot multiplication operators. In this way, these convolution units actually include 4×9=36 calculation groups, which can realize the data processing of 36 channels in the feature image, and the amount of data involved in each channel is 16.

[0103] Specifically, in the embodiments of this application, at least one convolution processing unit 10 is arranged in columns.

[0104] It should be noted that, in the embodiments of this application, the data processing for a frame of feature image actually involves at least one convolution processing unit 10 being arranged in columns and connected to the accumulation unit 110. In this way, the accumulation unit 110 realizes the accumulation of the dot product data output by at least one convolution processing unit 10 in the vertical direction, that is, the accumulation of the dot product data corresponding to a frame of feature image.

[0105] It should be noted that, in the embodiments of this application, as... Figure 8 As shown, the data processing device may actually include multiple columns of convolution processing units 10 to form a convolution processing unit array. Each column of convolution processing units is connected to an accumulation unit 110, so that fast convolution processing of multiple frames of feature images can be completed in parallel. This application embodiment does not limit the scope of the invention.

[0106] This application provides a data processing apparatus, including: at least one convolution processing unit and a conversion processing module connected to the at least one convolution processing unit; the conversion processing module is used to perform a first matrix transformation operation in the Winograd algorithm on the corresponding image data for each channel in the feature image, and to perform a second matrix transformation operation in the Winograd algorithm on the corresponding convolution kernel, to obtain the corresponding converted image data and convolution kernel; the at least one convolution processing unit includes a number of calculation groups matching the number of channels in the feature image, each calculation group supporting the execution of a dot product operation in the Winograd algorithm on the converted image data and convolution kernel corresponding to one channel in the feature image, to obtain the corresponding dot product data; the conversion processing module is further used to accumulate the dot product data corresponding to different channels in the feature image, and to perform a third matrix transformation operation in the Winograd algorithm on the obtained accumulated data, to obtain the convolution processing result corresponding to the feature image. The data processing apparatus provided in this application embodiment, firstly, from an algorithmic perspective, uses the Winograd algorithm to perform convolution operations, mapping data to another space and replacing some multiplication operations with addition and subtraction operations. Under the premise that "addition and subtraction operations are much faster than multiplication operations", it achieves a significant acceleration effect. For deployment at the terminal of the artificial intelligence network, its processing speed is higher than that of traditional convolution operations. From an architectural perspective, in this solution, a single hardware architecture can simultaneously complete traditional convolution operations and the Winograd fast convolution algorithm, and the hardware architecture has high flexibility and scalability.

[0107] This application provides a data processing method, implemented using the aforementioned data processing apparatus. Figure 9 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 9 As shown, the data processing methods include:

[0108] S201. Obtain the number of channels in the feature image.

[0109] In embodiments of this application, the data processing device can obtain the number of channels in the feature image.

[0110] It should be noted that, in the embodiments of this application, the feature image can be any feature image, wherein there is at least one channel. The specific feature image and the number of channels in the feature image are not limited in the embodiments of this application. The data processing device can directly obtain the number of channels from the feature image.

[0111] S202. Based on the number of channels, select at least one convolutional processing unit from the convolutional processing unit array; wherein, at least one convolutional processing unit includes a number of computation groups matching the number of channels.

[0112] In the embodiments of this application, combined with Figure 5 After obtaining the number of channels in the feature image, the data processing device can select at least one convolution processing unit 10 from the convolution processing unit array based on the number of channels; wherein, the at least one convolution processing unit 10 includes a number of calculation groups that match the number of channels.

[0113] It should be noted that, in the embodiments of this application, the data processing device can select units in the convolution processing unit array based on the number of channels of the feature image. The number of calculation groups included in at least one selected convolution processing unit 10 must be at least equal to the number of channels, so as to support the realization of fast convolution processing of feature images.

[0114] It is understood that in the embodiments of this application, for each computation group included in each convolution processing unit 10, the processing of relevant data corresponding to one channel in the feature image can be realized. The data processing device selects at least one convolution processing unit 10 from the convolution processing unit array based on the number of channels. In fact, it groups the convolution processing units 10 so that the number of computation groups included in at least one convolution processing unit 10 is sufficient to support the processing of relevant data corresponding to different channels of the feature image. In this way, at least one convolution processing unit 10 as a whole finally outputs a single processing result, which brings better flexibility to the data processing device for data processing and has high flexibility and scalability.

[0115] Figure 10 This is an exemplary grouping diagram provided for an embodiment of this application. For example... Figure 10 As shown, a large number of convolutional processing units 10 form an array, and these convolutional processing units 10 can be vertically grouped according to actual needs. Different groups can bring certain advantages, but their corresponding hardware resources are also different. Each group can output a separate result. Smaller groups can bring better flexibility, but it means more hardware resources to process and transmit their output results. Larger groups can use less hardware resources to process and transmit the results of each group, but when processing data with small input channels, it is still necessary to allocate them to a group for processing, which means that some hardware resources are wasted.

[0116] In the embodiments of this application, when at least one convolution processing unit 10 is selected, the data processing device uses the at least one convolution processing unit 10 and the transformation processing module 11 connected to the at least one convolution processing unit 10 to perform fast convolution processing on the feature image using the Winograd algorithm to obtain the convolution processing result corresponding to the feature image.

[0117] S203. Using the transformation processing module, for each channel in the feature image, perform the first matrix transformation operation in the Winograd algorithm on the corresponding image data, and perform the second matrix transformation operation in the Winograd algorithm on the corresponding convolution kernel to obtain the corresponding transformed image data and convolution kernel.

[0118] S204. Using different computation groups in at least one convolution processing unit, perform the dot product operation in the Winograd algorithm on the transformed image data corresponding to different channels in the feature image and the convolution kernel to obtain the corresponding dot product data.

[0119] S205. Using the transformation processing module, the dot product data corresponding to different channels in the feature image are accumulated, and the third matrix transformation operation in the Winograd algorithm is performed on the accumulated data to obtain the convolution processing result corresponding to the feature image.

[0120] It is understood that, in the embodiments of this application, at least one convolution processing unit 10 is connected to the transformation processing module 11. The transformation processing module 11 can perform the matrix transformation operation involved in the Winograd algorithm on the image data and convolution kernel corresponding to each channel in the feature image. That is, the transformation processing module 11 transforms the image data d of each channel according to V=B. T dB performs matrix transformation processing, and the convolution kernel g corresponding to each channel is processed according to U = GgG T Matrix transformation processing is performed. Furthermore, the transformation processing module 11 also performs matrix transformation processing on the accumulated results of the dot product data obtained from different calculation groups in at least one convolution processing unit 10, according to A... T MA performs matrix transformation. Specifically, the B, G, and A matrices involved in the transformation process are the specific matrices defined in the Winograd algorithm described above.

[0121] It should be noted that, in the embodiments of this application, the conversion processing module 11 connected to at least one convolution processing unit 10 includes not only the accumulation unit 110, but also a first conversion unit 111 or a second conversion unit 112. The first conversion unit 111 can be deployed inside at least one convolution processing unit 10, and the second conversion unit 112 can be deployed outside at least one convolution processing unit 10. The functions defined by the two units are different. However, the functions implemented by the two units in combination with the accumulation unit 110 can cover the specific conversion and accumulation processes described above.

[0122] This application provides a data processing apparatus. Figure 11 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 4 .like Figure 11As shown, the data processing device includes: a processor 301, a memory 302, and a communication bus 303;

[0123] The communication bus 303 is used to realize the communication connection between the processor 301 and the memory 302;

[0124] The processor 301 is used to execute one or more programs stored in the memory 302 to implement the above-described data processing method.

[0125] This application also provides a neural network accelerator, which includes the above-described data processing device, thereby enabling the execution of corresponding data processing methods.

[0126] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned data processing method. The computer-readable storage medium may be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it may be a device including one or any combination of the above-mentioned memories, such as a mobile phone, computer, tablet device, or personal digital assistant.

[0127] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0128] This application is described with reference to schematic and / or block diagrams of implementations of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the schematic and / or block diagrams can be implemented by computer program instructions, and combinations of blocks in the schematic and / or block diagrams can be implemented. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the schematic and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the implementation flow diagram. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing apparatus, characterized in that, include: At least one convolution processing unit, and a transformation processing module connected to the at least one convolution processing unit; The conversion processing module is used to perform the first matrix transformation operation in the Winograd algorithm on the corresponding image data for each channel in the feature image, and to perform the second matrix transformation operation in the Winograd algorithm on the corresponding convolution kernel, so as to obtain the corresponding converted image data and convolution kernel. The at least one convolution processing unit includes a number of computation groups matching the number of channels in the feature image. Each computation group supports performing the dot product operation in the Winograd algorithm on the transformed image data corresponding to one channel in the feature image and the convolution kernel to obtain the corresponding dot product data. The conversion processing module is also used to accumulate the dot product data corresponding to different channels in the feature image, and perform the third matrix transformation operation in the Winograd algorithm on the accumulated data to obtain the convolution processing result corresponding to the feature image. In the at least one convolution processing unit, each convolution processing unit includes at least one multiply-accumulate tree, and each multiply-accumulate tree includes at least one dot multiplication operator; Each computation group includes a dot multiplication operator in each multiply-accumulate tree within a convolution processing unit; Each computation group supports multiplying the transformed image data corresponding to one channel in the feature image with the data at the same position in the convolution kernel using a dot product operator within the group to obtain the corresponding dot product data.

2. The apparatus according to claim 1, characterized in that, The conversion processing module includes: an accumulation unit; The accumulation unit is connected to the at least one convolution processing module and is used to accumulate the dot product data corresponding to different channels in the feature image to obtain the accumulated data.

3. The apparatus according to claim 2, characterized in that, The conversion processing module further includes: A first conversion unit is deployed in the at least one convolutional processing unit; The first conversion unit is used to perform the first matrix conversion operation on the corresponding image data for each channel in the feature image, and to perform the second matrix conversion operation on the corresponding convolution kernel, so as to obtain the corresponding converted image data and convolution kernel; The accumulation unit is also used to perform the third matrix transformation operation on the accumulated data to obtain the convolution processing result.

4. The apparatus according to claim 2, characterized in that, The conversion processing module further includes: a second conversion unit; The second conversion unit is connected to the at least one convolution processing module and the accumulation unit; The second conversion unit is used for: For each channel in the feature image, the first matrix transformation operation is performed on the corresponding image data, and the second matrix transformation operation is performed on the corresponding convolution kernel to obtain the corresponding transformed image data and convolution kernel; and, The third matrix transformation operation is performed on the accumulated data to obtain the convolution processing result.

5. The apparatus according to claim 1, characterized in that, The at least one convolution processing unit is arranged in columns.

6. A data processing method, characterized in that, include: Obtain the number of channels in the feature image; Based on the number of channels, at least one convolutional processing unit is selected from the convolutional processing unit array; wherein, the at least one convolutional processing unit includes a number of computation groups matching the number of channels; Using the transformation processing module, for each channel in the feature image, the first matrix transformation operation in the Winograd algorithm is performed on the corresponding image data, and the second matrix transformation operation in the Winograd algorithm is performed on the corresponding convolution kernel to obtain the corresponding transformed image data and convolution kernel; By utilizing different computation groups in at least one convolution processing unit, the dot product operation in the Winograd algorithm is performed on the transformed image data and convolution kernel corresponding to different channels in the feature image to obtain the corresponding dot product data. Using the conversion processing module, the dot product data corresponding to different channels in the feature image are accumulated, and the third matrix transformation operation in the Winograd algorithm is performed on the accumulated data to obtain the convolution processing result corresponding to the feature image. In the at least one convolution processing unit, each convolution processing unit includes at least one multiply-accumulate tree, and each multiply-accumulate tree includes at least one dot multiplication operator; Each computation group includes a dot multiplication operator in each multiply-accumulate tree within a convolution processing unit; The step of performing the dot product operation of the Winograd algorithm on the transformed image data corresponding to different channels in the feature image and the convolution kernel using different computation groups in at least one convolution processing unit to obtain the corresponding dot product data includes: Using each computation group, the transformed image data corresponding to one channel in the feature image and the data at the same position in the convolution kernel are multiplied by a dot product operator within the group to obtain the corresponding dot product data.

7. A neural network accelerator, characterized in that, Includes the data processing apparatus as described in any one of claims 1-5.

8. A data processing apparatus, characterized in that, include: Processor, memory, and communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is configured to execute one or more programs stored in the memory to implement the data processing method of claim 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data processing method as described in claim 6.

Images