Data processing method and device based on convolution

Abstract

The embodiment of the present application provides a data processing method and device based on convolution, wherein the method includes: reading the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, and reading the feature data corresponding to the convolution dimension under each moving window in the data to be processed; performing convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain each output data; according to the output data obtained on each feature data, obtaining the data combination on each output dimension to obtain the output data. Reduce the number of times the convolution kernel storage unit is read, and reduce the number of times the feature data is read. Reduce the occupation of the internal data transmission bandwidth, do not need to occupy more internal data transmission bandwidth, and reduce the impact on other data transmission and data processes.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a convolution-based data processing method and device. Background Art

[0002] With the development of artificial intelligence technology, intelligent computing devices have begun to be widely used. They can be applied to various technical fields, such as big data processing, computing power, and image processing. In addition, since deep learning has become an important algorithm in artificial intelligence technology, computing devices have gradually begun to run convolutional neural network algorithms in deep learning algorithms to complete data operations.

[0003] In the prior art, when a computing device runs a convolutional neural network algorithm to perform data computing, the computing device needs to read feature data under a window size and read a convolution kernel storage unit; then, the computing device performs a convolution operation between the feature data under a window size and a convolution kernel storage unit. The computing device then reads the feature data under the next window size and reads the next convolution kernel storage unit, and performs a convolution operation between the feature data read at this time and the convolution kernel storage unit. And so on. Then the computing device completes the data computing.

[0004] However, in the prior art, when a computing device runs a convolutional neural network algorithm to perform data calculations, the computing device needs to read the feature data and the convolution kernel storage unit multiple times, and the number of reads is relatively high. This will cause the internal data transmission bandwidth of the computing device to be occupied more, and the demand for the internal data transmission bandwidth is relatively high. Even when the computing device performs convolution operations, it will affect other data transmission and data processes. Summary of the invention

[0005] Based on this, it is necessary to provide a convolution-based data processing method and device to address the above technical problems.

[0006] In a first aspect, the present application provides a convolution-based data processing method, comprising:

[0007] Reading the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, and reading the feature data corresponding to the convolution dimension under each moving window in the data to be processed; wherein the data to be processed is a data matrix, the data to be processed includes at least one feature data, the convolution array includes at least one convolution kernel storage unit, and each of the convolution kernel storage units includes weight parameter units on different convolution dimensions;

[0008] Perform convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain output data;

[0009] According to the output data obtained on each of the feature data, a data combination on each output dimension is obtained to obtain output data, wherein the output data includes a data combination on each of the output dimensions.

[0010] In a second aspect, the present application provides a computing device, including: a processor;

[0011] The processor is configured to:

[0012] Reading the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, and reading the feature data corresponding to the convolution dimension under each moving window in the data to be processed; wherein the data to be processed is a data matrix, the data to be processed includes at least one feature data, the convolution array includes at least one convolution kernel storage unit, and each of the convolution kernel storage units includes weight parameter units on different convolution dimensions;

[0013] Perform convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain output data;

[0014] According to the output data obtained on each of the feature data, a data combination on each output dimension is obtained to obtain output data, wherein the output data includes a data combination on each of the output dimensions.

[0015] In a third aspect, the present application provides a processing device, the processing device comprising one or more computing devices according to the first aspect of the embodiment of the present application, configured to execute a specified data operation and transmit the execution result to other processing devices through an I / O interface;

[0016] When the processing device includes a plurality of the computing devices, the plurality of computing devices may be connected and transmit data through a specific structure;

[0017] Among them, multiple computing devices are interconnected and transmit data through a fast peripheral device interconnect bus PCIE bus; multiple computing devices share the same control system or have their own control systems; multiple computing devices share a memory unit or have their own memory units.

[0018] In a fourth aspect, the present application provides a combined processing device, including the processing device as described in the third aspect of the embodiment of the present application, a universal interconnection interface and other processing devices;

[0019] The processing device interacts with the other processing devices to jointly complete the operation specified by the user.

[0020] In a fifth aspect, the present application provides a chip, comprising the processing device as described in the third aspect of the embodiment of the present application or the combined processing device as described in the fourth aspect of the embodiment of the present application.

[0021] In a sixth aspect, the present application provides a board card, which includes the chip as described in the fifth aspect of the embodiment of the present application.

[0022] In a seventh aspect, the present application provides an electronic device, which includes the chip as described in the fifth aspect of the embodiment of the present application or the board as described in the sixth aspect of the embodiment of the present application.

[0023] As described above, in the scheme of the embodiment of the present application, when reading the convolution kernel storage unit, the weight parameter unit on the same convolution dimension in each convolution kernel storage unit can be read, thereby reducing the number of times the convolution kernel storage unit is read; and, when reading the data to be processed, the feature data corresponding to the convolution dimension under each moving window can be read, thereby eliminating the need to read the same feature data multiple times, thereby reducing the number of times the feature data is read, thereby reducing the number of times the data to be processed is read; then, the read feature data and the convolution kernel storage unit are subjected to convolution operations. Since the number of times the data to be processed is read is reduced, the occupancy of the internal data transmission bandwidth can be reduced, and there is no need to occupy more internal data transmission bandwidth, thereby reducing the impact on other data transmission and data processes. Furthermore, the number of times the convolution kernel storage unit is read can also be reduced, further reducing the occupancy of the internal data transmission bandwidth. Thus, hardware acceleration of the computing device is further achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The application scenario provided by the embodiment of the present application is shown in FIG. Figure 1 ;

[0025] Figure 2 The application scenario provided by the embodiment of the present application is shown in FIG. Figure 2 ;

[0026] Figure 3 The application scenario provided by the embodiment of the present application is shown in FIG. Figure 3 ;

[0027] Figure 4 A schematic diagram of a convolutional neural network algorithm provided in an embodiment of the present application;

[0028] Figure 5 A flowchart of a convolution-based data processing method provided in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of a convolution kernel storage unit provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of a convolution array provided in an embodiment of the present application;

[0031] Figure 8 A schematic diagram of data to be processed provided in an embodiment of the present application;

[0032] Fig. 9 A schematic diagram of a flow chart of another convolution-based data processing method provided in an embodiment of the present application;

[0033] Fig.10 Schematic diagram of the convolution operation process provided in the embodiment of the present application Figure 1 ;

[0034] Fig.11 Schematic diagram of the convolution operation process provided in the embodiment of the present application Figure 2 ;

[0035] Fig.12 A schematic diagram of a computing device provided in an embodiment of the present application;

[0036] Fig.13 A schematic diagram of another computing device provided in an embodiment of the present application;

[0037] Fig.14 A schematic diagram of a processing device provided in an embodiment of the present application;

[0038] Fig.15 A schematic diagram of a combined processing device provided in an embodiment of the present application;

[0039] Fig.16 A schematic diagram of a chip provided in an embodiment of the present application Figure 1 ;

[0040] Fig.17 A schematic diagram of a chip provided in an embodiment of the present application Figure 2 ;

[0041] Fig.18 A schematic diagram of a chip provided in an embodiment of the present application Figure 3 ;

[0042] Fig.19 A schematic diagram of a board provided in an embodiment of the present application;

[0043] Fig. 20A schematic diagram of an electronic device provided in an embodiment of the present application Figure 1 ;

[0044] Fig.21 A schematic diagram of an electronic device provided in an embodiment of the present application Figure 2 ;

[0045] Fig. 22 A schematic diagram of an electronic device provided in an embodiment of the present application Figure 3 . DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0047] It should be understood that the terms "first", "second", etc. in the claims, specification and drawings of the present application are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0048] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0049] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0050] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0051] With the development of artificial intelligence technology, intelligent computing devices have begun to be widely used. Intelligent computing devices can be applied to various technical fields, such as big data processing, computing power, and image processing.

[0052] Moreover, since deep learning has become an important algorithm in artificial intelligence technology, intelligent computing devices have gradually begun to adopt deep learning algorithms. Among them, since the convolutional neural network algorithm in the deep learning algorithm has the advantages of translation invariance and local correlation in time and space dimensions, computing devices can use convolutional neural network (CNN) algorithm to complete data calculations.

[0053] In addition, the convolutional neural network algorithm has a huge amount of calculation and a large amount of reusable data. Therefore, the computing device running the convolutional neural network algorithm to complete data calculation can effectively complete hardware parallel acceleration. It can be seen that the computing device running the convolutional neural network algorithm to complete data calculation has a beneficial hardware acceleration effect.

[0054] Figure 1 The application scenario provided by the embodiment of the present application is shown in FIG. Figure 1 ,like Figure 1 As shown, the computing device can be applied to face recognition. The computing device uses a convolutional neural network algorithm to recognize and detect face images, and then obtains a detection result, which can be an image or other data. Since the computing device runs the convolutional neural network algorithm to complete data operations, hardware parallel acceleration can be effectively completed. Therefore, when the computing device uses the convolutional neural network algorithm to process face images, the processing speed of the face images is faster.

[0055] Figure 2 The application scenario provided by the embodiment of the present application is shown in FIG. Figure 2 ,like Figure 2 As shown, the computing device can be applied to computing power calculations. The computing device can be a controller or processor of a server, or the computing device can be a chip of a computing power board. The computing device can run a convolutional neural network algorithm to complete computing power calculations. Since the computing device runs a convolutional neural network algorithm to complete data calculations, hardware parallel acceleration can be effectively completed. Therefore, when the computing device uses a convolutional neural network algorithm to perform computing power calculations, the hardware execution speed of the computing device is faster.

[0056] Figure 3Schematic diagram of the application scenario provided by the embodiments of the present application Figure 3 , such as Figure 3 shown, the computing device can be applied to a computer cluster. The computer cluster includes multiple computers, and at least one computing device is configured in each computer. The computing devices of each computer in the computer cluster can run the convolutional neural network algorithm to complete big data computing.

[0057] In one example, during the process of the computing device running the convolutional neural network algorithm to complete data computing, Figure 4 is a schematic diagram of the convolutional neural network algorithm provided by the embodiments of the present application. As Figure 4 shown, there are multiple convolution kernels (Kernels) in the convolutional neural network algorithm. During the computing process, the computing device performs computing between the acquired data and the convolution kernels.

[0058] The computing process between the data and the convolution kernels is equivalent to that the convolution kernel slides (moves) on the plane of the data with a preset window size (such as 2 x 2, or 3 x 3, etc.); each time the convolution kernel slides, the convolution kernel performs multiplication and accumulation with the data covered by the convolution kernel on the input plane, and after adding the corresponding bias, then outputs a value on the corresponding layer. Among them, the number of layers (depth) of the input data is the same as the number of layers (depth) of each convolution kernel. And when computing between the data and M convolution kernels, since each convolution kernel corresponds to a bias and an output plane, thus, M output planes can be obtained.

[0059] In one example, the convolutional neural network algorithm adopted by the computing device can adopt the data output formula as follows:

[0060]

[0061] where 0≤z<N, 0≤u<M, 0≤y<E=(H-R+U) / U, 0≤x<F=(W-S+U) / U. In the above data output process, the computing device inputs the data to be processed into the convolutional neural network algorithm, and after being processed by the convolution kernels, the output data is O[u][x][y]. u, x, and y are three dimensions. k, i, and j are integers greater than or equal to 0, k∈[0,C-1], i∈[0,R-1], j∈[0,S-1], C, R, and S are integers greater than or equal to 1. M, N, H, R, U, W, and S are integers greater than or equal to 1. z, u, y, and x are integers greater than or equal to 0. I is used to represent the data to be processed.

[0062] During the process of the computing device running the above convolutional neural network algorithm to complete data computing, the computing process between the data and the convolution kernels requires multiplication operations and addition operations.

[0063] For example, the data IF to be processed acquired by the computing device is as shown in Table 1, and each convolution kernel in the convolutional neural network algorithm is as shown in Table 2. Among them, the size of the data IF to be processed is 4x4, and the size of each convolution kernel is 3x3.

[0064] Table 1 Data to be processed IF

[0065] I0 I1 I2 I3 I4 I5 I6 I7 I8 I9 I10 I11 I12 I13 I14 I15

[0066] Table 2 Convolution kernel

[0067] k0 k1 k2 k3 k4 k5 k6 k7 k8

[0068] Table 3 Output data

[0069] R0 R1 R2 R3

[0070] A convolution kernel shown in Table 2 is used to perform a convolution operation on the data IF shown in Table 1, and the size of the output data is 2x2, as shown in Table 3. The operation process of the operation device is as follows:

[0071] The first bit of data in the output data is R0=k0×I0+k1×I1+k2×I2+k3×I4+k4×I5+k5×I6+k6×I8+k7×I9+k8×I10; 9 multiplication operations and 8 addition operations are performed.

[0072] The second bit data R1 in the output data = k0×I1+k1×I2+k2×I3+k3×I5+k4×I6+k5×I7+k6×I9+k7×I10+k8×I11; 9 multiplication operations and 8 addition operations are performed.

[0073] The third bit data R2 in the output data is R2=k0×I4+k1×I5+k2×I6+k3×I8+k4×I9+k5×I10+k6×I12+k7×I13+k8×I14; 9 multiplication operations and 8 addition operations are performed.

[0074] The fourth bit data R3 in the output data = k0×I5+k1×I6+k2×I7+k3×I9+k4×I10+k5×I11+k6×I13+k7×I14+k8×I15; 9 multiplication operations and 8 addition operations are performed.

[0075] According to the above operation process, it is known that for each window movement of each convolution kernel, 9 multiplication operations and 8 addition operations are required; and for each convolution kernel, 36 multiplication operations and 32 addition operations are required. Therefore, for 4 convolution kernels, there should be 36*4=144 multiplication operations and 32*4=128 addition operations; when the amount of data to be processed is large and the number of convolution kernels is large, the number of multiplication and addition operations will increase a lot. It can be seen that when the computing device runs the convolutional neural network algorithm for data calculation, the number of multiplication and addition operations is huge; thus, the complexity of the operation is very high, resulting in a long operation time and low operation efficiency of the computing device.

[0076] Moreover, when the computing device runs the convolutional neural network algorithm to complete data operations, the computing device needs to read the feature data under a window size and read a convolution kernel storage unit (Kernel Local Memory, referred to as KLMEM); then, the computing device performs a convolution operation between the feature data under a window size and a convolution kernel storage unit. The computing device then reads the feature data under the next window size and reads the next convolution kernel storage unit, and performs a convolution operation between the feature data read at this time and the convolution kernel storage unit. And so on. Therefore, the computing device needs to read the feature data and the convolution kernel storage unit multiple times, and the number of reads is high; it will cause the internal data transmission bandwidth of the computing device to occupy more, and the demand for internal data transmission bandwidth is high; even when the computing device performs a convolution operation, it will affect other data transmission and data processes.

[0077] In order to solve the problem of high number of data readings in the above-mentioned prior art, the present application provides a convolution-based data processing method and device to reduce the number of data readings and accelerate the operation process.

[0078] Figure 5 A flow chart of a convolution-based data processing method provided in an embodiment of the present application. Figure 5 As shown, the method includes:

[0079] 101. Read the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, and read the feature data corresponding to the convolution dimension under each moving window in the data to be processed; wherein the data to be processed is a data matrix, the data to be processed includes at least one feature data, the convolution array includes at least one convolution kernel storage unit, and each convolution kernel storage unit includes weight parameter units on different convolution dimensions.

[0080] Exemplarily, the execution subject of this embodiment may be a computing device, or a processor, or a controller, or a computing device, or other devices or equipment that can execute this embodiment. This embodiment is described with the execution subject being a computing device.

[0081] The computing device can obtain at least one convolution array and can obtain data to be processed.

[0082] In one example, the computing device may obtain the convolution array and the data to be processed from other devices. Alternatively, in one example, the computing device may pre-store the convolution array and the data to be processed.

[0083] Each convolution array includes at least one convolution kernel storage unit, and each convolution kernel storage unit includes weight parameter units on different convolution dimensions. In one example, the convolution array includes I convolution kernel storage units, where I is a positive integer greater than or equal to 1. Each convolution kernel storage unit includes J convolution kernel storage units, so that the convolution dimension of the convolution kernel storage unit of each convolution kernel storage unit is J, where J is a positive integer greater than or equal to 1.

[0084] For example, each convolution array includes four convolution kernel storage units, namely, convolution kernel storage unit k 0 , k 1 , k 2 , k 3 Each convolution kernel storage unit includes 4 convolution kernel storage units, so the convolution dimension of each convolution kernel storage unit is 4. Convolution kernel storage unit k 0 As shown in Table 4, the convolution kernel storage unit k 1 As shown in Table 5, the convolution kernel storage unit k 2 As shown in Table 6, the convolution kernel storage unit k 3 As shown in Table 7.

[0085] Table 4 Convolution kernel storage unit k 0

[0086] <![CDATA[k 00 ]]> <![CDATA[k 01 ]]> <![CDATA[k 02 ]]> <![CDATA[k 03 ]]> <![CDATA[k 04 ]]> <![CDATA[k 05 ]]> <![CDATA[k 06 ]]> <![CDATA[k 07 ]]> <![CDATA[k 08 ]]>

[0087] Table 5 Convolution kernel storage unit k 1

[0088] <![CDATA[k 10 ]]> <![CDATA[k 11 ]]> <![CDATA[k 12 ]]> <![CDATA[k 13 ]]> <![CDATA[k 14 ]]> <![CDATA[k 15 ]]> <![CDATA[k 16 ]]> <![CDATA[k 17 ]]> <![CDATA[k 18 ]]>

[0089] Table 6 Convolution kernel storage unit k 2

[0090] <![CDATA[k 20 ]]> <![CDATA[k 21 ]]> <![CDATA[k 22 ]]> <![CDATA[k 23 ]]> <![CDATA[k 24 ]]> <![CDATA[k 25 ]]> <![CDATA[k 26 ]]> <![CDATA[k 27 ]]> <![CDATA[k 28 ]]>

[0091] Table 7 Convolution kernel storage unit k 3

[0092] <![CDATA[k 30 ]]> <![CDATA[k 31 ]]> <![CDATA[k 32 <!-- 6 -->]]> <![CDATA[k 33 ]]> <![CDATA[k 34 ]]> <![CDATA[k 35 ]]> <![CDATA[k 36 ]]> <![CDATA[k 37 ]]> <![CDATA[k 38 ]]>

[0093] and, Figure 6 A schematic diagram of a convolution kernel storage unit provided in an embodiment of the present application, such as Figure 6 As shown, each convolution kernel storage unit is deep, that is, each convolution kernel storage unit is hierarchical.

[0094] thereby, Figure 7 A schematic diagram of a convolution array provided in an embodiment of the present application, such as Figure 7 As shown, multiple convolution kernel storage units can form a convolution array.

[0095] Figure 8 A schematic diagram of the data to be processed provided in the embodiment of the present application, such as Figure 8 As shown, the data to be processed is a data matrix, and at least one feature data can constitute the data to be processed.

[0096] For example, Table 8 is the data to be processed. As shown in Table 8, the data to be processed is a 4x4 matrix, and the data to be processed includes 16 feature data. And as shown in 8, the data to be processed has depth, that is, the data to be processed has layers.

[0097] Table 8 Data to be processed

[0098] I0 I1 I2 I3 I4 I5 I6 I7 I8 I9 I10 I11 I12 I13 I14 I15

[0099] When the computing device reads the convolution kernel storage unit, the computing device can read the weight parameter unit belonging to the same convolution dimension in each convolution kernel storage unit in the convolution array.

[0100] In one example, the computing device can read the weight parameter unit belonging to the jth convolution dimension in each convolution kernel storage unit in the convolution array; then, the computing device can read the weight parameter unit belonging to the j+1th convolution dimension in each convolution kernel storage unit in the convolution array; then, the computing device can read the weight parameter unit belonging to the j+2th convolution dimension in each convolution kernel storage unit in the convolution array; and so on. j is an integer greater than or equal to 0 and less than or equal to J; J is the total number of convolution dimensions of the convolution kernel storage unit.

[0101] For example, each convolution array includes four convolution kernel storage units, namely, convolution kernel storage unit k 0 , k 1 , k 2 , k 3Each convolution kernel storage unit includes 4 convolution kernel storage units, so the convolution dimension of each convolution kernel storage unit is 4, as shown in Figure 4-Table 7. Therefore, at one time, the computing device can read the convolution kernel storage unit k 0 The weight parameter unit k on the 0th convolution dimension 00 , convolution kernel storage unit k 1 The weight parameter unit k on the 0th convolution dimension 10 , convolution kernel storage unit k 2 The weight parameter unit k on the 0th convolution dimension 20 , convolution kernel storage unit k 3 The weight parameter unit k on the 0th convolution dimension 30 ; In the next moment, the computing device can read the convolution kernel storage unit k 0 The weight parameter unit k on the first convolution dimension 01 , convolution kernel storage unit k 1 The weight parameter unit k on the first convolution dimension 11 , convolution kernel storage unit k 2 The weight parameter unit k on the first convolution dimension 21 , convolution kernel storage unit k 3 The weight parameter unit k on the first convolution dimension 31 ; In the next moment, the computing device can read the convolution kernel storage unit k 0 The weight parameter unit k on the second convolution dimension 02 , convolution kernel storage unit k 1 The weight parameter unit k on the second convolution dimension 12 , convolution kernel storage unit k 2 The weight parameter unit k on the second convolution dimension 22 , convolution kernel storage unit k 3 The weight parameter unit k on the second convolution dimension 32 ; and so on.

[0102] Since a convolution kernel storage unit can be moved on the data to be processed, each movement constitutes a moving window; when the computing device reads the data to be processed, the computing device can read the feature data corresponding to the convolution dimension under each moving window.

[0103] In one example, when the convolution kernel storage unit moves on the data to be processed, a moving window is formed; the jth convolution dimension corresponds to a feature data, and the computing device reads the feature data. Then, when the convolution kernel storage unit moves on the data to be processed once, another moving window is formed; the jth convolution dimension corresponds to a feature data, and the computing device reads the feature data. Then, when the convolution kernel storage unit moves on the data to be processed once, another moving window is formed; the jth convolution dimension corresponds to a feature data, and the computing device reads the feature data. And so on. Thus, the computing device reads each feature data corresponding to each moving window.

[0104] For example, the data to be processed shown in Table 8 has a size of 4x4; the four convolution kernel storage units in Tables 4 to 7 constitute a convolution array, wherein the size of each convolution kernel storage unit in the convolution array is 3x3. When each convolution kernel storage unit moves on the data to be processed, as shown in Table 9, the convolution kernel storage unit corresponds to feature data I0, feature data I1, feature data I2, feature data I4, feature data I5, feature data I6, feature data I8, feature data I9, and feature data I10 in the data to be processed.

[0105] Table 9 Corresponding feature data under a moving window

[0106]

[0107]

[0108] Thus, the convolution kernel storage unit k i The weight parameter unit k of the 0th convolution dimension in i0 , corresponding to the feature data I0; convolution kernel storage unit k i The weight parameter unit k of the first convolution dimension in i1 , corresponding to the feature data I1; convolution kernel storage unit k I The weight parameter unit k of the second convolution dimension in i2 , corresponding to the feature data I2; convolution kernel storage unit k i The weight parameter unit k of the third convolution dimension in i3 , corresponding to the feature data I4; convolution kernel storage unit k i The weight parameter unit k of the fourth convolution dimension in i4 , corresponding to the feature data I5; convolution kernel storage unit k i The weight parameter unit k of the fifth convolution dimension in i5 , corresponding to the feature data I6; convolution kernel storage unit k iThe weight parameter unit k of the sixth convolution dimension in i6 , corresponding to the feature data I8; convolution kernel storage unit k i The weight parameter unit k of the 7th convolution dimension in i7 , corresponding to the feature data I9; convolution kernel storage unit k i The weight parameter unit k of the 8th convolution dimension in i8 , corresponding to the feature data I10. i is an integer greater than or equal to 0 and less than or equal to I, and I is the total number of convolution kernel storage units in a convolution array. For example, I=9.

[0109] Then, when each convolution kernel storage unit moves once on the data to be processed, as shown in Table 10, the convolution kernel storage unit corresponds to feature data I1, feature data I2, feature data I3, feature data I5, feature data I6, feature data I7, feature data I9, feature data I10 and feature data I11 in the data to be processed.

[0110] Table 10 Corresponding feature data under a moving window

[0111] I1 I2 I3 I5 I6 I7 I9 I10 I11

[0112] Thus, the convolution kernel storage unit k i The weight parameter unit k of the 0th convolution dimension in i0 , corresponding to the feature data I1; convolution kernel storage unit k i The weight parameter unit k of the first convolution dimension in i1 , corresponding to the feature data I2; convolution kernel storage unit k i The weight parameter unit k of the second convolution dimension in i2 , corresponding to the feature data I3; convolution kernel storage unit k i The weight parameter unit k of the third convolution dimension in i3 , corresponding to the feature data I5; convolution kernel storage unit k i The weight parameter unit k of the fourth convolution dimension in i4 , corresponding to the feature data I6; convolution kernel storage unit k i The weight parameter unit k of the fifth convolution dimension in i5 , corresponding to the feature data I7; convolution kernel storage unit k i The weight parameter unit k of the sixth convolution dimension in i6 , corresponding to the feature data I9; convolution kernel storage unit k i The weight parameter unit k of the 7th convolution dimension in i7 , corresponding to the feature data I10; convolution kernel storage unit k iThe weight parameter unit k of the 8th convolution dimension in i8 , which corresponds to the feature data I11. Then, each convolution kernel storage unit moves again on the data to be processed, and so on.

[0113] Furthermore, the computing device can read the feature data I0, I1, I4, and I5 corresponding to the 0th convolution dimension; the computing device can read the feature data I1, I2, I5, and I6 corresponding to the 1st convolution dimension; the computing device can read the feature data I2, I3, I6, and I7 corresponding to the 2nd convolution dimension; the computing device can read the feature data I4, I5, I8, and I9 corresponding to the 3rd convolution dimension; the computing device can read the feature data I4, I5, I8, and I9 corresponding to the 4th convolution dimension. The computing device can read the feature data I5, I6, I9, I10; the computing device can read the feature data I6, I7, I10, I11 corresponding to the 5th convolution dimension; the computing device can read the feature data I8, I9, I12, I13 corresponding to the 6th convolution dimension; the computing device can read the feature data I9, I10, I13, I14 corresponding to the 7th convolution dimension; the computing device can read the feature data I10, I11, I14, I15 corresponding to the 8th convolution dimension.

[0114] 102. Perform convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain output data.

[0115] Exemplarily, the computing device performs convolution operations on the read feature data and the read weight parameter unit to obtain each output data. The computing device may use the convolution process in a traditional convolutional neural network to perform convolution operations on the feature data and the weight parameter unit; or, the computing device may use the convolution process in other traditional convolutional neural networks to perform convolution operations on the feature data and the weight parameter unit; the specific process of the convolution operation is not limited in this embodiment.

[0116] The computing device can read the weight parameter unit belonging to the jth convolution dimension in each convolution kernel storage unit in the convolution array; and the computing device can read the feature data corresponding to the jth convolution dimension under each moving window. The computing device performs convolution operations between the read feature data and the read weight parameter unit respectively to obtain each output data.

[0117] For example, the size of the data to be processed shown in Table 8 is 4x4; the 4 convolution kernel storage units in Tables 4 to 7 constitute a convolution array, where the size of each convolution kernel storage unit in the convolution array is 3x3. The computing device can read the size of each convolution kernel storage unit ki The weight parameter unit k of the 0th convolution dimension in i0 , the computing device can read the feature data I0, I1, I4, I5 corresponding to the 0th convolution dimension; thus, the computing device converts the weight parameter unit k i0 The convolution operation is performed with the feature data I0, I1, I4, and I5. The computing device can read the convolution kernel storage unit k i The weight parameter unit k of the first convolution dimension in i1 , the computing device can read the feature data I1, I2, I5, I6 corresponding to the first convolution dimension; thus, the computing device converts the weight parameter unit k i1 Convolution operation is performed with feature data I1, I2, I5, and I6. The computing device can read each convolution kernel storage unit k i The weight parameter unit k of the second convolution dimension in i2 , the computing device can read the feature data I2, I3, I6, and I7 corresponding to the second convolution dimension; thus, the computing device converts the weight parameter unit k i2 Convolution operation is performed with feature data I2, I3, I6, I7, and so on.

[0118] 103. According to the output data obtained on each feature data, a data combination on each output dimension is obtained to obtain output data, wherein the output data includes the data combination on each output dimension.

[0119] Exemplarily, after the computing device adopts the above-mentioned data reading method and the above-mentioned convolution kernel storage unit reading method, and performs convolution operation on the read data and the convolution kernel storage unit, it needs to output the data obtained by the operation.

[0120] After the convolution operation in step 102, the computing device may re-calculate or re-arrange the obtained output combination data to obtain a data combination on each output dimension; the computing device combines the data on each output dimension to form output data.

[0121] Then, the computing device sends the output data obtained by the convolution operation to other devices; or, the computing device visualizes the output data obtained by the convolution operation and displays the visualized data.

[0122] For example, the data to be processed shown in Table 8 has a size of 4x4; the 4 convolution kernel storage units in Tables 4 to 7 form a convolution array, where the size of each convolution kernel storage unit in the convolution array is 3x3. Using the above reading method, after convolution between the feature data and the convolution kernel storage unit, the output data shown in Table 11 is obtained, and the size of the output data is 2x2; the output dimension of the output data is 4; where O 0 is the data combination of the 0th output dimension of the output data, O 1 is the data combination of the first output dimension of the output data, O 2 is the data combination of the second output dimension of the output data, O 3 It is the data combination of the third output dimension of the output data.

[0123] Table 11 Output data

[0124] <![CDATA[O 0 ]]> <![CDATA[O 1 ]]> <![CDATA[O 2 ]]> <![CDATA[O 3 ]]>

[0125] In this embodiment, the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array is read, and the feature data corresponding to the convolution dimension under each moving window in the data to be processed is read; wherein the data to be processed is a data matrix, the data to be processed includes at least one feature data, the convolution array includes at least one convolution kernel storage unit, and each convolution kernel storage unit includes weight parameter units on different convolution dimensions; convolution operations are performed on the feature data corresponding to the convolution dimension under each moving window and the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain each output data; according to the output data obtained on each feature data, a data combination on each output dimension is obtained to obtain the output data, wherein the output data includes the data combination on each output dimension. When reading the convolution kernel storage unit, the weight parameter unit on the same convolution dimension in each convolution kernel storage unit can be read, thereby reducing the number of times the convolution kernel storage unit is read; and when reading the data to be processed, the feature data corresponding to the convolution dimension under each moving window can be read, thereby eliminating the need to read the same feature data multiple times, thereby reducing the number of times the feature data is read, thereby reducing the number of times the data to be processed is read; then, the read feature data and the convolution kernel storage unit are subjected to convolution operations. Since the number of times the data to be processed is read is reduced, the occupancy of the internal data transmission bandwidth can be reduced, and there is no need to occupy more internal data transmission bandwidth, thereby reducing the impact on other data transmission and data processes. Furthermore, the number of times the convolution kernel storage unit is read can be reduced, further reducing the occupancy of the internal data transmission bandwidth. Thus, hardware acceleration of the computing device is further achieved.

[0126] Fig. 9 A flow chart of another convolution-based data processing method provided in an embodiment of the present application. Fig. 9 As shown, the method includes:

[0127] 201. Obtain the data to be processed, and store the data to be processed in a cache.

[0128] Exemplarily, the execution subject of this embodiment may be a computing device, or a processor, or a controller, or a computing device, or other devices or equipment that can execute this embodiment. This embodiment is described with the execution subject being a computing device.

[0129] The computing device can obtain the data to be processed from other devices. The computing device can store the data to be processed in a cache. In this case, the cache is a cache area in the computing device. Among them, the cache area has the characteristic of large capacity.

[0130] 202. Obtain multiple convolution kernel storage units; construct a convolution array by storing the multiple convolution kernel storage units.

[0131] Exemplarily, the computing device may obtain multiple convolution kernel storage units from other devices. Alternatively, in one example, multiple convolution kernel storage units are pre-stored in the computing device. Then, the computing device combines the multiple convolution kernel storage units into a convolution array.

[0132] For example, four convolution kernel storage units constitute a convolution array.

[0133] There is no restriction on the execution order of step 201 and step 202. Step 201 may be executed first, and then step 202; or step 202 may be executed first, and then step 201; or step 201 and step 202 may be executed simultaneously.

[0134] 203. Read the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, wherein the convolution array includes at least one convolution kernel storage unit, and each convolution kernel storage unit includes weight parameter units on different convolution dimensions.

[0135] In one example, step 203 specifically includes the following process:

[0136] Repeat the following steps until all weight parameter units of the convolution kernel storage units in the convolution array are read, where the initial values ​​of i and j are 0.

[0137] In each clock cycle, the weight parameter unit on the jth convolution dimension of the i-th convolution kernel storage unit in the convolution array is read.

[0138] If i is less than I, it is determined that i is incremented by 1. If i is equal to I, it is determined that i=0, and j is incremented by 1.

[0139] Among them, i is an integer greater than or equal to 0 and less than or equal to I, j is an integer greater than or equal to 0 and less than or equal to J; I is the total number of convolution kernel storage units in a convolution array, J is the total number of convolution dimensions of the convolution kernel storage unit, and I and J are integers greater than or equal to 1.

[0140] Exemplarily, a convolution array includes at least one convolution kernel storage unit, and each convolution kernel storage unit includes weight parameter units on different convolution dimensions.

[0141] Furthermore, when reading the convolution kernel storage unit, the computing device sets the convolution array to include I convolution kernel storage units, I is a positive integer greater than or equal to 1, and sets a value i, i is an integer greater than or equal to 0 and less than or equal to 1. The computing device sets each convolution kernel storage unit to include J convolution kernel storage units, J is a positive integer greater than or equal to 1, and further, the convolution dimension of the convolution kernel storage unit of each convolution kernel storage unit is J; and sets a value j, j is an integer greater than or equal to 0 and less than or equal to J.

[0142] During the convolution operation, the computing device can read the feature data and the convolution kernel storage unit at the same time in each clock cycle. The computing device sets a value t, where t represents the t-th clock cycle and t is an integer greater than or equal to 0.

[0143] For the process of reading the convolution kernel storage unit in each clock cycle, first, the computing device sets the initial values ​​of i and j to 0. Then, the computing device reads the weight parameter unit on the j=0th convolution dimension of the i=0th convolution kernel storage unit in the convolution array in the t=0th clock cycle. Then, the computing device determines that i is less than I, and the computing device continues to read the weight parameter unit on the j=0th convolution dimension of the i=1th convolution kernel storage unit in the convolution array in the t=1th clock cycle. Then, the computing device determines that i is less than I, and the computing device continues to read the weight parameter unit on the j=0th convolution dimension of the i=2th convolution kernel storage unit in the convolution array in the t=2th clock cycle. Then, the computing device determines that i is less than I, and the computing device continues to read the weight parameter unit on the j=0th convolution dimension of the i=3th convolution kernel storage unit in the convolution array in the t=3th clock cycle. And so on.

[0144] If after the t=3th clock cycle, the computing device determines that i is equal to I, then the computing device determines that it has read the weight parameter unit on the j=0th convolution dimension of each convolution kernel storage unit in a convolution array, and then the preset device resets i to 0 and accumulates j by 1, and then the computing device reads the weight parameter unit on the j=1th convolution dimension of each convolution kernel storage unit in the next clock cycle.

[0145] Furthermore, at this time, j=1. The computing device reads the weight parameter unit on the j=1th convolution dimension of the i=0th convolution kernel storage unit in the convolution array in the t=4th clock cycle. Then, the computing device determines that i is less than I, and the computing device continues to read the weight parameter unit on the j=1th convolution dimension of the i=1th convolution kernel storage unit in the convolution array in the t=5th clock cycle. Then, the computing device determines that i is less than I, and the computing device continues to read the weight parameter unit on the j=1th convolution dimension of the i=2th convolution kernel storage unit in the convolution array in the t=6th clock cycle. Then, the computing device determines that i is less than I, and the computing device continues to read the weight parameter unit on the j=1th convolution dimension of the i=3th convolution kernel storage unit in the convolution array in the t=7th clock cycle. And so on.

[0146] If after the t=7th clock cycle, the computing device determines that i is equal to I, then the computing device determines that it has read the weight parameter unit on the j=1th convolution dimension of each convolution kernel storage unit in a convolution array, and then the preset device resets i to 0 and accumulates j by 1, and then the computing device reads the weight parameter unit on the j=2th convolution dimension of each convolution kernel storage unit in the next clock cycle.

[0147] By analogy, the computing device reads the weight parameter unit on the j-th convolution dimension of each convolution kernel storage unit.

[0148] 204. Read feature data corresponding to the convolution dimension under each moving window in the data to be processed, wherein the data to be processed is a data matrix and includes at least one feature data.

[0149] In one example, step 204 specifically includes the following process:

[0150] Repeat the following steps until all feature data in the data to be processed are read, where the initial values ​​of m and j are 0:

[0151] In each clock cycle, the feature data corresponding to the j-th convolution dimension under the m-th moving window is read.

[0152] If m is less than M, then m is determined to be incremented by 1. If m is equal to M, then M=0 is determined, and j is incremented by 1.

[0153] Wherein, m is an integer greater than or equal to 0 and less than or equal to M, M is the total number of moving windows, and M is an integer greater than or equal to 1.

[0154] Exemplarily, since a convolution kernel storage unit can be moved on the data to be processed, each movement constitutes a moving window; when the computing device reads the data to be processed, the computing device can read the feature data corresponding to the convolution dimension under each moving window.

[0155] When reading from the cache buffer, the computing device can determine the total number M of moving windows based on the size of the convolution kernel storage unit and the size of the data to be processed, where M is an integer greater than or equal to 1; and set a value m, which is an integer greater than or equal to 0 and less than or equal to M.

[0156] For example, if the size of the convolution kernel storage unit is 3x3, and the size of the data to be processed is 4x4; according to the movement process of the convolution kernel storage unit in the data to be processed, the total number of moving windows M=4.

[0157] During the convolution operation, the computing device can read the feature data and the convolution kernel storage unit at the same time in each clock cycle. The computing device sets a value t, where t represents the t-th clock cycle and t is an integer greater than or equal to 0.

[0158] For the process of reading the data to be processed in each clock cycle, first, the computing device sets the initial values ​​of m and j to 0. Then, the computing device reads the feature data corresponding to the j=0th convolution dimension under the m=0th moving window in the t=0th clock cycle; further, the computing device can store the "feature data corresponding to the j=0th convolution dimension under the m=0th moving window" read from the cache buffer into the buffer cache area of ​​the computing device, so that when the computing device performs convolution operation later, the data can be directly obtained from the buffer cache area; since the buffer cache area has the characteristics of fast speed and small capacity, the computing device can quickly read the data without reading the data currently required to be operated from the cache cache area, which can speed up the operation. Then, the computing device determines that m is less than M, and the computing device continues to read the feature data corresponding to the j=0th convolution dimension under the m=1th moving window in the t=1th clock cycle; further, the computing device can store the "feature data corresponding to the j=0th convolution dimension under the m=1th moving window" read from the cache buffer into the buffer cache area of ​​the computing device; subsequent reading can refer to the above-mentioned buffer cache area operation and will not be repeated. Then, the computing device determines that m is less than M, and the computing device continues to read the feature data corresponding to the j=0th convolution dimension under the m=2th moving window in the t=2th clock cycle. Then, the computing device determines that m is less than M, and the computing device continues to read the feature data corresponding to the j=0th convolution dimension under the m=3th moving window in the t=3th clock cycle. And so on.

[0159] If after the t=3rd clock cycle, the computing device determines that m is equal to M, then the computing device determines that the feature data corresponding to the j=0th convolution dimension under each moving window has been read. Then, the preset device resets m to 0 and adds 1 to j, and then the computing device reads the feature data corresponding to the j=1th convolution dimension under each moving window in the next clock cycle.

[0160] Furthermore, at this time, j=1. The computing device reads the feature data corresponding to the j=1 convolution dimension under the m=0 moving window in the t=4th clock cycle. Then, the computing device determines that m is less than M, and the computing device continues to read the feature data corresponding to the j=1 convolution dimension under the m=1 moving window in the t=5th clock cycle. Then, the computing device determines that m is less than M, and the computing device continues to read the feature data corresponding to the j=1 convolution dimension under the m=2 moving window in the t=6th clock cycle. Then, the computing device determines that m is less than M, and the computing device continues to read the feature data corresponding to the j=1 convolution dimension under the m=3 moving window in the t=7th clock cycle. And so on.

[0161] If after the t=7th clock cycle, the computing device determines that m is equal to M, then the computing device determines that the feature data corresponding to the j=1th convolution dimension under each moving window has been read. Then, the preset device resets m to 0 and adds 1 to j, and then the computing device reads the feature data corresponding to the j=2th convolution dimension under each moving window in the next clock cycle.

[0162] By analogy, the computing device reads the feature data corresponding to the j-th convolution dimension under each moving window.

[0163] According to the above process, in the above step 203-step 204, the computing device can pass the weight parameter unit read in the tth clock cycle and the characteristic parameter read in the tth clock cycle to the t+1th clock cycle.

[0164] There is no restriction on the execution order of step 203 and step 204. Step 203 may be executed first, and then step 204; or step 204 may be executed first, and then step 203; or step 203 and step 204 may be executed simultaneously.

[0165] 205. Perform convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain output data.

[0166] In an example, step 205 specifically includes: in each clock cycle, performing a convolution operation between the weight parameter unit on the j-th convolution dimension of each convolution kernel storage unit that has been read and the feature data corresponding to the j-th convolution dimension under each moving window that has been read to obtain each output data.

[0167] In one example, performing a convolution operation to obtain various output data includes: running a winograd algorithm to perform a convolution operation to obtain various output data.

[0168] Exemplarily, during the convolution operation, the computing device can simultaneously read feature data and convolution kernel storage units in each clock cycle, and after reading, perform a convolution operation on the weight parameter units on the j-th convolution dimension of each convolution kernel storage unit that has been read and the feature data corresponding to the j-th convolution dimension under each moving window that has been read, so as to obtain each output data.

[0169] The computing device sets a value t, t represents the tth clock cycle, and t is an integer greater than or equal to 0. In the above-mentioned step 203-step 204 process, the computing device can pass the weight parameter unit read in the tth clock cycle and the feature parameter read in the tth clock cycle to the t+1th clock cycle. Thus, the computing device can perform a convolution operation on the tth clock cycle between the weight parameter unit on the jth convolution dimension of each convolution kernel storage unit that has been read and the feature data corresponding to the jth convolution dimension under each moving window that has been read. This greatly reduces the number of times the features to be processed are read, and reduces the number of times the convolution kernel storage unit is read.

[0170] For example, the data to be processed shown in the above Table 8 has a size of 4x4; the 4 convolution kernel storage units in the above Tables 4 to 7 constitute a convolution array, wherein the size of each convolution kernel storage unit in the convolution array is 3x3.

[0171] The computing device reads the weight parameter unit k on the j=0th convolution dimension in the i=0th convolution kernel storage unit at the t=0th clock cycle. 00 , and reads the feature data I0 corresponding to the j=0th convolution dimension under the m=0th moving window. The computing device converts the weight parameter unit k 00 A convolution operation is performed with the feature data I0.

[0172] The computing device reads the weight parameter unit k on the j=0th convolution dimension of the i=1th convolution kernel storage unit in the convolution array at the t=1th clock cycle. 10 , and read the feature data I1 corresponding to the j=0th convolution dimension under the m=1th moving window. Since i is less than I and m is less than M, the computing device can read the weight parameter unit k in the t=0th clock cycle. 00 and feature data I0, directly passed to the t=1th clock cycle. The computing device transfers the weight parameter unit k 00Perform convolution operation with feature data I1, and set weight parameter unit k 10 A convolution operation is performed between the feature data I0. Furthermore, the computing device can perform parallel operations on two multiplication operations at the same time in the t=1th clock cycle.

[0173] The computing device reads the weight parameter unit k on the j=0th convolution dimension of the i=2th convolution kernel storage unit in the convolution array at the t=2th clock cycle. 20 , and reads the feature data I4 corresponding to the j=0th convolution dimension under the m=2th moving window. The computing device can read the weight parameter unit k in the t=0th clock cycle 00 The weight parameter unit k read from the feature data I0 at the t=1th clock cycle 10 and feature data I1 are directly transmitted to the t=2nd clock cycle. The computing device transfers the weight parameter unit k 00 Perform convolution operation with feature data I4, and set weight parameter unit k 10 Perform convolution operation with feature data I1, and set weight parameter unit k 20 A convolution operation is performed between the feature data I0. Furthermore, the computing device can perform parallel operations on three multiplication operations at the same time in the t=2nd clock cycle.

[0174] The computing device reads the weight parameter unit k on the j=0th convolution dimension of the i=3th convolution kernel storage unit in the convolution array at the t=3rd clock cycle. 30 , and reads the feature data I5 corresponding to the j=0th convolution dimension under the m=3th moving window. The computing device can read the weight parameter unit k in the t=0th clock cycle 00 The weight parameter unit k read from the feature data I0 at the t=1th clock cycle 10 The weight parameter unit k read from the feature data I1 at the t=2nd clock cycle 20 and feature data I4 are directly transmitted to the t=3rd clock cycle. The computing device transfers the weight parameter unit k 00 Perform convolution operation with feature data I5, and convert weight parameter unit k 10 Perform convolution operation with feature data I4, and set weight parameter unit k 20 Perform convolution operation with feature data I1, and set weight parameter unit k 30 A convolution operation is performed between the feature data I0. Furthermore, the computing device can perform parallel operations on four multiplication operations at the same time in the t=3rd clock cycle.

[0175] At this point, the weight parameter units on the j=0th convolution dimension of each convolution kernel storage unit have been read.

[0176] Then, the computing device reads the weight parameter unit k on the j=1 convolution dimension in the i=0 convolution kernel storage unit in the t=4th clock cycle. 01 , and reads the feature data I1 corresponding to the j=1 convolution dimension under the m=0th moving window. The computing device can read the weight parameter unit k in the t=1th clock cycle 10 The weight parameter unit k read from the feature data I1 at the t=2nd clock cycle 20 The weight parameter unit k read from the feature data I4 at the t=3rd clock cycle 30 and feature data I5 are directly passed to the t=4th clock cycle, that is, the weight parameter unit and feature data read in the three clock cycles before t=4 are passed to the t=4th clock cycle. 01 Perform convolution operation with feature data I1, and set weight parameter unit k 10 Perform convolution operation with feature data I5, and convert weight parameter unit k 20 Perform convolution operation with feature data I4, and set weight parameter unit k 30 A convolution operation is performed between the feature data I1 and the feature data I2. Furthermore, the computing device can perform parallel operations on four multiplication operations at the same time in the t=4th clock cycle.

[0177] The computing device reads the weight parameter unit k of the j=1 convolution dimension of the i=1 convolution kernel storage unit in the convolution array at the t=5th clock cycle. 11 , and reads the feature data I2 corresponding to the j=1 convolution dimension under the m=1th moving window. The computing device can pass the weight parameter unit and feature data read in the three clock cycles before t=5 to the t=5th clock cycle. The computing device transfers the weight parameter unit k 01 Perform convolution operation with feature data I2, and set weight parameter unit k 11 Perform convolution operation with feature data I1, and set weight parameter unit k 20 Perform convolution operation with feature data I5, and convert weight parameter unit k 30 A convolution operation is performed between the feature data I4. Furthermore, the computing device can perform parallel operations on four multiplication operations at the same time in the t=5th clock cycle.

[0178] The computing device reads the weight parameter unit k of the j=1 convolution dimension of the i=2 convolution kernel storage unit in the convolution array at the t=6th clock cycle. 21, and reads the feature data I5 corresponding to the j=1 convolution dimension under the m=2th moving window. The computing device can pass the weight parameter unit and feature data read in the three clock cycles before t=6 to the t=6th clock cycle. The computing device transfers the weight parameter unit k 01 Perform convolution operation with feature data I5, and convert weight parameter unit k 11 Perform convolution operation with feature data I2, and set weight parameter unit k 21 Perform convolution operation with feature data I1, and set weight parameter unit k 30 A convolution operation is performed between the feature data I5. Furthermore, the computing device can perform parallel operations on four multiplication operations at the same time in the t=6th clock cycle.

[0179] The computing device reads the weight parameter unit k of the j=1 convolution dimension of the i=3 convolution kernel storage unit in the convolution array at the t=7th clock cycle. 31 , and reads the feature data I6 corresponding to the j=1 convolution dimension under the m=3th moving window. The computing device can pass the weight parameter unit and feature data read in the three clock cycles before t=7 to the t=7th clock cycle. The computing device transfers the weight parameter unit k 01 Perform convolution operation with feature data I6, and set weight parameter unit k 11 Perform convolution operation with feature data I5, and convert weight parameter unit k 21 Perform convolution operation with feature data I2, and set weight parameter unit k 31 A convolution operation is performed between the feature data I1 and the feature data I2. Furthermore, the computing device can perform parallel operations on seven multiplication operations at the same time in the t=3rd clock cycle.

[0180] At this point, the weight parameter units on the j=1th convolution dimension of each convolution kernel storage unit have been read.

[0181] Then, the computing device reads the weight parameter units on each convolution dimension of each convolution kernel storage unit in the subsequent clock cycle, and reads the feature data under each moving window accordingly, and performs convolution operation between the weight parameter unit and the feature data. The reading process and the computing process in the above example are shown in Table 12. In addition, the computing device can perform parallel operations on multiple multiplication operations at the same time in each clock cycle.

[0182] Table 12 Weight parameter unit and feature data reading

[0183]

[0184]

[0185] In one example, in order to reduce the computational complexity when the computing device runs a convolutional neural network algorithm, the winograd algorithm is introduced. The winograd algorithm is a deep learning algorithm in the neural network algorithm. In the process of the computing device running the winograd algorithm to complete the data operation, the computing device can train and fix the convolution kernel, thereby reducing the number of multiplication operations. For example, when the size of the convolution kernel storage unit is 3x3, the computing device runs the winograd algorithm to complete the data operation according to the preset configuration; wherein the preset configuration means that the computing device is provided with the above-mentioned cache cache area, the above-mentioned buffer cache area, the above-mentioned operation method, and the like.

[0186] Also, see Figure 6-Figure 8 As shown, each convolution kernel storage unit has depth, that is, each convolution kernel storage unit has levels; the data to be processed has depth, that is, the data to be processed has levels. Therefore, in each clock cycle, when the weight parameter units are operated in parallel, for each weight parameter unit, in the depth dimension of the convolution kernel storage unit, the weight parameter unit is also operated in parallel in the depth dimension.

[0187] In one example, the Winograd algorithm used by the computing device may use a data output formula as follows:

[0188]

[0189] Among them, A, B, and G are constant matrices. In the above data output process, the computing device inputs the data to be processed into the winograd algorithm, and after being processed by the convolution kernel, the output data is R(Oc,In). (GKG T ) characterizes the convolution kernel using GKG T Convolution kernel training. (BIB T ) represents the data to be processed by BIB T Matrix transformation. Bias is a kind of operation mode, which represents the deviation operation.

[0190] For example, the computing device uses a convolution kernel shown in Table 4-7 to perform a convolution operation of the convolutional neural network algorithm on the data to be processed shown in Table 8, and the size of the output data is 2x2, as shown in Table 13. A window shift is required for each convolution kernel, and then for each convolution kernel, 16 multiplications and 32 additions are required. Therefore, for 4 convolution kernels, there should be 16*4=64 multiplication operations and 32*4=128 addition operations.

[0191] It can be seen that when the computing device runs the convolutional neural network algorithm, 36 multiplication operations are required for each convolution kernel; however, when the computing device runs the winograd algorithm, 16 multiplication operations are required for each convolution kernel. Therefore, the computational complexity of the computing device can be reduced by 2.25 times.

[0192] In one example, in the above process, when the computing device runs the Winograd algorithm, the constant matrix A is shown in Table 14, the constant matrix B is shown in Table 15, and the constant matrix G is shown in Table 16. However, the constant matrix A, the constant matrix B, and the constant matrix G are not limited to the methods shown in Tables 14-16.

[0193] Table 13 Output data

[0194] R0 R1 R2 R3

[0195] Table 14 Constant matrix A

[0196] 1 1 1 0 0 1 -1 -1

[0197] Table 15 Constant matrix G

[0198] 1 0 1 0 0 1 1 0 0 -1 1 0 0 1 0 -1

[0199] Table 16 Constant matrix G

[0200] 1 0 0 0.5 0.5 0.5 0.5 -0.5 0.5 0 0 1

[0201] In one example, when the computing device runs the winograd algorithm to complete the data operation, the computing device trains the convolution kernel to fix the convolution kernel, and the computing device can store the fixed convolution kernel in a multiply-accumulate (MAC) array. The computing device can read the data to be processed, broadcast the data to the MAC array, and use an adder tree to add the product of the data and the convolution kernel to obtain output data.

[0202] In this embodiment, in addition to the method for reading the data to be processed and the method for reading the convolution kernel storage unit provided in this embodiment, the computing device can also use the winograd algorithm to complete the convolution operation.

[0203] Based on the above reading method, when the computing device performs convolution operation between the feature data and the weight parameter unit of the convolution kernel storage unit, Fig.10 Schematic diagram of the convolution operation process provided in the embodiment of the present application Figure 1 ,like Fig.10As shown, the computing device reads the characteristic data that can be read in the tth clock cycle from the cache buffer area, and caches the characteristic data in the buffer buffer area; the computing device reads the characteristic data in the buffer buffer area, and performs BIB of the characteristic data on the characteristic data T Operation, where Fig.10 The IFMAP in is the feature data read from the buffer cache.

[0204] For example, the to-be-processed processing shown in Table 8 can be expressed in the form of the following matrix I. Each element in I is a feature data.

[0205]

[0206] Then, the computing device can use the following formula to complete BIB T Operation.

[0207]

[0208] and, Fig.10 The KLMEM in the figure is a convolution kernel storage unit. The computing device can combine multiple convolution kernel storage units into a convolution array (KArray). Fig.10 Four convolution arrays are shown, namely KArray0, KArray1, KArray2, and KArray3.

[0209] Furthermore, after reading multiple convolution kernel storage units, the computing device performs GKG on the convolution kernel storage units. T Operation. Then, the convolution kernel storage unit is fixed, which can reduce the number of multiplication operations.

[0210] For example, for Figure 4-Figure 7 The convolution kernel storage unit shown in FIG. 1 can be represented in the form of the following matrix K for each convolution kernel storage unit. Each element in K is a weight parameter unit.

[0211]

[0212] Then, the computing device performs GKG on the convolution kernel storage unit K T Operation is performed to obtain a fixed convolution kernel storage unit Z. Among them, Z can be represented in the form of the following matrix Z. Each element in Z is a fixed weight parameter unit.

[0213]

[0214] Among them, each element in Z is as follows.

[0215] Z00 =p 00

[0216]

[0217]

[0218] Z 03 =p 02

[0219]

[0220]

[0221]

[0222]

[0223]

[0224]

[0225]

[0226]

[0227] Z 30 =p 20

[0228]

[0229]

[0230] Z 33 =p 22

[0231] Then, the computing device reads the fixed convolution kernel storage unit in the convolution array and uses the winograd algorithm to compare the fixed convolution kernel storage unit with the convolution kernel storage unit after BIB. T The characteristic data of the operation is convolved to complete the multiplication and accumulation operation (Multiply Accumulate, referred to as MAC). Fig.10 The MAC ENGINE in represents the accumulation operation.

[0232] Furthermore, this embodiment can also use the winograd algorithm to compare the fixed convolution kernel storage unit with the BIB T The characteristic data of the operation can be convolved to reduce the number of multiplication operations in the convolution operation process, thereby reducing the complexity of the operation and improving the operation efficiency.

[0233] For example, Fig.11 Schematic diagram of the convolution operation process provided in the embodiment of the present application Figure 2 ,like Fig.11 As shown, for a convolution array, the computing device adopts the above-mentioned method for reading the to-be-processed data and the above-mentioned method for reading the convolution kernel storage unit provided in this embodiment, and can broadcast the weight parameter unit of each convolution storage unit in a convolution array to Fig.11 Each column (Col) of ; and the feature unit corresponding to the jth weight parameter unit under each moving window is broadcast to Fig.11 Each column (Col). Among them, Fig.11 The OPA shown in refers to the weight parameter unit of each convolution storage unit in a convolution array. Fig.11 The OPB shown in refers to the feature unit corresponding to the jth weight parameter unit under each moving window. Fig.11 The mul shown in is a multiplication operation, or mul is an operation of the winograd algorithm. Then, after the convolution operation or the winograd algorithm, the output data is obtained. For example, the Winograd fast convolution operation can be implemented for the 3x3 convolution kernel storage unit shown in Table 12.

[0234] When it is necessary to perform convolution operations between the data to be processed and multiple convolution arrays, the processing method of each convolution array can adopt the solution provided in this embodiment.

[0235] 206. According to the output data obtained on each feature data, obtain a data combination on each output dimension to obtain output data, wherein the output data includes a data combination on each output dimension.

[0236] For example, this step can be referred to Figure 5 Step 103 shown is not repeated any more.

[0237] In this embodiment, on the basis of the above embodiment, the computing device can also transfer the feature parameters and weight parameter units read in multiple clock cycles to the next clock cycle, and then the computing device will perform convolution operation between the weight parameter units on the jth convolution dimension of each convolution kernel storage unit that has been read and the feature data corresponding to the jth convolution dimension under each moving window that has been read; thereby greatly reducing the number of times the features to be processed are read, and reducing the number of times the convolution kernel storage unit is read. It is achieved that multiple convolution kernel storage units share the data to be processed, improves the reuse rate of feature data in the data to be processed, and reduces the number of times the feature data is read; and the data to be processed can also share multiple convolution kernel storage units, improves the reuse rate of convolution kernel storage units, and reduces the number of times the convolution kernel storage units are read. Moreover, the computing device can perform parallel operations on multiple multiplication operations in each clock cycle. Furthermore, this embodiment can also use the winograd algorithm to fix the convolution kernel storage unit, and perform BIB on the feature data. T Operation, and then the fixed convolution kernel storage unit and the BIB T The characteristic data of the operation is used to perform convolution operation; thereby, complex control logic is avoided and the number of multiplication operations in the convolution operation process can be reduced; thereby reducing the complexity of the operation and improving the operation efficiency.

[0238] Fig.12 A schematic diagram of a computing device provided in an embodiment of the present application. Fig.12 As shown, the computing device includes: a processor 121;

[0239] The processor 121 is configured to:

[0240] Read the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, and read the feature data corresponding to the convolution dimension under each moving window in the data to be processed; wherein the data to be processed is a data matrix, the data to be processed includes at least one feature data, the convolution array includes at least one convolution kernel storage unit, and each convolution kernel storage unit includes weight parameter units on different convolution dimensions.

[0241] Convolution operations are performed on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain each output data.

[0242] According to the output data obtained on each feature data, a data combination on each output dimension is obtained to obtain output data, wherein the output data includes the data combination on each output dimension.

[0243] In one example, when the processor 121 reads the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, it is specifically used to:

[0244] Repeat the following steps until the weight parameter units of each convolution kernel storage unit in the convolution array are read, where the initial values ​​of i and j are 0: in each clock cycle, read the weight parameter unit on the jth convolution dimension of the i-th convolution kernel storage unit in the convolution array; if i is less than I, determine that i is accumulated by 1; if i is equal to I, determine that i=0, and j is accumulated by 1; where i is an integer greater than or equal to 0 and less than or equal to I, and j is an integer greater than or equal to 0 and less than or equal to J; I is the total number of convolution kernel storage units in a convolution array, J is the total number of convolution dimensions of the convolution kernel storage unit, and I and J are integers greater than or equal to 1.

[0245] In one example, when the processor 121 reads the feature data corresponding to the convolution dimension under each moving window in the data to be processed, it is specifically used to: repeat the following steps until the feature data in the data to be processed is read, wherein the initial values ​​of m and j are 0: in each clock cycle, read the feature data corresponding to the jth convolution dimension under the mth moving window; if m is less than M, determine that m is accumulated by 1; if m is equal to M, determine that M=0, and j is accumulated by 1; wherein m is an integer greater than or equal to 0 and less than or equal to M, M is the total number of moving windows, and M is an integer greater than or equal to 1.

[0246] In one example, when the processor 121 performs convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain each output data, it is specifically used to: in each clock cycle, perform a convolution operation on the weight parameter unit on the j-th convolution dimension of each convolution kernel storage unit that has been read and the feature data corresponding to the j-th convolution dimension under each moving window that has been read to obtain each output data.

[0247] In one example, when the processor 121 performs a convolution operation to obtain various output data, it is specifically used to: run a winograd algorithm to perform a convolution operation to obtain various output data.

[0248] In one example, before reading the feature data corresponding to the convolution dimension under each moving window in the data to be processed, the processor 121 is further used to: obtain the data to be processed and store the data to be processed in a cache.

[0249] In one example, before reading the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, the processor 121 is further used to: obtain multiple convolution kernel storage units;

[0250] Multiple convolution kernel storage units are used to construct a convolution array.

[0251] In one example, the computing device further includes a memory 122; and may also include a communication interface 123 and a bus 124. The processor 121, the memory 122, and the communication interface 123 may communicate with each other through the bus 124. The communication interface 123 may be used for information transmission. The processor 121 may call the logic instructions in the memory 124 to execute the method of the above embodiment.

[0252] In addition, the logic instructions in the memory 122 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. The memory 122 can include a cache area and a buffer area.

[0253] The memory 122 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present application. The processor 121 executes functional applications and data processing by running the software programs, instructions, and modules stored in the memory 122, that is, implementing the methods in the above method embodiments.

[0254] The memory 122 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 122 may include a high-speed random access memory and may also include a non-volatile memory.

[0255] The principle and technical effects of the computing device of this embodiment are shown in the above method embodiment and will not be described in detail.

[0256] Fig.13 A schematic diagram of another computing device provided in an embodiment of the present application. Fig.13 As shown, the computing device includes:

[0257] The first reading unit 131 is used to read the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array.

[0258] The second reading unit 132 is used to read the feature data corresponding to the convolution dimension under each moving window in the data to be processed; wherein the data to be processed is a data matrix, the data to be processed includes at least one feature data, the convolution array includes at least one convolution kernel storage unit, and each convolution kernel storage unit includes weight parameter units on different convolution dimensions.

[0259] The operation unit 133 is used to perform convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain each output data.

[0260] The output unit 134 is used to obtain data combinations on each output dimension according to the output data obtained on each feature data, so as to obtain output data, wherein the output data includes the data combination on each output dimension.

[0261] In one example, the first reading unit 131 is specifically configured to:

[0262] Repeat the following steps until the weight parameter units of each convolution kernel storage unit in the convolution array are read, where the initial values ​​of i and j are 0: in each clock cycle, read the weight parameter unit on the jth convolution dimension of the i-th convolution kernel storage unit in the convolution array; if i is less than I, determine that i is accumulated by 1; if i is equal to I, determine that i=0, and j is accumulated by 1; where i is an integer greater than or equal to 0 and less than or equal to I, and j is an integer greater than or equal to 0 and less than or equal to J; I is the total number of convolution kernel storage units in a convolution array, J is the total number of convolution dimensions of the convolution kernel storage unit, and I and J are integers greater than or equal to 1.

[0263] In one example, the second reading unit 132 is specifically used to: repeatedly perform the following steps until the feature data in the data to be processed is read, wherein the initial values ​​of m and j are 0: in each clock cycle, read the feature data corresponding to the jth convolution dimension under the mth moving window; if m is less than M, determine that m is accumulated by 1; if m is equal to M, determine that M=0, and j is accumulated by 1; wherein m is an integer greater than or equal to 0 and less than or equal to M, M is the total number of moving windows, and M is an integer greater than or equal to 1.

[0264] In one example, the computing unit 133 is specifically configured to:

[0265] In each clock cycle, a convolution operation is performed between the weight parameter unit on the j-th convolution dimension of each convolution kernel storage unit that has been read and the feature data corresponding to the j-th convolution dimension under each moving window that has been read to obtain each output data.

[0266] In one example, the operation unit 133 is specifically used to: run the Winograd algorithm to perform convolution operations to obtain various output data.

[0267] In one example, the computing device provided by this embodiment further includes:

[0268] The first acquisition unit is used to acquire the data to be processed and store the data to be processed in the cache before the second reading unit 132 reads the feature data corresponding to the convolution dimension under each moving window in the data to be processed.

[0269] In one example, the computing device provided by this embodiment further includes:

[0270] The second acquisition unit is used to acquire multiple convolution kernel storage units before the first reading unit 131 reads the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array; and construct a convolution array with multiple convolution kernel storage units.

[0271] The principle and technical effects of the computing device of this embodiment are shown in the above method embodiment and will not be described in detail.

[0272] Each of the above units may be a hardware circuit including a digital circuit, an analog circuit, etc. The physical implementation of the hardware circuit includes but is not limited to physical devices, and the physical devices include but are not limited to transistors, memristors, etc. The computing unit in the above computing device may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. The above storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as RRAM, DRAM, SRAM, EDRAM, HBM, HMC, etc.

[0273] In some embodiments, Fig.14 A schematic diagram of a processing device provided in an embodiment of the present application, such as Fig.14 As shown, the processing device includes one or more of the above-mentioned processors, which are used to obtain data or receive thread tasks from other processing devices, and perform specified data operations, and transmit the execution results to other processing devices through I / O interfaces.

[0274] When the processing device includes multiple processors, the multiple processors can be connected and transmit data through a specific structure.

[0275] Among them, multiple processors are interconnected and transmit data through a fast peripheral device interconnection bus PCIE bus. Multiple processors share the same control system or have their own control systems. Multiple processors share memory units or have their own memory units. The interconnection mode of multiple processors is any interconnection topology. The processing device has high compatibility and can be connected to various types of servers through a PCIE interface.

[0276] In some embodiments, Fig.15 A schematic diagram of a combined processing device provided in an embodiment of the present application is shown in FIG. Fig.15 As shown, the combined processing device includes the processing device as described above, a universal interconnection interface and other processing devices.

[0277] The processing device interacts with other processing devices to jointly complete the operations specified by the user.

[0278] The above-mentioned combined device can be used as a system for smart devices such as mobile phones, robots, and drones, effectively reducing the core area of ​​the control part, improving the processing speed, and reducing the overall power consumption.

[0279] In some embodiments, Fig.16 A schematic diagram of a chip provided in an embodiment of the present application Figure 1 ,like Fig.16 As shown, the chip includes the processing device as described above.

[0280] In one example, a chip includes at least one of the above-mentioned processing devices.

[0281] In some embodiments, Fig.17 A schematic diagram of a chip provided in an embodiment of the present application Figure 2 ,like Fig.17 As shown, the chip includes the combined processing device as described above.

[0282] In one example, a chip includes at least one of the above-mentioned combined processing devices.

[0283] In some embodiments, Fig.18 A schematic diagram of a chip provided in an embodiment of the present application Figure 3 ,like Fig.18 As shown, the chip includes the processing device as described above and the combined processing device as described above.

[0284] In one example, the chip includes at least one processing device and at least one combined processing device. The processing device and the combined processing device may or may not exchange data.

[0285] In some embodiments, Fig.19 A schematic diagram of a board provided in an embodiment of the present application is shown as follows: Fig.19As shown, the board includes the chip as described above.

[0286] In one example, at least one chip is disposed in the board.

[0287] In some embodiments, Fig. 20 A schematic diagram of an electronic device provided in an embodiment of the present application Figure 1 , Fig.21 A schematic diagram of an electronic device provided in an embodiment of the present application Figure 2 , Fig. 22 A schematic diagram of an electronic device provided in an embodiment of the present application Figure 3 ,like Figure 20-22 As shown, the present application provides an electronic device, the electronic device includes the chip as described above or the board as described above, or the electronic device includes the chip as described above and the board as described above.

[0288] Among them, the electronic device includes: a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a cloud server, a camera, a camcorder, a projector, a watch, a headset, a mobile storage, a wearable device, a vehicle, a household appliance, and / or a medical device.

[0289] The above-mentioned means of transportation include airplanes, ships and / or vehicles; the above-mentioned household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; medical equipment includes magnetic resonance imaging, ultrasound machines and / or electrocardiographs.

[0290] Reference to "embodiment" in this application indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0291] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method based on convolution, characterized in that: include: Reading the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, and reading the feature data corresponding to the convolution dimension under each moving window in the data to be processed; wherein the data to be processed is a data matrix, the data to be processed includes at least one feature data, the convolution array includes at least one convolution kernel storage unit, and each of the convolution kernel storage units includes weight parameter units on different convolution dimensions; Perform convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain output data; According to the output data obtained on each of the feature data, a data combination on each output dimension is obtained to obtain output data, wherein the output data includes a data combination on each of the output dimensions.

2. The method according to claim 1, characterized in that Reading the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array includes: Repeat the following steps until all weight parameter units of the convolution kernel storage units in the convolution array are read, where the initial values ​​of i and j are 0: In each clock cycle, read the weight parameter unit on the jth convolution dimension of the i-th convolution kernel storage unit in the convolution array; If i is less than I, then i is determined to be accumulated by 1; If i is equal to I, then i=0 is determined, and j is accumulated by 1; Among them, i is an integer greater than or equal to 0 and less than or equal to I, j is an integer greater than or equal to 0 and less than or equal to J; I is the total number of convolution kernel storage units in a convolution array, J is the total number of convolution dimensions of the convolution kernel storage unit, and I and J are integers greater than or equal to 1.

3. The method according to claim 2, characterized in that The reading of feature data corresponding to the convolution dimension under each moving window in the data to be processed includes: Repeat the following steps until all feature data in the data to be processed are read, wherein the initial values ​​of m and j are 0: In each of the clock cycles, feature data corresponding to the j-th convolution dimension under the m-th moving window is read; If m is less than M, then m is incremented by 1; If m is equal to M, then M=0 is determined, and j is incremented by 1; Wherein, m is an integer greater than or equal to 0 and less than or equal to M, M is the total number of moving windows, and M is an integer greater than or equal to 1.

4. The method according to claim 3, characterized in that Convolution operations are performed on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain output data, including: In each of the clock cycles, a convolution operation is performed between the weight parameter units on the j-th convolution dimension of each convolution kernel storage unit that has been read and the feature data corresponding to the j-th convolution dimension under each moving window that has been read to obtain each output data.

5. The method according to claim 4, characterized in that The convolution operation is performed to obtain various output data, including: Run the winograd algorithm to perform convolution operations and obtain the output data.

6. The method according to any one of claims 1 to 5, characterized in that: Before reading the feature data corresponding to the convolution dimension under each moving window in the data to be processed, the method further includes: The data to be processed is obtained, and the data to be processed is stored in a cache.

7. The method according to any one of claims 1 to 5, characterized in that: Before reading the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array, the method further includes: Obtain multiple convolution kernel storage units; The multiple convolution kernel storage units are used to construct the convolution array.

8. A computing device, characterized in that: include: processor; The processor is configured to: Reading the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, and reading the feature data corresponding to the convolution dimension under each moving window in the data to be processed; wherein the data to be processed is a data matrix, the data to be processed includes at least one feature data, the convolution array includes at least one convolution kernel storage unit, and each of the convolution kernel storage units includes weight parameter units on different convolution dimensions; Perform convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain output data; According to the output data obtained on each of the feature data, a data combination on each output dimension is obtained to obtain output data, wherein the output data includes a data combination on each of the output dimensions.

9. The computing device according to claim 8, characterized in that: When the processor reads the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array, it is specifically used to: Repeat the following steps until all weight parameter units of the convolution kernel storage units in the convolution array are read, where the initial values ​​of i and j are 0: In each clock cycle, read the weight parameter unit on the jth convolution dimension of the i-th convolution kernel storage unit in the convolution array; If i is less than I, then i is determined to be accumulated by 1; If i is equal to I, then i=0 is determined, and j is accumulated by 1; Among them, i is an integer greater than or equal to 0 and less than or equal to I, j is an integer greater than or equal to 0 and less than or equal to J; I is the total number of convolution kernel storage units in a convolution array, J is the total number of convolution dimensions of the convolution kernel storage unit, and I and J are integers greater than or equal to 1.

10. The computing device according to claim 9, characterized in that: When the processor reads the feature data corresponding to the convolution dimension under each moving window in the data to be processed, it is specifically used to: Repeat the following steps until all feature data in the data to be processed are read, wherein the initial values ​​of m and j are 0: In each of the clock cycles, feature data corresponding to the j-th convolution dimension under the m-th moving window is read; If m is less than M, then m is incremented by 1; If m is equal to M, then M=0 is determined, and j is incremented by 1; Wherein, m is an integer greater than or equal to 0 and less than or equal to M, M is the total number of moving windows, and M is an integer greater than or equal to 1.

11. The computing device according to claim 10, characterized in that: The processor performs convolution operations on the feature data corresponding to the convolution dimension under each moving window and the weight parameter units on the same convolution dimension in each convolution kernel storage unit in the convolution array to obtain each output data, specifically for: In each of the clock cycles, a convolution operation is performed between the weight parameter units on the j-th convolution dimension of each convolution kernel storage unit that has been read and the feature data corresponding to the j-th convolution dimension under each moving window that has been read to obtain each output data.

12. The computing device according to claim 11, characterized in that: When performing the convolution operation to obtain the output data, the processor is specifically used to: Run the winograd algorithm to perform convolution operations and obtain the output data.

13. The computing device according to any one of claims 8 to 12, characterized in that: Before reading the feature data corresponding to the convolution dimension under each moving window in the data to be processed, the processor is further configured to: The data to be processed is obtained, and the data to be processed is stored in a cache.

14. The computing device according to any one of claims 8 to 12, characterized in that: The processor is further configured to, before reading the weight parameter unit on the same convolution dimension in each convolution kernel storage unit in the convolution array: Obtain multiple convolution kernel storage units; The multiple convolution kernel storage units are used to construct the convolution array.

15. A processing device, characterized in that: The processing device comprises one or more computing devices as claimed in any one of claims 8 to 14, for performing specified data operations and transmitting the execution results to other processing devices through an I / O interface; When the processing device includes a plurality of computing devices, the plurality of computing devices may be connected and transmit data through a specific structure; Among them, multiple computing devices are interconnected and transmit data through a fast peripheral device interconnect bus PCIE bus; multiple computing devices share the same control system or have their own control systems; multiple computing devices share a memory unit or have their own memory units.

16. A combined processing device, characterized in that: comprising the processing device as claimed in claim 15, a universal interconnection interface and other processing devices; The processing device interacts with the other processing devices to jointly complete the operation specified by the user.

17. A chip, characterized in that: The chip comprises the processing device according to claim 15 or the combined processing device according to claim 16 .

18. A board, characterized in that: The board includes the chip as claimed in claim 17.

19. An electronic device, characterized in that: The electronic device comprises the chip as claimed in claim 17 or the board as claimed in claim 18.

Images