A data processing method, apparatus, and storage medium

Abstract

This application provides a data processing method, apparatus, and storage medium. The apparatus deploys a computing unit array, where each computing unit consists of a set of multipliers and an adder register connected to the set of multipliers. Each multiplier performs a multiplication operation on a convolution kernel and an input feature data, and the adder register accumulates and stores the data output by the set of multipliers. The input feature data is propagated through data pulsation in each set of computing units along the horizontal direction of the computing unit array. Convolution kernel data is loaded into each computing unit of the computing unit array. At least one adder register in at least one computing unit along the vertical direction of the computing unit array performs accumulation and output operations.

Description

Technical Field

[0001] This application relates to the field of deep learning, and more particularly to a data processing method, apparatus, and storage medium. Background Technology

[0002] Currently, dedicated accelerators for the field of artificial intelligence (AI) are designed with a careful balance between energy efficiency and flexibility in their overall architecture. For convolution operations, which constitute the largest proportion of AI algorithms, dedicated convolution acceleration units are used, while other smaller operations are handled by highly flexible digital signal processing (DSP) units. For convolution acceleration units, the architecture is typically designed based on the parameters of the convolutional layers, and then the neural network is run on that architecture.

[0003] However, in a neural network, different convolutional layers have different parameters. When performing convolution calculations on different convolutional layers using a fixed hardware architecture, the computing units cannot be fully utilized, resulting in low resource utilization. Summary of the Invention

[0004] This application provides a data processing method, apparatus, and storage medium that can fully utilize the computing units in the hardware architecture and improve resource utilization.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, embodiments of this application propose a data processing apparatus in which a computing unit array is deployed. Each computing unit in the computing unit array consists of a set of multipliers and an adder register connected to the set of multipliers. A multiplier is used to perform a multiplication operation on a convolution kernel data and an input feature data, and an adder register is used to perform an accumulation and storage operation on a set of data output by the set of multipliers.

[0007] Each group of computing units in the horizontal direction of the computing unit array propagates input feature data through data pulsation; each computing unit in the computing unit array loads convolutional kernel data.

[0008] The computing unit array performs an accumulation and output operation on at least one addition register of at least one computing unit along the vertical direction.

[0009] Secondly, embodiments of this application propose a data processing method applied to the aforementioned data processing apparatus, the method comprising:

[0010] Obtain the algorithm layer parameters and data scheduling method of the first algorithm layer in the neural network model;

[0011] Based on the algorithm layer parameters and the data scheduling method, determine the loading method of input feature data and convolution kernel data on the computing unit array and the corresponding result output method of the computing unit array.

[0012] According to the loading method of input feature data and convolution kernel data on the computing unit array, the input feature data and convolution kernel are loaded into the computing unit array, and multiply-accumulate operation is performed on the computing unit array to obtain multiply-accumulate data;

[0013] The multiply-accumulate data is processed based on the aforementioned data output method to obtain output feature data.

[0014] Thirdly, embodiments of this application provide a data processing apparatus, the apparatus comprising:

[0015] The acquisition unit is used to acquire the algorithm layer parameters and data scheduling method of the first algorithm layer in the neural network model.

[0016] The determining unit is used to determine the loading method of input feature data and convolution kernel data on the computing unit array and the result output method corresponding to the computing unit array based on the algorithm layer parameters and the data scheduling method.

[0017] The loading unit is used to load the input feature data and convolution kernel into the computing unit array according to the loading method of the input feature data and convolution kernel data on the computing unit array, and perform multiply-accumulate operations on the computing unit array to obtain multiply-accumulate data;

[0018] The processing unit is used to process the multiply-accumulate data based on the data output method to obtain output feature data.

[0019] Fourthly, embodiments of this application propose a data processing apparatus, the apparatus comprising: a processor, a memory, and a communication bus; the processor executes a running program stored in the memory to implement the data processing method described above.

[0020] Fifthly, embodiments of this application propose a storage medium storing a computer program thereon, which, when executed by a processor, implements the data processing method described above.

[0021] This application provides a data processing method, apparatus, and storage medium. The apparatus deploys a computing unit array, where each computing unit consists of a set of multipliers and an adder register connected to the multipliers. Each multiplier performs a multiplication operation on a convolution kernel and an input feature, while the adder register accumulates and stores the data output by the multipliers. Each set of computing units in the horizontal direction propagates the input feature data via data pulsation. Each computing unit in the array loads convolution kernel data. At least one adder register in at least one computing unit in the vertical direction performs accumulation and output operations. By employing this implementation, each computing unit can perform multiplication, accumulation, and storage operations, allowing the data processing apparatus to flexibly configure different data flow directions according to different algorithm layer parameters, ensuring full utilization of the computing units and improving resource utilization. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0023] Figure 2 A schematic diagram of an exemplary computing unit provided in an embodiment of this application;

[0024] Figure 3 This application provides an exemplary illustration of data flow in a computing unit array. Figure 1 ;

[0025] Figure 4 This application provides an exemplary illustration of data flow in a computing unit array. Figure 2 ;

[0026] Figure 5 A schematic diagram of a multiplexer provided in an embodiment of this application;

[0027] Figure 6 A schematic diagram of an accumulator output provided in an embodiment of this application;

[0028] Figure 7 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0029] Figure 8 A schematic diagram of an exemplary output feature data reuse process provided in this application embodiment. Figure 1 ;

[0030] Figure 9 A schematic diagram of an exemplary output feature data reuse process provided in this application embodiment. Figure 2;

[0031] Figure 10 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 1 ;

[0032] Figure 11 A schematic diagram of the structure of a data processing device provided in this application embodiment. Figure 2 . Detailed Implementation

[0033] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.

[0034] This application provides a data processing device 1, such as... Figure 1 As shown, the data processing device 1 deploys an array of computing units 10. Each computing unit 10 in the array of computing units 10 consists of a set of multipliers 100 and an adder register 101 connected to the set of multipliers 100. A multiplier 100 is used to perform a multiplication operation on a convolution kernel data and an input feature data, and an adder register 101 is used to accumulate and store a set of data output by the set of multipliers 100.

[0035] The array of computing units 10 propagates input feature data through data pulsation in each group of computing units along the horizontal direction; convolutional kernel data is loaded in each computing unit of the array of computing units.

[0036] The calculation unit 10 array performs an accumulation and output operation on at least one addition register of at least one calculation unit along the vertical direction.

[0037] An embodiment of this application proposes a data processing device with a computing unit array deployed thereon, on which a neural network model can run.

[0038] In this embodiment of the application, the neural network model can be a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), etc. The specific model can be selected according to the actual situation, and this embodiment of the application does not impose any specific limitations.

[0039] In this embodiment, the hardware architecture of the data processing device is in the form of a computing unit array. For example, the computing unit array consists of 14 computing units in the horizontal direction and 4 computing units in the vertical direction. It should be noted that the above is only an optional embodiment. Specifically, the number of computing units in the horizontal direction and the number of computing units in the vertical direction can be determined according to the total number of computing units and the shape of the hardware architecture. This embodiment does not impose any specific limitations.

[0040] It should be noted that the computing unit in the computing unit array proposed in this application embodiment includes two parts: the first part is a set of multipliers, and the second part is an adder register. The number of multipliers in the set can be two, three, etc., and can be selected according to the actual situation. This application embodiment does not make a specific limitation. According to the number of multipliers, the corresponding number of input feature data and convolution kernel data are loaded into the set of multipliers for multiplication operation. At this time, each multiplier performs a multiplication operation on one input feature data and one convolution kernel data. After that, the set of data calculated by the set of multipliers is input into the adder register. The adder register accumulates the set of data and stores the accumulation result.

[0041] For example, such as Figure 2 As shown, the computational unit array is a 16×4 array, where each computational unit consists of two multipliers and one adder register. Each multiplier receives a convolution kernel data W and an input feature data F as inputs. The two multipliers input their calculation results into an adder register, and the results are added together to obtain the multiplicative accumulated data. Figure 2 In the 16×4 array, circles represent adders, and ellipses connected to circles represent multipliers.

[0042] It is understandable that, since each computational unit in the computational unit matrix can perform multiplication and accumulation output operations, the computational unit matrix can implement different data scheduling methods and different configurations of data flow according to different algorithm parameters.

[0043] In this embodiment, the input feature data is sequentially fed into the computing unit array in the horizontal direction through data pulsation. The data pulsation method is determined by the number of convolution kernels in the first algorithm layer and the size information of the input feature data.

[0044] For example, such as Figure 3 As shown, there are 8 convolutional kernels (K0-K7), and the size of the input feature data is relatively small. Therefore, the 16×4 computation array is divided into 8 groups, each group containing two columns of computational units. These two columns of computational units share one convolutional kernel data. For the two columns of computational units in group K0, their input feature data corresponds to two windows of the input feature data, as shown below. Figure 3In the first and second windows of the K0 group, one cycle can generate the multiplication and addition result of the first column and the first window of K0, and the multiplication and addition result of the second column and the second window of K0, respectively. Similarly, for the two-column calculation unit of the K1 group, its input feature data corresponds to the two windows of input feature data, such as... Figure 3 In the first and second windows of the array, for group K1, one cycle can generate the multiplication and addition result of the first column of K1 and the data of the first window, and the multiplication and addition result of the second column of K1 and the data of the second window, and so on. Therefore, the data in the first window pulsates in the 16×4 computation array in the following order from right to left: first column, third column, fifth column, seventh column, ninth column, eleventh column, thirteenth column, and fifteenth column. The data in the second window pulsates in the following order from right to left: second column, fourth column, sixth column, eighth column, tenth column, twelfth column, fourteenth column, and sixteenth column.

[0045] For example, such as Figure 4 As shown, to reduce the number of convolutional kernels and increase the size of the input feature image, the grouping of computational units can be changed. For example, for the four convolutional kernels K0-K3, the 16×4 computational array is divided into four groups, each group corresponding to one convolutional kernel data. Each group includes four columns of computational units, and these four columns of computational units share one convolutional kernel data. For the four columns of computational units in group K0, their input feature data corresponds to four windows of the feature map in the X direction, such as... Figure 4If we consider the first, second, third, and fourth windows in the graph, then for group K0, one cycle can generate the multiplication and addition results of the first column of K0 with the data in the first window, the second column of K0 with the data in the second window, the third column of K0 with the data in the third window, and the fourth column of K0 with the data in the fourth window, respectively. Similarly, for the four-column calculation unit of group K1, its input feature data corresponds to the four windows of the feature map in the X direction. For group K1, one cycle can generate the multiplication and addition results of the first column of K1 with the data in the first window, the second column of K1 with the data in the second window, the third column of K1 with the data in the third window, and the fourth column of K1 with the data in the fourth window, and so on. Therefore, the data in the first window pulsates in the 16×4 computation array in the following order from right to left: first column, fifth column, ninth column, and thirteenth column; the data in the second window pulsates in the following order from right to left: second column, sixth column, tenth column, and fourteenth column; the data in the third window pulsates in the following order from right to left: third column, seventh column, eleventh column, and fifteenth column; and the data in the fourth window pulsates in the following order from right to left: fourth column, eighth column, twelfth column, and sixteenth column.

[0046] In this embodiment of the application, convolutional kernel data is also loaded into each computing unit in the computing unit array. Each computing unit in the computing unit array loads convolutional kernel data according to the number of convolutional kernels in the first algorithm layer and the size information of the input feature data.

[0047] It should be noted that, for the two data scheduling methods of weight data reuse and input feature data reuse, the pulsation mode of the input feature data in the horizontal direction of the computing unit array and the grouping mode of the convolution kernel data in the horizontal direction of the computing unit array can be determined simultaneously based on the number of convolution kernels in the first algorithm layer and the size information of the input feature data.

[0048] For example, refer to Figure 3 In the 16×4 computation array, following the order from right to left, the first and second column computation units share the K0 convolution kernel data, so the multi-channel data of the K0 convolution kernel is sequentially input into the first and second columns; the third and fourth column computation units share the K1 convolution kernel data, so the multi-channel data of the K1 convolution kernel is sequentially input into the third and fourth columns; the fifth and sixth column computation units share the K2 convolution kernel data, so the multi-channel data of the K2 convolution kernel is sequentially input into the fifth and sixth columns, and so on, until the multi-channel data of the K7 convolution kernel is sequentially input into the fifteenth and sixteenth columns.

[0049] For example, refer to Figure 4 In a 16×4 computation array, from right to left, the computation units in the first to fourth columns share K0 convolution kernel data, so the multi-channel data of the K0 convolution kernel is sequentially input into the first to second columns; the computation units in the fifth to eighth columns share K1 convolution kernel data, so the multi-channel data of the K1 convolution kernel is sequentially input into the fifth to eighth columns; the computation units in the ninth to twelfth columns share K2 convolution kernel data, so the multi-channel data of the K2 convolution kernel is sequentially input into the ninth to twelfth columns; and the computation units in the thirteenth to sixteenth columns share K3 convolution kernel data, so the multi-channel data of the K3 convolution kernel is sequentially input into the thirteenth and sixteenth columns.

[0050] It should be noted that, regarding the output feature data reuse method, two distribution methods for convolutional kernel data can be determined based on the number of convolutional kernels in the first algorithm layer and the size of the input feature data. The first method is when there are many convolutional kernels and the size of the input feature data is small. In this case, one convolutional kernel data can be loaded in each computation unit, and the input feature data can be sequentially input into each computation unit for convolution operation. The second method is when there are few convolutional kernels and the size of the input feature data is large. In this case, the same multiple convolutional kernel data can be loaded in each row of computation units, and one input feature data can be input into each row of computation units for convolution operation.

[0051] In this embodiment, if the data scheduling method of the first algorithm layer is weight data reuse or input feature data reuse, then multiple computing units along the vertical direction sequentially process the convolution kernel data corresponding to a set of channels and the input feature data corresponding to a set of channels in a convolution kernel; the multiply-accumulate feature data generated in each round in multiple addition registers along the vertical direction are accumulated and output; the number of multiple computing units is determined by the number of channels and convolution kernel size information of the first algorithm layer, as well as the number of multipliers in a computing unit; for example, in the case of 4 channels, 3×3 convolution kernel and two multipliers in a computing unit, then there are 4×3×3 / 2=18 computing units in the vertical direction.

[0052] In this embodiment, if the data scheduling method of the first algorithm layer is output feature data reuse, then each computing unit in the computing unit array processes a set of convolution kernel data corresponding to a set of channels and a set of input feature data corresponding to a set of channels in a convolution kernel; an adder register of each computing unit in the computing unit array accumulates and outputs multiple rounds of output feature data. At this time, each computing unit can output a final output feature data.

[0053] Optional, such as Figure 5As shown, the data processing device 1 further includes a multiplexer 11, which is used to select the data source of the input feature data in each computing unit 10. The multiplexer 11 includes an output terminal 110 and at least one input terminal 111. One output terminal 110 of the multiplexer 11 is connected to the multiplier 100 of a computing unit, and at least one output terminal 111 of the multiplexer 11 is connected to the multiplier 100 of at least one computing unit. The at least one computing unit is a computing unit that is in the same horizontal direction as the computing unit, has the same input feature data, and is executed before the computing unit.

[0054] It should be noted that the number of at least one input terminal is determined by the pulsation pattern of the input feature data in the horizontal direction. (Reference) Figure 3 If the data in the first window of the input feature data pulsates in columns 1, 3, 5, 7, 9, 11, 13, and 15, then for column 3, the input of the multiplexer is the multiplier of column 1, and the output of the multiplexer is the multiplier of column 3. For column 5, the input of the multiplexer is the multipliers of columns 1 and 3, and the output of the multiplexer is the multiplier of column 5. That is, the input feature data of column 5 can be derived from the pulsation of the input feature data of column 1 or column 3.

[0055] Optionally, the data processing device further includes an accumulator output unit 12, which is used to accumulate feature data stored in at least one addition register 101 along the vertical direction in the computing unit array and generate output feature data.

[0056] The accumulator output 12 includes a multi-layer adder register 120; the two input terminals of each bottom-level adder register in the multi-layer adder register 120 are respectively connected to the two adder registers 101 of the two calculation units 10 in the vertical direction, and the two input terminals of each other layer adder register in the multi-layer adder register 120 are respectively connected to the output terminals of the two next-level adder registers.

[0057] The output of each layer of the multi-layer adder register and at least one adder register in the multi-layer adder register also generate output result data.

[0058] In this embodiment of the application, reference is made to Figure 6An accumulator output is set up for a column of 4 computing units. P3, P4, P5, and P6 represent the accumulation result of one computing unit, P0 and P1 represent the accumulation result of two multiply-accumulate units, and P2 represents the accumulation result of four multiply-accumulate units. Therefore, the computing unit array can adapt to different numbers of multiply-accumulate operations in the input channel direction.

[0059] It should be noted that the accumulator output proposed in this application embodiment can enable each computing unit to output data separately, or it can accumulate and output a preset number of computing units in the vertical direction according to the number of channels of the convolution kernel.

[0060] Understandably, each computing unit can perform multiplication, accumulation, and storage operations, allowing the data processing device to flexibly configure different data flow directions according to different algorithm layer parameters, so that the computing units can be fully utilized, thereby improving resource utilization.

[0061] Based on the above-described data processing apparatus, this application also proposes a data processing method, such as... Figure 7 As shown, the method may include:

[0062] S101. Obtain the algorithm layer parameters and data scheduling method of the first algorithm layer in the neural network model.

[0063] The data processing method proposed in this application is applicable to scenarios where neural network models are run in a computing unit array.

[0064] In this embodiment, each algorithm layer in the neural network model corresponds to different algorithm layer parameters. Depending on the different algorithm layer parameters, the data scheduling method applicable to the first algorithm layer is also different. As can be seen from the above embodiments, the computing unit array running the neural network model can implement different data scheduling methods. Therefore, before the computing unit array implements the algorithm logic of the first algorithm layer, the algorithm layer parameters and data scheduling method of the first algorithm layer are obtained first.

[0065] In this embodiment, the first algorithm layer can be a convolutional layer or other neural network layer. The specific layer can be selected according to the actual situation, and this embodiment does not impose any specific limitations.

[0066] In this embodiment, the algorithm layer parameters of the first algorithm layer include parameters such as the number of convolution kernels, input feature size information, number of channels, and convolution kernel size information. The specific parameters can be selected according to the actual situation, and this embodiment does not impose specific limitations.

[0067] It should be noted that the input feature size information includes the width and height information of the output feature, and the convolution kernel size information includes the width and height information of the convolution kernel.

[0068] In this embodiment of the application, the data scheduling method can be input feature data reuse, weight data reuse, or output feature data reuse. The specific method can be selected according to the actual situation, and this embodiment of the application does not impose any specific limitations.

[0069] It is understandable that configuring different data flow directions for input feature data and convolution kernel data for different algorithm layer parameters and data scheduling methods can improve the execution time of the algorithm and thus improve the overall computational performance.

[0070] S102. Based on the algorithm layer parameters and data scheduling method, determine the loading method of input feature data and convolution kernel data on the computing unit array and the corresponding result output method of the computing unit array.

[0071] In an optional embodiment, if the data scheduling method is input feature data reuse or weight data reuse, then based on the number of convolutional kernels and the input feature size information, the data pulsation method of the input feature data in the horizontal direction of the computing unit array and the grouping method of the convolutional kernel data in the horizontal direction of the computing unit array are determined; based on the number of channels and the convolutional kernel size information, the first number of computing units in each vertical direction for accumulation operations is determined; the data pulsation method and the grouping method are determined as the loading method of the input feature data and the convolutional kernel data on the computing unit array; the first number and the accumulation and output of each computing unit in the vertical direction are determined as the result output method.

[0072] For example, such as Figure 3 As shown, if there are 8 convolutional kernels K0-K7 and the size of the input feature data is small, the 16×4 computation array is divided into 8 groups, each group including two columns of computation units. The two columns of computation units share one convolutional kernel data. At this time, the input feature data of two windows are input at one time. The data pulsation pattern of the first window in the computation unit array is from right to left: column 1, column 3, column 5, column 7, column 9, column 11, column 13 and column 15. The data pulsation pattern of the first window in the computation unit array is from right to left: column 2, column 4, column 6, column 8, column 10, column 12, column 14 and column 16.

[0073] For example, such as Figure 4As shown, if the size information of the input feature data of the four convolution kernels K0-K3 is relatively small, the 16×4 computation array is divided into 4 groups, each group including four columns of computation units. Then, the four columns of computation units share one convolution kernel data. At this time, the input feature data of four windows are input at one time. Then, the data of the first window is arranged in the data pulsation pattern of the computation unit array from right to left: the first column, the fifth column, the ninth column, and the thirteenth column; the data of the second window is arranged in the data pulsation pattern of ...

[0074] The above two examples illustrate the process of determining the data pulsation pattern of the input feature data in the horizontal direction of the computing unit array, and the grouping pattern of the convolution kernel data in the horizontal direction of the computing unit array, based on the number of convolution kernels and the input feature size information. Next, regarding the process of determining the first number of computing units in each vertical direction of the computing unit array for accumulation operations based on the number of channels and the convolution kernel size information, this is specifically obtained by multiplying the number of channels by the convolution kernel size information and then dividing by the number of multipliers in a computing unit. For example, in the case of 4 channels, 3×3 convolution kernels, and two multipliers in a computing unit, there are 4×3×3 / 2=18 computing units in the vertical direction.

[0075] In another optional embodiment, if the data scheduling method is output feature data reuse, then based on the number of convolution kernels and the input feature size information, the data flow direction of the input feature data in each horizontal computing unit in the computing unit array and the distribution method of the convolution kernel data in the computing unit array are determined; the data flow direction and distribution method are determined as the loading method of the input feature data and convolution kernel data on the computing unit array; the result output method of the computing unit array is determined as each computing unit accumulates and outputs the multi-round multiplication and accumulation data.

[0076] In this embodiment, if the number of convolutional kernels is large and the input feature size is small, then one convolutional kernel data is loaded into each computing unit, and at the same time, a window of input feature data is sequentially input into the computing unit array from right to left. For example, as shown below... Figure 8 As shown, there are 64 convolutional kernels, K0-K63, and the input feature data size is small. In this case, a convolutional kernel is loaded in each computing unit in the computing unit array, and the window data of the first window in the input feature data is input into the computing unit array in the order from right to left.

[0077] If the number of convolutional kernels is small but the input feature size is large, then the same set of convolutional kernel data is loaded into each row of computational units, and at the same time, different window data of the input feature data are input into each row of computational units respectively. For example, Figure 9 As shown, there are 16 convolutional kernels, K0-K15, and the input feature data size is relatively large. In this case, 16 convolutional kernels, K0-K15, are loaded into each row of computational units. The window data of the first window in the input feature data is input into the first row of computational units in order from right to left. The window data of the second window in the input feature data is input into the second row of computational units in order from right to left. The window data of the third window in the input feature data is input into the third row of computational units in order from right to left. The window data of the fourth window in the input feature data is input into the fourth row of computational units in order from right to left.

[0078] It is understandable that configuring the data flow of input feature data and convolution kernel data according to the number of convolution kernels, number of channels, size of input feature data, and size of convolution kernel in the first algorithm layer can reduce the number of memory accesses, improve MAC utilization, and thus reduce power consumption.

[0079] S103. According to the loading method of input feature data and convolution kernel data on the computing unit array, load the input feature data and convolution kernel into the computing unit array, and perform multiply-accumulate operation on the computing unit array to obtain multiply-accumulate data.

[0080] In this embodiment, after determining the loading method of the input feature data and convolution kernel data on the computing array, the input feature data and convolution kernel are loaded into the computing unit array according to the loading method of the input feature data and convolution kernel data on the computing unit array, and multiply-accumulate operations are performed in the computing unit array to obtain multiply-accumulate data.

[0081] S104. Process the multiply-accumulate data based on the result output method to obtain the output feature data.

[0082] In this embodiment of the application, after determining the result output method corresponding to the computing unit array based on the algorithm layer parameters and data scheduling method, the multiply-accumulate data is processed based on the result output method to obtain the output feature data.

[0083] In one optional embodiment, if the data scheduling method is weight data reuse or input feature data reuse, the multiply-accumulate data output by the calculation unit in the vertical direction is accumulated to obtain the output feature data.

[0084] In another optional embodiment, if the data scheduling method is output feature data reuse, then each computing unit accumulates the multi-round multiplication and accumulation data to directly obtain the output feature data.

[0085] Understandably, each computing unit can perform multiplication, accumulation, and storage operations, allowing the data processing device to flexibly configure different data flow directions according to different algorithm layer parameters and data scheduling methods, so that the computing units can be fully utilized, thereby improving resource utilization.

[0086] Based on the above data processing method, this application embodiment also provides a data processing device 1. For example... Figure 10 As shown, the data processing device 1 includes:

[0087] Acquisition unit 13 is used to acquire the algorithm layer parameters and data scheduling method of the first algorithm layer in the neural network model;

[0088] The determining unit 14 is used to determine the loading method of the input feature data and convolution kernel data on the computing unit array and the result output method corresponding to the computing unit array based on the algorithm layer parameters and the data scheduling method.

[0089] The loading unit 15 is used to load the input feature data and convolution kernel into the computing unit array according to the loading method of the input feature data and convolution kernel data on the computing unit array, and perform multiply-accumulate operation on the computing unit array to obtain multiply-accumulate data;

[0090] Processing unit 16 is used to process the multiply-accumulate data based on the data output method to obtain output feature data.

[0091] Optionally, the algorithm layer parameters of the first algorithm layer include the number of convolution kernels, input feature size information, number of channels, and convolution kernel size information.

[0092] Optionally, the determining unit 14 is further configured to, if the data scheduling method is input feature data reuse or weight data reuse, determine the data pulsation mode of the input feature data in the horizontal direction of the computing unit array and the grouping mode of the convolution kernel data in the horizontal direction of the computing unit array based on the number of convolution kernels and the input feature size information; determine the first number of computing units in the computing unit array that perform accumulation operations in each vertical direction based on the number of channels and the convolution kernel size information; determine the data pulsation mode and the grouping mode as the loading mode of the input feature data and the convolution kernel data on the computing unit array; and determine the first number and the accumulation and output of each computing unit in the vertical direction as the result output mode.

[0093] Optionally, the determining unit 14 is further configured to, if the data scheduling method is output feature data reuse, determine the data flow direction of the input feature data in each horizontal computing unit in the computing unit array and the distribution method of the convolution kernel data in the computing unit array based on the number of convolution kernels and the input feature size information; determine the data flow direction and the distribution method as the loading method of the input feature data and the convolution kernel data on the computing unit array; and determine the result output method of the computing unit array as each computing unit accumulating and outputting multi-round multiplication and accumulation data.

[0094] This application provides a data processing apparatus that acquires the algorithm layer parameters and data scheduling method of the first algorithm layer in a neural network model; determines the loading method of input feature data and convolution kernel data on the computing unit array and the corresponding result output method of the computing unit array based on the algorithm layer parameters and data scheduling method; loads the input feature data and convolution kernel data into the computing unit array according to the loading method of the input feature data and convolution kernel data on the computing unit array, and performs multiplication and accumulation operations on the computing unit array to obtain multiply-accumulated data; processes the multiply-accumulated data based on the data output method to obtain output feature data. Therefore, the data processing apparatus proposed in this embodiment allows each computing unit to perform multiplication, accumulation, and storage operations, enabling the data processing apparatus to flexibly configure different data flow directions according to different algorithm layer parameters, thus fully utilizing the computing units and improving resource utilization.

[0095] Figure 11 A schematic diagram of the composition structure of a data processing device 1 provided in this application embodiment. Figure 2 In practical applications, based on the same disclosed concept of the above embodiments, such as Figure 11 As shown, the data processing device 1 in this embodiment includes a processor 17, a memory 18, and a communication bus 19.

[0096] In a specific embodiment, the acquisition unit 13, determination unit 14, loading unit 15, and processing unit 16 described above can be implemented by a processor 17 located on the device 1. The processor 17 can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), CPU, controller, microcontroller, and microprocessor. It is understood that for different devices, the electronic device used to implement the above processor functions can also be other types; this embodiment does not specifically limit this.

[0097] In this embodiment, the communication bus 19 is used to establish a connection between the processor 17 and the memory 18; when the processor 17 executes the running program stored in the memory 18, it implements the following data processing method:

[0098] Obtain the algorithm layer parameters and data scheduling method of the first algorithm layer in the neural network model; based on the algorithm layer parameters and the data scheduling method, determine the loading method of the input feature data and convolution kernel data on the computing unit array and the corresponding result output method of the computing unit array; according to the loading method of the input feature data and convolution kernel data on the computing unit array, load the input feature data and convolution kernel into the computing unit array, and perform multiply-accumulate operations on the computing unit array to obtain multiply-accumulate data; process the multiply-accumulate data based on the data output method to obtain output feature data.

[0099] Furthermore, the algorithm layer parameters of the first algorithm layer include the number of convolution kernels, input feature size information, number of channels, and convolution kernel size information.

[0100] Furthermore, the processor 17 is also configured to, if the data scheduling method is input feature data reuse or weight data reuse, determine the data pulsation mode of the input feature data in the horizontal direction of the computing unit array and the grouping mode of the convolution kernel data in the horizontal direction of the computing unit array based on the number of convolution kernels and the input feature size information; determine the first number of computing units in the computing unit array that perform accumulation operations in each vertical direction based on the number of channels and the convolution kernel size information; determine the data pulsation mode and the grouping mode as the loading mode of the input feature data and the convolution kernel data on the computing unit array; and determine the first number and the accumulation and output of each computing unit in the vertical direction as the result output mode.

[0101] Furthermore, the processor 17 is also configured to, if the data scheduling method is output feature data reuse, determine the data flow direction of the input feature data in each horizontal computing unit in the computing unit array and the distribution method of the convolution kernel data in the computing unit array based on the number of convolution kernels and the input feature size information; determine the data flow direction and the distribution method as the loading method of the input feature data and the convolution kernel data on the computing unit array; and determine the result output method of the computing unit array as each computing unit accumulating and outputting multi-round multiplication and accumulation data.

[0102] This application provides a storage medium storing a computer program thereon. The computer-readable storage medium stores one or more programs, which can be executed by one or more processors and applied in a data processing device. The computer program implements the data processing method described above.

[0103] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause an image display device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this disclosure.

[0105] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A data processing apparatus, characterized in that, The data processing device deploys an array of computing units. Each computing unit in the array consists of a set of multipliers and an adder register connected to the set of multipliers. A multiplier is used to perform a multiplication operation on a convolution kernel and an input feature, and an adder register is used to accumulate and store a set of data output by the set of multipliers. Each group of computing units in the horizontal direction of the computing unit array propagates input feature data through data pulsation; each computing unit in the computing unit array inputs convolutional kernel data according to the number of convolutional kernels of the first algorithm layer and the size information of the input feature data; the data pulsation method is determined by the number of convolutional kernels of the first algorithm layer and the size information of the input feature data. The computing unit array performs an accumulation and output operation on at least one addition register of at least one computing unit along the vertical direction.

2. The apparatus according to claim 1, characterized in that, The data processing device further includes a multiplexer for selecting the data source of input feature data in each computing unit. The multiplexer includes an output terminal and at least one input terminal. One output terminal of the multiplexer is connected to a multiplier of a computing unit, and at least one input terminal of the multiplexer is connected to a multiplier of at least one computing unit. The at least one computing unit is a computing unit located at the same horizontal level as the first computing unit, receiving the same input feature data, and whose execution order precedes that of the first computing unit.

3. The apparatus according to claim 2, characterized in that, The number of at least one input terminal is determined by the pulsation pattern of the input feature data in the horizontal direction, and the pulsation pattern is determined by the number of convolution kernels in the first algorithm layer and the size information of the input feature data.

4. The apparatus according to any one of claims 1-3, characterized in that, The data processing device further includes an accumulator output unit, which is used to accumulate the multiply-accumulate data stored in at least one addition register along the vertical direction in the computing unit array and generate output feature data.

5. The apparatus according to claim 4, characterized in that, The accumulator output includes a multi-layer adder register; the two input terminals of each bottom-level adder register in the multi-layer adder register are respectively connected to the two adder registers of two calculation units in the vertical direction, and the two input terminals of each other-level adder register in the multi-layer adder register are respectively connected to the output terminals of two next-level adder registers; The output of each layer of the multi-layer adder register and at least one adder register in the multi-layer adder register also generate output multiply-accumulate data respectively.

6. The apparatus according to claim 5, characterized in that, If the data scheduling method of the first algorithm layer is weight data reuse or input feature data reuse, then the multiple computing units along the vertical direction sequentially process the convolution kernel data corresponding to a set of channels and the input feature data corresponding to a set of channels in a convolution kernel; the number of multiple computing units is determined by the number of channels and convolution kernel size information of the first algorithm layer, as well as the number of multipliers in a computing unit; The accumulator output is used to accumulate and output the multiply-accumulate feature data generated in each round in the multiple adder registers along the vertical direction.

7. The apparatus according to claim 5, characterized in that, If the data scheduling method of the first algorithm layer is output feature data reuse, then each computing unit in the computing unit array processes a set of convolution kernel data corresponding to a set of channels and a set of input feature data corresponding to a set of channels in a convolution kernel; The addition register in each computing unit of the computing unit array performs an accumulation operation on the multi-round multiplication and accumulation data and outputs the data through the accumulation output device.

8. The apparatus according to claim 5, characterized in that, Each computing unit in the computing unit array loads convolutional kernel data according to the number of convolutional kernels in the first algorithm layer, the size information of the input feature data, and the data scheduling method.

9. A data processing method, characterized in that, Applied to the data processing apparatus according to any one of claims 1-7, the method comprises: Obtain the algorithm layer parameters and data scheduling method of the first algorithm layer in the neural network model; Based on the algorithm layer parameters and the data scheduling method, the loading method of input feature data and convolutional kernel data on the computing unit array and the corresponding result output method of the computing unit array are determined; the algorithm layer parameters include the number of convolutional kernels, input feature size information, number of channels, and convolutional kernel size information; the data scheduling method is input feature data reuse, weight data reuse, or output feature data reuse; According to the loading method of input feature data and convolution kernel data on the computing unit array, the input feature data and convolution kernel are loaded into the computing unit array, and multiply-accumulate operation is performed on the computing unit array to obtain multiply-accumulate data; The multiply-accumulate data is processed based on the aforementioned data output method to obtain output feature data.

10. The method according to claim 9, characterized in that, The algorithm layer parameters of the first algorithm layer include the number of convolution kernels, input feature size information, number of channels, and convolution kernel size information.

11. The method according to claim 10, characterized in that, Based on the algorithm layer parameters and the data scheduling method, the loading method of input feature data and convolutional kernel data on the computing unit array and the corresponding result output method of the computing unit array are determined, including: If the data scheduling method is input feature data reuse or weight data reuse, then based on the number of convolution kernels and the input feature size information, the data pulsation method of the input feature data in the horizontal direction of the computing unit array and the grouping method of the convolution kernel data in the horizontal direction of the computing unit array are determined. Based on the number of channels and the convolution kernel size information, determine the first number of computing units in the computing unit array that perform accumulation operations in each vertical direction; The data pulsation method and the grouping method are determined as the loading method of input feature data and convolution kernel data on the computing unit array; the first quantity and the summation and output of each computing unit in the vertical direction are determined as the result output method.

12. The method according to claim 10, characterized in that, The step of determining the loading method of input feature data and convolutional kernel data on the computing unit array and the corresponding result output method of the computing unit array based on the algorithm layer parameters and the data scheduling method includes: If the data scheduling method is output feature data reuse, then based on the number of convolution kernels and the input feature size information, the data flow direction of the input feature data in each horizontal computing unit in the computing unit array, and the distribution method of the convolution kernel data in the computing unit array are determined. The data flow direction and the distribution method are determined as the loading method of input feature data and convolutional kernel data on the computing unit array; The output method of the computing unit array is determined to be that each computing unit accumulates and outputs multi-round multiplication and accumulation data.

13. A data processing apparatus, characterized in that, The device includes: The acquisition unit is used to acquire the algorithm layer parameters and data scheduling method of the first algorithm layer in the neural network model. The determining unit is used to determine, based on the algorithm layer parameters and the data scheduling method, the loading method of input feature data and convolutional kernel data on the computing unit array and the corresponding result output method of the computing unit array; the algorithm layer parameters include the number of convolutional kernels, input feature size information, number of channels, and convolutional kernel size information; the data scheduling method is input feature data reuse, weight data reuse, or output feature data reuse; A loading unit is used to load the input feature data and convolutional kernels into the computing unit array according to the loading method of the input feature data and convolutional kernel data on the computing unit array, and perform multiplication and accumulation operations in the computing unit array to obtain multiplication and accumulation data; the input feature data is propagated through data pulsation in each group of computing units along the horizontal direction of the computing unit array; convolutional kernel data is input in each computing unit of the computing unit array according to the number of convolutional kernels of the first algorithm layer and the size information of the input feature data; the data pulsation method is determined by the number of convolutional kernels of the first algorithm layer and the size information of the input feature data; the computing unit array performs accumulation and output operations in at least one addition register of at least one computing unit along the vertical direction; The processing unit is used to process the multiply-accumulate data based on the data output method to obtain output feature data.

14. A data processing apparatus, characterized in that, The device includes a processor, a memory, and a communication bus; when the processor executes the running program stored in the memory, it implements the method as described in any one of claims 9-12.

15. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 9-12.

Images