A data processing apparatus and a data processing method

Abstract

The data processing device comprises a product calculation circuit for calculating a first group of products and a second group of products, the first group of products comprising products of high N bits of a first multiplier and a first multiplicand and products of high N bits of a second multiplier and a second multiplicand, and the second group of products comprising products of low N bits of the first multiplier and the first multiplicand of the data processing device and products of low N bits of the second multiplier and the second multiplicand of the data processing device, the first multiplier and the second multiplier are both 2N bits, and N is a positive integer. An accumulation circuit is used for accumulating the first group of products and the second group of products respectively. By merging partial products with the same left shift number in multiple groups of multiplication operations and accumulating the merged results respectively, the logic overhead of the data processing device is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital signal processing, in particular to a data processing device and a data processing method. BACKGROUND

[0002] Convolutional neural network (CNN) has a relatively wide application scenario in the fields of image and voice recognition. In a model implementing a convolutional neural network algorithm, convolution calculation accounts for 90% of the calculation amount of the entire algorithm model, and therefore efficient calculation of the convolution layer is the key to greatly improving the calculation efficiency of the CNN algorithm model, and hardware acceleration to implement convolution calculation is an effective approach.

[0003] At present, Nbit*2Nbit and 2Nbit*2Nbit processors have large logic resource overhead and low performance, where * represents convolution, and N is a positive integer. Specifically, as shown in FIG. 1, a schematic diagram of an Nbit*Nbit data processing device includes Nbit*Nbit multipliers and a 2Nbit-bit-width adder. Each Nbit*Nbit multiplier outputs 2Nbit data, and when there are two or more groups of data for convolution operation, a 2Nbit-bit-width adder is needed to accumulate the output results of the multipliers. The area of a 2Nbit*Nbit multiplier is twice that of an Nbit*Nbit multiplier, and the area of a 2Nbit*2Nbit multiplier is four times that of an Nbit*Nbit multiplier, which greatly increases the area of the multiplier compared with the Nbit*Nbit multiplier. In addition, as shown in FIG. 2, in order to enable the processor to process 4 groups of Nbit*Nbit operations, 2 groups of 2Nbit*Nbit operations, and 1 group of 2Nbit*2Nbit operations in one clock, a 4Nbit-bit-width adder is needed, which is one time larger than the Nbit*Nbit scheme in terms of adder bit width. The increase in the area of the multiplier and the expansion of the bit width of the adder will increase the logic resource overhead of the processor and reduce the performance of the processor. Therefore, how to design a processor with small logic resource overhead needs to be solved urgently. Figure 1 Figure 2 The present application provides a data processing device and a data processing method, which, compared with a scheme in which the high-bit part of each group of multipliers is directly added to the product of the low-bit part of the corresponding multiplier and the multiplicand after being shifted, combines the partial products with the same number of left shifts in a plurality of multiplication operations, and respectively accumulates the combined results, thereby greatly saving the logic resources. SUMMARY

[0004] The present application provides a data processing device and a data processing method, which, compared with a scheme in which the high-bit part of each group of multipliers is directly added to the product of the low-bit part of the corresponding multiplier and the multiplicand after being shifted, combines the partial products with the same number of left shifts in a plurality of multiplication operations, and respectively accumulates the combined results, thereby greatly saving the logic resources. ​

[0005] The first aspect of the present application provides a data processing device, which can comprise: a product calculation circuit, configured to calculate a first group of products and a second group of products, the first group of products can comprise products of high N bits of a first multiplier and a first multiplicand, and products of high N bits of a second multiplier and a second multiplicand, the second group of products can comprise products of low N bits of the first multiplier and the first multiplicand, and products of low N bits of the second multiplier and the second multiplicand, the first multiplier and the second multiplier are both 2N bits, and N is a positive integer; and an accumulation circuit, configured to accumulate the first group of products and the second group of products respectively. According to the first aspect, when two groups or more groups of data are subjected to convolution operation, the high bit part of each group of multipliers is separately accumulated and then subjected to shift processing, thereby avoiding the expansion of the adder caused by directly adding the low bit part of the corresponding multiplier and the product of the multiplicand after the high bit part of each group of multipliers is shifted.

[0006] Optionally, in combination with the first aspect, in a first possible implementation manner, the data processing device can further comprise a first shifter and a first adder, the first multiplicand and the second multiplicand are both N bits, the first shifter is configured to shift the accumulation result of the first group of products to obtain a first shift result, and the first adder is configured to accumulate the first shift result and the second group of products. According to the first possible implementation manner of the first aspect, a specific scheme of how to use an Nbit*Nbit multiplier to form a 2Nbit*Nbit processor is given.

[0007] Optionally, in combination with the first aspect, in a second possible implementation manner, the first multiplicand and the second multiplicand are 2N bits, the first group of products can include a high bit product and a high-low bit product, the second group of products can include a low-high bit product and a low bit product, the product calculation circuit is specifically configured to: calculate the high bit product, the high bit product can include a product of high N bits of the first multiplier and high N bits of the first multiplicand, and a product of high N bits of the second multiplier and high N bits of the second multiplicand; calculate the high-low bit product, the high-low bit product can include a product of high N bits of the first multiplier and low N bits of the first multiplicand, and a product of high N bits of the second multiplier and low N bits of the second multiplicand; calculate the low-high bit product, the low-high bit product can include a product of low N bits of the first multiplier and high N bits of the first multiplicand, and a product of low N bits of the second multiplier and high N bits of the second multiplicand; calculate the low bit product, the low bit product can include a product of low N bits of the first multiplier and low N bits of the first multiplicand, and a product of low N bits of the second multiplier and low N bits of the second multiplicand; and the accumulation circuit is specifically configured to accumulate the high bit product, the high-low bit product, the low-high bit product, and the low bit product by the second adder with a bit width of 2N bits. According to the second possible implementation manner of the first aspect, the 2Nbit*2Nbit processor can be constructed by the Nbit*Nbit multiplier.

[0008] Optionally, in combination with the second possible implementation manner of the first aspect, in a third possible implementation manner, the data processing apparatus can further include a second shifter, a third adder, a third shifter, a fourth shifter, and a fourth adder, the second shifter is configured to left shift a result of the high bit product accumulation by N bits to obtain a second shift result; the third adder is configured to accumulate the second shift result and a result of the high-low bit product accumulation; the third shifter is configured to left shift a result output by the third adder by N bits to obtain a third shift result; the fourth shifter is configured to left shift a result of the low-high bit product accumulation by N bits to obtain a fourth shift result; and the fourth adder is configured to accumulate the third shift result, the fourth shift result, and the low bit product.

[0009] Optionally, in combination with the third possible implementation manner of the first aspect, in a fourth possible implementation manner, the data processing apparatus can further include a fifth adder configured to accumulate the fourth shift result and the low bit product; and the fourth adder is specifically configured to accumulate the third shift result and a result output by the fifth adder.

[0010] Optionally, in combination with the first aspect or the first possible implementation manner of the first aspect, in a fifth possible implementation manner, the data processing apparatus can further include a split logic circuit configured to output high N bits and low N bits of the first multiplier, and high N bits and low N bits of the second multiplier by a selector MUX.

[0011] Optionally, in combination with the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner, the splitting logic circuit is further configured to construct a first association relationship, the first association relationship can include an association relationship between the high N bits of the first multiplier and the first multiplicand, an association relationship between the low N bits of the first multiplier and the first multiplicand, an association relationship between the high N bits of the second multiplier and the second multiplicand, and an association relationship between the low N bits of the second multiplier and the second multiplicand.

[0012] Optionally, in combination with the second to fourth possible implementation manners of the first aspect, in a seventh possible implementation manner, the data processing apparatus can further include a splitting logic circuit, configured to output the high N bits and the low N bits of the first multiplier, the high N bits and the low N bits of the second multiplier, the high N bits and the low N bits of the first multiplicand, and the high N bits and the low N bits of the second multiplicand through a selector MUX.

[0013] Optionally, in combination with the seventh possible implementation manner of the first aspect, in an eighth possible implementation manner, the splitting logic circuit is further configured to construct a second association relationship, the second association relationship can include an association relationship between the high N bits of the first multiplier and the high N bits of the first multiplicand, an association relationship between the high N bits of the first multiplier and the low N bits of the first multiplicand, an association relationship between the low N bits of the first multiplier and the high N bits of the first multiplicand, an association relationship between the low N bits of the first multiplier and the low N bits of the first multiplicand, an association relationship between the high N bits of the second multiplier and the high N bits of the second multiplicand, an association relationship between the high N bits of the second multiplier and the low N bits of the second multiplicand, an association relationship between the low N bits of the second multiplier and the high N bits of the second multiplicand, and an association relationship between the low N bits of the second multiplier and the low N bits of the second multiplicand.

[0014] Optionally, in combination with the eighth possible implementation manner of the first aspect, in a ninth possible implementation manner, the data processing apparatus can further include a data random access memory (RAM) and a weight RAM, the data RAM is configured to store the high N bits and the low N bits of the first multiplier and the high N bits and the low N bits of the second multiplier, and the weight RAM is configured to store the high N bits and the low N bits of the first multiplicand and the high N bits and the low N bits of the second multiplicand according to the second association relationship.

[0015] Optionally, in combination with the first aspect or the first to ninth possible implementation manners of the first aspect, in a tenth possible implementation manner, the first multiplier and the second multiplier are feature layer data, and the first multiplicand and the second multiplicand are convolution kernel data, or the first multiplier and the second multiplier are convolution kernel data, and the first multiplicand and the second multiplicand are feature layer data.

[0016] The second aspect of the application provides a data processing method, which can include: calculating a first group of products and a second group of products, the first group of products can include products of high N bits of a first multiplier and a first multiplicand, products of high N bits of a second multiplier and a second multiplicand, and the second group of products can include products of low N bits of the first multiplier and the first multiplicand, products of low N bits of the second multiplier and the second multiplicand, the first multiplier and the second multiplier are both 2N bits, and N is a positive integer. The first group of products and the second group of products are respectively accumulated.

[0017] Optionally, in combination with the second aspect, in a first possible implementation, the method can further include: performing shift processing on a result of the accumulation of the first group of products to obtain a first shift result. The first shift result and the second group of products are accumulated.

[0018] Optionally, in combination with the second aspect, in a second possible implementation, the calculating of the first group of products and the second group of products can include: calculating high-bit products, the high-bit products can include products of high N bits of the first multiplier and high N bits of the first multiplicand, and products of high N bits of the second multiplier and high N bits of the second multiplicand. Calculating high-low-bit products, the high-low-bit products can include products of high N bits of the first multiplier and low N bits of the first multiplicand, and products of high N bits of the second multiplier and low N bits of the second multiplicand. Calculating low-high-bit products, the low-high-bit products can include products of low N bits of the first multiplier and high N bits of the first multiplicand, and products of low N bits of the second multiplier and high N bits of the second multiplicand. Calculating low-bit products, the low-bit products can include products of low N bits of the first multiplier and low N bits of the first multiplicand, and products of low N bits of the second multiplier and low N bits of the second multiplicand. The accumulation of the first group of products and the second group of products can include: respectively accumulating the high-bit products, the high-low-bit products, the low-high-bit products, and the low-bit products.

[0019] Optionally, in combination with the second aspect and the second possible implementation, in a third possible implementation, the method can further include: left shifting a result of the accumulation of the high-bit products by N bits to obtain a second shift result. Accumulating the second shift result and a result of the accumulation of the high-low-bit products. Left shifting a result of the accumulation of the second shift result and the result of the accumulation of the high-low-bit products by N bits to obtain a third shift result. Left shifting a result of the accumulation of the low-high-bit products by N bits to obtain a fourth shift result. Accumulating the third shift result, the fourth shift result, and the low-bit products.

[0020] Optionally, in combination with the third possible implementation of the second aspect, in a fourth possible implementation, the method can further include: accumulating the fourth shift result and the low-bit products. The accumulation of the third shift result, the fourth shift result, and the low-bit products can include: accumulating a result of the accumulation of the third shift result and the fourth shift result and the low-bit products.

[0021] Optionally, in combination with the second aspect or the first possible implementation manner of the second aspect, in a fifth possible implementation manner, the method further includes: outputting the high N bits and the low N bits of the first multiplier, and the high N bits and the low N bits of the second multiplier.

[0022] Optionally, in combination with the fifth possible implementation manner of the second aspect, in a sixth possible implementation manner, the method further includes: constructing a first association relationship, the first association relationship including an association relationship between the high N bits of the first multiplier and the first multiplicand, an association relationship between the low N bits of the first multiplier and the first multiplicand, an association relationship between the high N bits of the second multiplier and the second multiplicand, and an association relationship between the low N bits of the second multiplier and the second multiplicand.

[0023] Optionally, in combination with the second to fourth possible implementation manners of the second aspect, in a seventh possible implementation manner, the method further includes: outputting the high N bits and the low N bits of the first multiplier, the high N bits and the low N bits of the second multiplier, the high N bits and the low N bits of the first multiplicand, and the high N bits and the low N bits of the second multiplicand.

[0024] Optionally, in combination with the eighth possible implementation manner of the second aspect, in a ninth possible implementation manner, the method further includes: constructing a second association relationship, the second association relationship including an association relationship between the high N bits of the first multiplier and the high N bits of the first multiplicand, an association relationship between the high N bits of the first multiplier and the low N bits of the first multiplicand, an association relationship between the low N bits of the first multiplier and the high N bits of the first multiplicand, an association relationship between the low N bits of the first multiplier and the low N bits of the first multiplicand, an association relationship between the high N bits of the second multiplier and the high N bits of the second multiplicand, an association relationship between the high N bits of the second multiplier and the low N bits of the second multiplicand, an association relationship between the low N bits of the second multiplier and the high N bits of the second multiplicand, and an association relationship between the low N bits of the second multiplier and the low N bits of the second multiplicand.

[0025] Optionally, in combination with the second aspect or the first to ninth possible implementation manners of the second aspect, in a tenth possible implementation manner, the first multiplier and the second multiplier are feature layer data, and the first multiplicand and the second multiplicand are convolution kernel data, or the first multiplier and the second multiplier are convolution kernel data, and the first multiplicand and the second multiplicand are feature layer data.

[0026] The third aspect of the present application provides a data processing device, which can comprise: a product calculation module, configured to calculate a first group of products and a second group of products, the first group of products can comprise products of high N bits of a first multiplier and a first multiplicand, and products of high N bits of a second multiplier and a second multiplicand, the second group of products can comprise products of low N bits of the first multiplier and the first multiplicand, and products of low N bits of the second multiplier and the second multiplicand, the first multiplier and the second multiplier are both 2N bits, and N is a positive integer; and an accumulation module, configured to accumulate the first group of products and the second group of products respectively. According to the third aspect, when two groups or more groups of data are subjected to convolution operation, the high bit part of each group of multipliers is separately accumulated and then subjected to shift processing, thereby avoiding the expansion of the addition module caused by directly adding the low bit part of the corresponding multiplier to the product of the high bit part of each group of multipliers and the multiplicand after the shift.

[0027] Optionally, in combination with the third aspect, in a first possible implementation, the data processing device can further comprise a first shift module and a first addition module, the first multiplicand and the second multiplicand are both N bits, the first shift module is configured to shift the accumulation result of the first group of products to obtain a first shift result, and the first addition module is configured to accumulate the first shift result and the second group of products. According to the first possible implementation of the third aspect, a specific scheme of how to use an Nbit*Nbit multiplication module to form a 2Nbit*Nbit processing module is given.

[0028] Optionally, in combination with the third aspect, in a second possible implementation manner, the first multiplicand and the second multiplicand are 2N bits, the first group of products can include a high bit product and a high-low bit product, the second group of products can include a low-high bit product and a low bit product, the product calculation module is specifically configured to: calculate the high bit product, the high bit product can include a product of high N bits of the first multiplier and high N bits of the first multiplicand, and a product of high N bits of the second multiplier and high N bits of the second multiplicand; calculate the high-low bit product, the high-low bit product can include a product of high N bits of the first multiplier and low N bits of the first multiplicand, and a product of high N bits of the second multiplier and low N bits of the second multiplicand; calculate the low-high bit product, the low-high bit product can include a product of low N bits of the first multiplier and high N bits of the first multiplicand, and a product of low N bits of the second multiplier and high N bits of the second multiplicand; and calculate the low bit product, the low bit product can include a product of low N bits of the first multiplier and low N bits of the first multiplicand, and a product of low N bits of the second multiplier and low N bits of the second multiplicand. The accumulation module is specifically configured to accumulate the high bit product, the high-low bit product, the low-high bit product, and the low bit product by using a second addition module with a bit width of 2N bits. According to the second possible implementation manner of the third aspect, the 2Nbit*2Nbit processing module can be constructed by using the Nbit*Nbit multiplication module.

[0029] Optionally, in combination with the third aspect or the second possible implementation manner of the third aspect, in a third possible implementation manner, the data processing apparatus can further include a second shift module, a third addition module, a third shift module, a fourth shift module, and a fourth addition module. The second shift module is configured to left shift a result of the high bit product accumulation by N bits to obtain a second shift result. The third addition module is configured to accumulate the second shift result and a result of the high-low bit product accumulation. The third shift module is configured to left shift a result output by the third addition module by N bits to obtain a third shift result. The fourth shift module is configured to left shift a result of the low-high bit product accumulation by N bits to obtain a fourth shift result. The fourth addition module is configured to accumulate the third shift result, the fourth shift result, and the low bit product.

[0030] Optionally, in combination with the third aspect or the third possible implementation manner of the third aspect, in a fourth possible implementation manner, the data processing apparatus can further include a fifth addition module configured to accumulate the fourth shift result and the low bit product. The fourth addition module is specifically configured to accumulate the third shift result and a result output by the fifth addition module.

[0031] Optionally, in combination with the third aspect or the first possible implementation manner of the third aspect, in a fifth possible implementation manner, the data processing apparatus can further include a split logic module configured to output high N bits and low N bits of the first multiplier and high N bits and low N bits of the second multiplier by using a selection module MUX.

[0032] Optionally, in combination with the fifth possible implementation manner of the third aspect, in a sixth possible implementation manner, the splitting logic module is further configured to construct a first association relationship, the first association relationship can include an association relationship between the high N bits of the first multiplier and the first multiplicand, an association relationship between the low N bits of the first multiplier and the first multiplicand, an association relationship between the high N bits of the second multiplier and the second multiplicand, and an association relationship between the low N bits of the second multiplier and the second multiplicand.

[0033] Optionally, in combination with the second to fourth possible implementation manners of the third aspect, in a seventh possible implementation manner, the data processing apparatus can further include a splitting logic module, configured to output the high N bits and the low N bits of the first multiplier, the high N bits and the low N bits of the second multiplier, the high N bits and the low N bits of the first multiplicand, and the high N bits and the low N bits of the second multiplicand through the selection module MUX.

[0034] Optionally, in combination with the seventh possible implementation manner of the third aspect, in an eighth possible implementation manner, the splitting logic module is further configured to construct a second association relationship, the second association relationship can include an association relationship between the high N bits of the first multiplier and the high N bits of the first multiplicand, an association relationship between the high N bits of the first multiplier and the low N bits of the first multiplicand, an association relationship between the low N bits of the first multiplier and the high N bits of the first multiplicand, an association relationship between the low N bits of the first multiplier and the low N bits of the first multiplicand, an association relationship between the high N bits of the second multiplier and the high N bits of the second multiplicand, an association relationship between the high N bits of the second multiplier and the low N bits of the second multiplicand, an association relationship between the low N bits of the second multiplier and the high N bits of the second multiplicand, and an association relationship between the low N bits of the second multiplier and the low N bits of the second multiplicand.

[0035] Optionally, in combination with the eighth possible implementation manner of the third aspect, in a ninth possible implementation manner, the data processing apparatus can further include a data random access memory module RAM, and a weight RAM, the data RAM is configured to store the high N bits and the low N bits of the first multiplier and the high N bits and the low N bits of the second multiplier, and the weight RAM is configured to store the high N bits and the low N bits of the first multiplicand and the high N bits and the low N bits of the second multiplicand according to the second association relationship.

[0036] Optionally, in combination with the third aspect or the first to ninth possible implementation manners of the third aspect, in a tenth possible implementation manner, the first multiplier and the second multiplier are feature layer data, and the first multiplicand and the second multiplicand are convolution kernel data, or the first multiplier and the second multiplier are convolution kernel data, and the first multiplicand and the second multiplicand are feature layer data.

[0037] The fourth aspect of the present application provides a field programmable gate array FPGA, the FPGA can include the data processing apparatus described in the first aspect or any one of the possible implementation manners of the first aspect.

[0038] Through the technical scheme provided by the embodiment of the application, 2Nbit data is split into Nbit, and multiplication including 2Nbit data can be processed by Nbit*Nbit data processing device, avoiding increase of area of multiplier. In addition, by separately accumulating products of high bit part of each group of multipliers and multiplicand, expansion of adder caused by directly adding products of low bit part of each group of multipliers and multiplicand after shifting is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 It is an Nbit*Nbit convolution processor;

[0040] Figure 2 It is an Nbit*2Nbit and 2Nbit*2Nbit processor composed of Nbit*Nbit multiplier;

[0041] Figure 3 It is a schematic diagram of convolution processing principle of CNN;

[0042] Figure 4 It is an Nbit*2Nbit convolution processing scheme provided by the embodiment of the application;

[0043] Figure 5 It is an Nbit*2Nbit convolution processing scheme provided by the embodiment of the application;

[0044] Figure 6 It is a 2Nbit*2Nbit convolution processing scheme provided by the embodiment of the application;

[0045] Figure 7 It is a computing flow schematic diagram of the scheme provided by the embodiment of the application applied in a product;

[0046] Figure 8 It is a 2Nbit*Nbit splitting mode schematic diagram provided by the embodiment of the application;

[0047] Figure 9 It is a 2Nbit*2Nbit splitting mode schematic diagram provided by the embodiment of the application;

[0048] Figure 10 It is another computing flow schematic diagram of the scheme provided by the embodiment of the application applied in a product;

[0049] Figure 11 It is another computing flow schematic diagram of the scheme provided by the embodiment of the application applied in a product;

[0050] Figure 12Another computing flow diagram in which the scheme provided by the embodiments of the present application is applied in a product is shown in FIG. 6.

[0051] Figure 13 A flow diagram of a data processing method provided by the embodiments of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0052] The technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0053] For the convenience of understanding, first, the technical scheme provided by the embodiments of the present application will be described with reference to the drawings in the embodiments of the present application. Figure 3 The process of convolution calculation is briefly introduced. Convolution operation is a process of weighted summation, for example, each element in the used image region is multiplied with each element in the convolution kernel respectively, and the sum of all products is taken as the new value of the center pixel of the region. The convolution kernel is a matrix with fixed size and composed of numerical parameters. In the process of convolution operation, the convolution kernel is moved pixel by pixel along the row direction, and the value of the center pixel of the region is updated according to the multiplication result of the element in the region and the element in the convolution kernel. Figure 3 In the convolution neural network, for a feature map, the convolution kernel data A1, A2, …, An and the feature layer data w1, w2, …, w3 are processed. Specifically, for each convolution kernel, it is moved pixel by pixel along the row direction from the first pixel of the feature map. When it moves to the end of the row, it is moved one pixel in the column direction, the row direction returns to the starting position, and the above process of moving in the row direction is repeated until all pixels in the feature map are traversed.

[0054] The technical scheme provided in the application can be applied in the field of original image processing, for example, can be applied in the scene of de-noising the original image. Generally, each pixel point of the original image is represented by an integer of 10 bits to 12 bits. If a traditional 8-bit data format is used for quantization processing, the pixel information of the original image will be greatly lost, and the de-noising effect is not ideal. Therefore, when processing the original image, a high-bit floating-point (FP) data format or a high-bit integer (INT) data format is needed for quantization processing. Using the floating-point data format for processing will cause additional exponent processing area and power consumption overhead, and using the INT data format for processing can save this part of the overhead. The application uses an Nbit*2Nbit and 2Nbit*2Nbit processor composed of an Nbit*Nbit multiplier, where N is a positive integer. When the scheme of the application is applied in the field of original image processing, the convolution kernel data can be Nbit, the feature layer data can be 2Nbit, or the convolution kernel data can be 2Nbit and the feature layer data can be Nbit, or both the convolution kernel data and the feature layer data can be 2Nbit. It should be noted that the scheme provided in the application can not only be applied in the field of original image processing, but also how the Nbit*Nbit multiplier constitutes the Nbit*2Nbit and 2Nbit*2Nbit processor will be described below.

[0055] As shown in Figure 4 , an Nbit*2Nbit convolution processing scheme provided by an embodiment of the application. In this scheme, the 2Nbit data is split into a high bit part and a low bit part, the high bit part is the high N bits or the first N bits of the 2Nbit data, and the low bit part is the low N bits or the last N bits of the 2Nbit data. For example, assuming that N is 8, 2Nbit is 16 bits, such as FF1A, then the high bit part is FF, and the low bit part is 1A. If it is 32-bit data, such as 3F68415B, the high bit part is 3F68, and the low bit part is 415B. The way how the 2Nbit data is split into a high bit part and a low bit part in the prior art can be used in the embodiment of the application, such as splitting the high and low bits of the data by a selector, selecting the high N bits of the output data A to the multiplier 401, or selecting the low N bits of the output data A to the multiplier 403. When there are two or more groups of data for multiplication, the product results that need to be left shifted by the same number of bits in the partial products are merged, which will be described below.

[0056] Suppose we have two sets of data undergoing a convolution operation: A×B+C×D, where A and C are 2N bits, and B and D are N bits. Hereinafter, "A-high" represents the high-order bits of data A (the high N bits), and "C-low" represents the low-order bits of data C (the low N bits). A×B+C×D=[(A-high×B+C-high×D)<<N]+A-low×B+C-low×D. For example... Figure 4 As shown, multipliers 401, 402, 403, and 404 are all N-bit × N-bit multipliers. Multiplier 401 can be used to calculate A - high × B, multiplier 402 can be used to calculate C - high × D, multiplier 403 can be used to calculate A - low × B, and multiplier 404 can be used to calculate C - low × D. An adder 405 with an input width of 2N-bit can be used to accumulate the results of multipliers 401 and 402 to obtain the first accumulated product. An adder 406 with an input width of 2N-bit can be used to accumulate the results of multipliers 403 and 404 to obtain the second accumulated product. The solution provided in this application merges partial products that require the same number of left shifts. For example, in the case of A×B+C×D listed above, the product of A-high×B and C-high×D both need to be shifted left by N bits, so these partial products are merged, i.e., accumulated using adder 405. The product of A-low×B and C-low×D does not need to be shifted left, i.e., shifted left by 0 bits, so these two partial products are merged, i.e., accumulated using adder 406. Shifter 407 shifts the output of adder 405, specifically shifting it left by N bits. Adder 408 accumulates the outputs of shifter 407 and adder 406 to output the final result. Adder 408 has a bit width of 2N bits.

[0057] It should be noted that the convolution operation of the two sets of data listed above does not mean that the technical solution provided in this application is only applicable to the convolution operation of two sets of data. This application does not limit the number of data involved in the convolution operation, and this will not be repeated below. To better understand this solution, the following uses four sets of data as an example to explain how an Nbit × Nbit multiplier is constructed into an Nbit * 2Nbit multiplier. Figure 5As shown, the Nbit*2Nbit convolution processing scheme provided by the embodiment of the present application is shown. Assuming that there are four groups of data convolution operations: A*B+C*D+E*F+G*H. Among them, A, C, E, G are 2Nbit, or 2N bits, B, D, F, H are Nbit, or N bits. Hereinafter, "A-high" represents the high bit part of data A, that is, the high N bits of A data, and "A-low" represents the low bit part of A data, that is, the low N bits of A data. "C-high" represents the high bit part of data C, and "C-low" represents the low bit part of C data, that is, the low N bits of C data. "E-high" represents the high bit part of data E, that is, the high N bits of E data, and "E-low" represents the low bit part of E data, that is, the low N bits of E data. "G-high" represents the high bit part of data G, that is, the high N bits of G data, and "G-low" represents the low bit part of G data, that is, the low N bits of E data. A*B+C*D+E*F+G*H = [(A-high*B+C-high*D+E-high*F+G-high*H) « N] + A-low*B+C-low*D+E-low*F+G-low*H. In this example, A-high*B, C-high*D, E-high*F, and G-high*H can be calculated by multipliers 501 to 504, respectively, and A-low*B, C-low*D, E-low*F, and G-low*H can be calculated by multipliers 505 to 508, respectively. The product results of A-high*B, C-high*D, E-high*F, and G-high*H all need to be left shifted by N bits, so their product results are combined, and specifically, as shown in FIG. 5, the output results of the multipliers 501 and 502 can be accumulated by an adder 509, the output results of the multipliers 503 and 504 can be accumulated by an adder 510, and the output results of the adder 509 and the adder 510 can be accumulated by an adder 513. Among them, the bit width of the adder 509, the adder 510, and the adder 513 is 2Nbit. Figure 5 As shown, the Nbit*2Nbit convolution processing scheme provided by the embodiment of the present application is shown. Assuming that there are four groups of data convolution operations: A*B+C*D+E*F+G*H. Among them, A, C, E, G are 2Nbit, or 2N bits, B, D, F, H are Nbit, or N bits. Hereinafter, "A-high" represents the high bit part of data A, that is, the high N bits of A data, and "A-low" represents the low bit part of A data, that is, the low N bits of A data. "C-high" represents the high bit part of data C, and "C-low" represents the low bit part of C data, that is, the low N bits of C data. "E-high" represents the high bit part of data E, that is, the high N bits of E data, and "E-low" represents the low bit part of E data, that is, the low N bits of E data. "G-high" represents the high bit part of data G, that is, the high N bits of G data, and "G-low" represents the low bit part of G data, that is, the low N bits of E data. A*B+C*D+E*F+G*H = [(A-high*B+C-high*D+E-high*F+G-high*H) « N] + A-low*B+C-low*D+E-low*F+G-low*H. In this example, A-high*B, C-high*D, E-high*F, and G-high*H can be calculated by multipliers 501 to 504, respectively, and A-low*B, C-low*D, E-low*F, and G-low*H can be calculated by multipliers 505 to 508, respectively. The product results of A-high*B, C-high*D, E-high*F, and G-high*H all need to be left shifted by N bits, so their product results are combined, and specifically, as shown in FIG. 5, the output results of the multipliers 501 and 502 can be accumulated by an adder 509, the output results of the multipliers 503 and 504 can be accumulated by an adder 510, and the output results of the adder 509 and the adder 510 can be accumulated by an adder 513. Among them, the bit width of the adder 509, the adder 510, and the adder 513 is 2Nbit. Figure 5 As shown, the Nbit*2Nbit convolution processing scheme provided by the embodiment of the present application is shown. Assuming that there are four groups of data convolution operations: A*B+C*D+E*F+G*H. Among them, A, C, E, G are 2Nbit, or 2N bits, B, D, F, H are Nbit, or N bits. Hereinafter, "A-high" represents the high bit part of data A, that is, the high N bits of A data, and "A-low" represents the low bit part of A data, that is, the low N bits of A data. "C-high" represents the high bit part of data C, and "C-low" represents the low bit part of C data, that is, the low N bits of C data. "E-high" represents the high bit part of data E, that is, the high N bits of E data, and "E-low" represents the low bit part of E data, that is, the low N bits of E data. "G-high" represents the high bit part of data G, that is, the high N bits of G data, and "G-low" represents the low bit part of G data, that is, the low N bits of E data. A*B+C*D+E*F+G*H = [(A-high*B+C-high*D+E-high*F+G-high*H) « N] + A-low*B+C-low*D+E-low*F+G-low*H. In this example, A-high*B, C-high*D, E-high*F, and G-high*H can be calculated by multipliers 501 to 504, respectively, and A-low*B, C-low*D, E-low*F, and G-low*H can be calculated by multipliers 505 to 508, respectively. The product results of A-high*B, C-high*D, E-high*F, and G-high*H all need to be left shifted by N bits, so their product results are combined, and specifically, as shown in FIG. 5, the output results of the multipliers 501 and 502 can be accumulated by an adder 509, the output results of the multipliers 503 and 504 can be accumulated by an adder 510, and the output results of the adder 509 and the adder 510 can be accumulated by an adder 513. Among them, the bit width of the adder 509, the adder 510, and the adder 513 is 2Nbit.

[0058] Assuming that A, C, E, G are regarded as multipliers, and B, D, F, H are regarded as multiplicands, the scheme is to separately accumulate the product of the high bit part of the multipliers and the multiplicand and the product of the low bit part of the multipliers and the multiplicand, shift the accumulation result of the product of the high bit part of the multipliers and the multiplicand as a whole, and add the accumulation result of the product of the low bit part of the multipliers and the multiplicand to form the final result.

[0059] The technical scheme provided in the application does not need to directly expand the Nbit*Nbit multiplier, thereby avoiding the increase of the area of the multiplier. In addition, by separately accumulating the product of the high bit part of the multipliers and the multiplicand and then performing the shift processing, the expansion of the adder caused by directly adding the product of the high bit part of each group of multipliers and the multiplicand after the shift to the product of the low bit part of the corresponding multiplier and the multiplicand is avoided. For example, in some schemes, A*B+C*D=[(A-high*B)<<N+A-low*B]+[(C-high*D)<<N+C-low*D], a 3Nbit adder is needed to calculate the sum of A-high*B and A-low*B and the sum of C-high*D and C-low*D. However, as shown in the scheme, the adder for outputting the final result only needs to be 3Nbit, and other adders in the scheme can be 2Nbit. Compared with the scheme in which the product of the high bit part of each group of multipliers and the multiplicand is directly added to the product of the low bit part of the corresponding multiplier and the multiplicand after the shift, the scheme does not need to perform the shift processing multiple times, thereby saving the logic resources. Figure 5

[0060] As shown in Figure 6 , it is a 2Nbit*2Nbit convolution processing scheme provided by the embodiment of the application. In the scheme, the 2Nbit data is split into high bit part and low bit part. The explanation of the high bit part and the low bit part can be referred to Figure 4 ​The description in the foregoing is understood and will not be repeated here. When there are two groups or more than two groups of data for multiplication, the partial products that need to be left shifted by the same number of bits are combined. Specifically, the products of the high bit part and the high bit part, the high bit part and the low bit part, the low bit part and the high bit part, and the low bit part and the low bit part are respectively combined by polynomials, the products after the combination are respectively accumulated, and the accumulated results are shifted as a whole and then summed to obtain the final result. The following will be illustrated by an example. Assume that there are two groups of data convolution operations: AxC+ExG, wherein A, C, E, and G are all 2Nbit. Herein, "A-high" represents the high bit part of the data A, i.e., the high N bits of the data A, "A-low" represents the low bit part of the data A, i.e., the low N bits of the data A. "C-high" represents the high bit part of the data C, "C-low" represents the low bit part of the data C, i.e., the low N bits of the data C. "E-high" represents the high bit part of the data E, i.e., the high N bits of the data E, "E-low" represents the low bit part of the data E, i.e., the low N bits of the data E. "G-high" represents the high bit part of the data G, i.e., the high N bits of the data G, and "G-low" represents the low bit part of the data G, i.e., the low N bits of the data G. AxC+ExG = [(A-highxC-high+E-highxG-high) << 2N] + [(A-highxC-low+E-highxG-low) << N] + [(A-lowxC-high+E-lowxG-high) << N] + (A-lowxC-low+E-lowxG-low). For example, Figure 6As shown, each of the multipliers 601-608 is an NbitxNbit multiplier. The high-bit portion and the high-bit portion product, such as A-highx C-high and E-highx G-high, can be calculated by the multipliers 601 and 602. The high-bit portion and the low-bit portion product, or the low-bit portion and the high-bit portion product, such as A-highx C-low and E-highx G-low, or A-lowx C-high and E-lowx G-high, can be calculated by the multipliers 603 and 604. If the high-bit portion and the low-bit portion product is calculated by the multipliers 603 and 604, the low-bit portion and the high-bit portion product is calculated by the multipliers 605 and 606, and if the low-bit portion and the high-bit portion product is calculated by the multipliers 603 and 604, the high-bit portion and the low-bit portion product is calculated by the multipliers 605 and 606, wherein the high-bit portion and the high-bit portion product refers to A-highx C-low and E-highx G-low, and the low-bit portion and the high-bit portion product refers to A-lowx C-high and E-lowx G-high. The low-bit portion and the low-bit portion product, such as A-lowx C-low and E-lowx G-low, can be calculated by the multipliers 607 and 608. The output results of the multipliers 601 and 602 are accumulated by the adder 609, the output results of the multipliers 603 and 604 are accumulated by the adder 610, the output results of the multipliers 605 and 606 are accumulated by the adder 611, and the output results of the multipliers 607 and 608 are accumulated by the adder 612. The bit width of the adders 609, 610, 611 and 612 is 2Nbit. The high-bit portion and the high-bit portion product (hereinafter referred to as high-bit product) needs to be left shifted by 2Nbit, the high-bit portion and the low-bit portion product (hereinafter referred to as high-low-bit product) and the low-bit portion and the high-bit portion product (hereinafter referred to as low-high-bit product) need to be left shifted by Nbit, and the low-bit portion and the low-bit portion product (hereinafter referred to as low-bit product) does not need to be left shifted, i.e. left shifted by 0bit. In the implementation, the output result of the adder 609 can be left shifted by Nbit by the shifter 613, and the data output by the shifter 613 is 3Nbit, which is the first shift of the high-bit product. The output results of the shifter 613 and the adder 610 are accumulated by the adder 615, and the bit width of the adder 615 is 3Nbit. The output result of the adder 615 is left shifted by Nbit by the shifter 617, and the data output by the shifter 617 is 4Nbit, at this time, the high-bit product has been shifted by 2Nbit.The shifter 614 shifts the output result of the adder 611 left by N bits, the adder 616 accumulates the output results of the shifter 614 and the adder 612, and the bit width of the adder 616 is 3N bits. The adder 618 accumulates the output results of the shifter 617 and the adder 616, and the bit width of the adder 618 is 4N bits.

[0061] The technical scheme provided in the application separately accumulates the products of the high bit part of each group of multipliers and the high bit part of the multiplicand, the products of the high bit part of each group of multipliers and the low bit part of the multiplicand, the products of the low bit part of each group of multipliers and the high bit part of the multiplicand, and the products of the low bit part of each group of multipliers and the low bit part of the multiplicand, and then performs corresponding bit shift and addition operations on the four accumulated results to obtain the final result. The scheme provided in the application avoids expanding the bit of the adder due to separately performing the shift processing on the product of the high bit part of each group of multipliers and the high bit part of the multiplicand and the product of the high bit part and the low bit part, such as AxC+E x G = [(A-high x C-high) << 2N] + [(A-high x C-low+) << N] + [(A-low x C-high << N] + (A-low x C-low) + [(E-high x G-high) << 2N] + [(E-high x G-low+) << N] + [(E-low x G-high << N] + (E-low x G-low). This scheme needs multiple times of shift, and the more data participating in the convolution operation, the more times of shift are needed. In addition, a large number of adders with a bit width of 3N bits and adders with a bit width of 4N bits are needed in this scheme, and the present scheme combines the partial products with the same left shift number, separately performs accumulation operations, and greatly saves the logic resources.

[0062] In a specific embodiment of the present application, by controlling the opening and closing of the shifters, 4 groups of Nbit x Nbit operations, 2 groups of 2Nbit x Nbit operations, and 1 group of 2Nbit x 2Nbit operation can be processed in one clock. The following will be described in combination with Figure 6 The specific description is as follows. The opening and closing of the shifter 613, the shifter 614, and the shifter 617 can be controlled by a state machine, and specifically, a user can input instructions to control the opening and closing of the shifter 613, the shifter 614, and the shifter 617 by the state machine. In a specific embodiment, the shifter 613, the shifter 614, and the shifter 617 can be controlled to be in the opening state, and at this time, 2 groups of 2Nbit x 2Nbit operations can be processed, as described above. Figure 6The description in the document is as follows. In one specific embodiment, shifter 613 and shifter 614 can be turned on, while shifter 617 can be turned off, allowing for the processing of 4 groups of 2Nbit×Nbit operations. In another specific embodiment, shifters 613, 614, and 617 can all be turned off, allowing for the processing of 8 groups of Nbit×Nbit operations.

[0063] The above explains how to perform calculations based on feature layer data and convolution kernel data. In specific application scenarios, the above scheme can be implemented using any convolution operation device, such as multipliers, central processing units (CPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), graphics processing units (GPUs), or other artificial intelligence (AI) chips, etc.

[0064] The following describes the application scenarios of the solution provided in this application to specific products, and explains the calculation process involved in this application. Specific products can refer to any of the convolution operation devices mentioned above. For example... Figure 7 As shown, the double data rate (DDR) controller 702 reads data from DDR 701, which includes feature layer data and convolution kernel data. The DDR controller 702 sends the read data to the splitting logic circuit 703, which splits the 2N-bit feature layer data into high-order and low-order parts and stores the split data in the random access memory (RAM) 705. The splitting logic circuit 703 also splits the 2N-bit convolution kernel data into high-order and low-order parts and stores the split data in the weight RAM 704. The calculation circuit 706 retrieves the feature layer data from the data RAM and performs calculations with the pre-loaded convolution kernel data in the calculation circuit 706. The specific calculation process can be found in [reference needed]. Figure 4 to Figure 6 The following description is provided for understanding. After the calculation circuit 706 completes the calculation, it writes the result to DDR 701 through the DDR controller 702, completing the entire process. In a specific embodiment, a state machine (not shown in the figure) may also be included to control the switching off of the shifters in the calculation circuit. The specific principle has been described in detail above and will not be repeated here.

[0065] The following example uses a multiplier and multiplicand both being 2N bits, or in other words, the convolution kernel data and feature layer data both being 2N bits. Figure 7 The data calculation process in the product structure shown is explained. The DDR controller reads data from the DDR and sends the read data to the splitting logic circuit. The splitting logic circuit splits the acquired data and establishes corresponding relationships. For example, ... Figure 8 As shown, a 2Nbit × Nbit splitting method is given. Assume there are four sets of data to be multiplied: A × B + C × D + E × F + G × H, where A, C, E, and G are 2N bits, and B, D, F, and H are N bits. A × B + C × D + E × F + G × H = [(A-high × B + C-high × D + E-high × F + G-high × H) << N] + A-low × B + C-low × D + E-low × F + G-low × H. The splitting logic circuit splits the data A into A-high and A-low, and then... High and A-low are respectively associated with data B. Data C is split into C-high and C-low, and C-high and C-low are respectively associated with data D. Data E is split into E-high and E-low, and E-high and E-low are respectively associated with data F. Data G is split into G-high and G-low, and G-high and G-low are respectively associated with data H. It should be noted that the embodiments of this application do not limit the number of data involved in the calculation. In actual application scenarios, the data involved in the calculation can be two or more groups. For example... Figure 9 As shown, a 2N-bit × 2N-bit splitting method is given, A × C + E × G, where A, C, E, and G are all 2N bits. A × C + E × G = [(A-high × C-high + E-high × G-high) << 2N] + [(A-high × C-low + E-high × G-low) << N] + [(A-low × C-high + E-low × G-high) << N] + (A-low × C-low + E-low × G-low). The splitting logic circuit will then split the data A into A-high and A-... The data C is split into C-high and C-low; data E is split into E-high and E-low; data G is split into G-high and G-low. A correspondence is established between A-high and C-high, G-high and E-high, A-high and C-low, G-high and E-low, A-low and C-high, G-low and E-high, A-low and C-low, and G-low and E-low. The splitting logic circuit stores the split feature layer data into data RAM, and stores the split convolution kernel data into weight RAM according to the established correspondences. Figure 10As shown, taking 2Nbit × 2Nbit as an example, a schematic diagram of a splitting logic circuit is given, which splits 2Nbit data into two parts, a high-order part and a low-order part, and stores them in data RAM and parameter RAM. Figure 11 As shown, data in the weight RAM is preloaded into the computing circuit. Specifically, the first data segment is preloaded into computing circuit 1, the second data segment into computing circuit 2, ..., and the nth data segment into computing circuit n. Figure 12 As shown, the first segment of data is extracted from the data RAM and calculated together with the first segment of data preloaded in the computing circuit 1 to obtain the result. The specific calculation process can be found in [reference needed]. Figure 6 The description in the text is for reference only and will not be repeated here. After computation circuit 1 completes the calculation of data segment I and data segment 1, it forwards data segment I to computation circuit 2, retrieves data segment II from data RAM, and performs calculations on data segment II and the pre-loaded data segment 1 in computation circuit 1 to obtain the result. Subsequently, each clock cycle, computation circuit 1 retrieves new data from data RAM, and computation circuits 2 through n forward the feature layer data processed in the previous clock cycle to the next computation circuit. When all data stored in data RAM has been processed, computation circuits 1 through n output data, which is then stored in DDR via the DDR controller.

[0066] The data processing apparatus and the apparatus including the data processing apparatus provided in the embodiments of this application have been described above. The data processing method provided in the embodiments of this application will be described below.

[0067] like Figure 13 The diagram shown is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 13 As shown in the embodiments of this application, a data processing method may include the following steps:

[0068] 1301. Calculate the first product and the second product. The first product may include the product of the high N bits of the first multiplier and the first multiplicand, and the product of the high N bits of the second multiplier and the second multiplicand. The second product may include the product of the low N bits of the first multiplier and the first multiplicand, and the product of the low N bits of the second multiplier and the second multiplicand. Both the first and second multipliers are 2N bits, where N is a positive integer.

[0069] 1302. Accumulate the first product and the second product respectively.

[0070] The technical scheme provided in the application can, when two groups or more groups of data are subjected to convolution operation, separately accumulate the products of the high-bit part of each group of multipliers and the multiplicand, and then perform shift processing, thereby avoiding the expansion of the adder caused by directly adding the products of the low-bit part of each group of multipliers and the multiplicand after the products of the high-bit part of each group of multipliers and the multiplicand are shifted.

[0071] In one specific implementation, the result of the accumulation of the first group of products is subjected to shift processing to obtain a first shift result. The first shift result and the second group of products are accumulated. The application can process 2Nbit*2Nbit calculation through an Nbit*Nbit multiplier.

[0072] In one specific implementation, the calculation of the first group of products and the second group of products can specifically include: calculating high-bit products, which can include the product of the high N bits of the first multiplier and the high N bits of the first multiplicand, and the product of the high N bits of the second multiplier and the high N bits of the second multiplicand; calculating high-low-bit products, which can include the product of the high N bits of the first multiplier and the low N bits of the first multiplicand, and the product of the high N bits of the second multiplier and the low N bits of the second multiplicand; calculating low-high-bit products, which can include the product of the low N bits of the first multiplier and the high N bits of the first multiplicand, and the product of the low N bits of the second multiplier and the high N bits of the second multiplicand; and calculating low-bit products, which can include the product of the low N bits of the first multiplier and the low N bits of the first multiplicand, and the product of the low N bits of the second multiplier and the low N bits of the second multiplicand. The accumulation of the first group of products and the second group of products can include: separately accumulating the high-bit products, the high-low-bit products, the low-high-bit products, and the low-bit products. The scheme provided in the application can process 2Nbit*2Nbit calculation through an Nbit*Nbit multiplier.

[0073] In one specific implementation, the scheme can further include: left-shifting the result of the accumulation of the high-bit products by N bits to obtain a second shift result. The second shift result and the result of the accumulation of the high-low-bit products are accumulated. The result obtained by accumulating the second shift result and the result of the accumulation of the high-low-bit products is left-shifted by N bits to obtain a third shift result. The result of the accumulation of the low-high-bit products is left-shifted by N bits to obtain a fourth shift result. The third shift result, the fourth shift result, and the low-bit products are accumulated.

[0074] In one specific implementation, the scheme can further include: accumulating the fourth shift result and the low-bit products. The accumulation of the third shift result, the fourth shift result, and the low-bit products can include: accumulating the result obtained by accumulating the third shift result and the fourth shift result and the low-bit products.

[0075] In one specific implementation, the method further includes outputting the high N bits and the low N bits of the first multiplier, and the high N bits and the low N bits of the second multiplier.

[0076] In one specific implementation, the method further includes constructing a first association relationship, which can include an association relationship between the high N bits of the first multiplier and the first multiplicand, an association relationship between the low N bits of the first multiplier and the first multiplicand, an association relationship between the high N bits of the second multiplier and the second multiplicand, and an association relationship between the low N bits of the second multiplier and the second multiplicand.

[0077] In one specific implementation, the method further includes outputting the high N bits and the low N bits of the first multiplier, the high N bits and the low N bits of the second multiplier, the high N bits and the low N bits of the first multiplicand, and the high N bits and the low N bits of the second multiplicand.

[0078] In one specific implementation, the method further includes constructing a second association relationship, which can include an association relationship between the high N bits of the first multiplier and the high N bits of the first multiplicand, an association relationship between the high N bits of the first multiplier and the low N bits of the first multiplicand, an association relationship between the low N bits of the first multiplier and the high N bits of the first multiplicand, an association relationship between the low N bits of the first multiplier and the low N bits of the first multiplicand, an association relationship between the high N bits of the second multiplier and the high N bits of the second multiplicand, an association relationship between the high N bits of the second multiplier and the low N bits of the second multiplicand, an association relationship between the low N bits of the second multiplier and the high N bits of the second multiplicand, and an association relationship between the low N bits of the second multiplier and the low N bits of the second multiplicand.

[0079] In one specific implementation, the first multiplier and the second multiplier are feature layer data, and the first multiplicand and the second multiplicand are convolution kernel data, or the first multiplier and the second multiplier are convolution kernel data, and the first multiplicand and the second multiplicand are feature layer data. When the scheme of the present application is applied in the field of original image processing, the convolution kernel data can be Nbit, and the feature layer data can be 2Nbit, or the convolution kernel data is 2Nbit, and the feature layer data is Nbit, or both the convolution kernel data and the feature layer data are 2Nbit. It should be noted that the scheme provided by the present application can be applied not only in the field of original image processing.

[0080] The data processing apparatus and the data processing method provided by the embodiments of the present application are described in detail above, and the principle and implementation manner of the present application are described by applying specific examples in the present document; the above description of the embodiments is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, the specific implementation manner and application range can be changed according to the idea of the present application; in conclusion, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A data processing apparatus, characterized in that, include: A product calculation circuit is used to calculate a first set of products and a second set of products. The first set of products includes the product of the high N bits of the first multiplier and the first multiplicand, and the product of the high N bits of the second multiplier and the second multiplicand. The second set of products includes the product of the low N bits of the first multiplier and the first multiplicand, and the product of the low N bits of the second multiplier and the second multiplicand. Both the first multiplier and the second multiplier are 2N bits, where N is a positive integer. An accumulation circuit is used to accumulate the first set of products and the second set of products respectively; The first shifter is used to shift the result of the first group of multiply-accumulated data to obtain the first shift result; The first adder is used to accumulate the first shift result and the second set of products.

2. The data processing apparatus according to claim 1, characterized in that, Both the first multiplicand and the second multiplicand are N bits.

3. The data processing apparatus according to claim 1, characterized in that, The first and second multiplicands are both 2N bits. The first product group includes a high-order product and a high-low-order product, and the second product group includes a low-high-order product and a low-order product. The product calculation circuit is specifically used for: Calculate the high-order product, which includes the product of the high N bits of the first multiplier and the high N bits of the first multiplicand, and the product of the high N bits of the second multiplier and the high N bits of the second multiplicand; Calculate the high-low bit product, which includes the product of the high N bits of the first multiplier and the low N bits of the first multiplicand, and the product of the high N bits of the second multiplier and the low N bits of the second multiplicand. Calculate the low-high bit product, which includes the product of the low N bits of the first multiplier and the high N bits of the first multiplicand, and the product of the low N bits of the second multiplier and the high N bits of the second multiplicand. Calculate the low-order product, which includes the product of the low N bits of the first multiplier and the low N bits of the first multiplicand, and the product of the low N bits of the second multiplier and the low N bits of the second multiplicand. The accumulation circuit is specifically used to accumulate the high-order product, the high-low-order product, the low-high-order product, and the low-order product through a second adder with a bit width of 2Nbit.

4. The data processing apparatus according to claim 3, characterized in that, The data processing device further includes a second shifter, a third adder, a third shifter, a fourth shifter, and a fourth adder. The second shifter is used to shift the result of the high-order multiplication accumulation to the left by N bits to obtain the second shift result; The third adder is used to accumulate the second shift result and the result of the high-low bit multiplication accumulation; The third shifter is used to shift the output of the third adder to the left by N bits to obtain the third shift result; The fourth shifter is used to left-shift the result of the multiplication and accumulation of the low and high bits by N bits to obtain the fourth shift result; The fourth adder is used to accumulate the third shift result, the fourth shift result, and the low-order product.

5. The data processing apparatus according to claim 4, characterized in that, The data processing device further includes a fifth adder, used for: The fourth shift result and the product of the least significant bits are accumulated; The fourth adder is specifically used to accumulate the third shift result and the result output by the fifth adder.

6. The data processing apparatus according to claim 1 or 2, characterized in that, It also includes split logic circuitry, used for: The selector MUX outputs the high N bits and low N bits of the first multiplier, and the high N bits and low N bits of the second multiplier.

7. The data processing apparatus according to claim 6, characterized in that, The splitting logic circuit is also used for: Construct a first association relationship, which includes the association relationship between the high N bits of the first multiplier and the first multiplicand, the association relationship between the low N bits of the first multiplier and the first multiplicand, the association relationship between the high N bits of the second multiplier and the second multiplicand, and the association relationship between the low N bits of the second multiplier and the second multiplicand.

8. The data processing apparatus according to any one of claims 3 to 5, characterized in that, It also includes split logic circuitry, used for: The selector MUX outputs the high N bits and low N bits of the first multiplier, the high N bits and low N bits of the second multiplier, the high N bits and low N bits of the first multiplicand, and the high N bits and low N bits of the second multiplicand.

9. The data processing apparatus according to claim 8, characterized in that, The splitting logic circuit is also used for: Construct a second association relationship, which includes the association relationship between the high N bits of the first multiplier and the high N bits of the first multiplicand, the association relationship between the high N bits of the first multiplier and the low N bits of the first multiplicand, the association relationship between the low N bits of the first multiplier and the high N bits of the first multiplicand, the association relationship between the low N bits of the first multiplier and the low N bits of the first multiplicand, the association relationship between the high N bits of the second multiplier and the low N bits of the second multiplicand, the association relationship between the low N bits of the second multiplier and the high N bits of the second multiplicand, and the association relationship between the low N bits of the second multiplier and the low N bits of the second multiplicand.

10. The data processing apparatus according to claim 9, characterized in that, The data processing device further includes a random access memory (RAM) and a weighted RAM. The data RAM is used to store the high N bits and low N bits of the first multiplier, and the high N bits and low N bits of the second multiplier; The weight RAM is used to store the high N bits and low N bits of the first multiplicand, and the high N bits and low N bits of the second multiplicand according to the second association relationship.

11. The data processing apparatus according to any one of claims 1 to 5, characterized in that, The first multiplier and the second multiplier are feature layer data, and the first multiplicand and the second multiplicand are convolution kernel data, or the first multiplier and the second multiplier are convolution kernel data, and the first multiplicand and the second multiplicand are feature layer data.

12. A data processing method, characterized in that, include: Calculate the first set of products and the second set of products. The first set of products includes the product of the high N bits of the first multiplier and the first multiplicand, and the product of the high N bits of the second multiplier and the second multiplicand. The second set of products includes the product of the low N bits of the first multiplier and the first multiplicand, and the product of the low N bits of the second multiplier and the second multiplicand. Both the first multiplier and the second multiplier are 2N bits, where N is a positive integer. The first set of products and the second set of products are summed separately. The result of the first multiplication and accumulation is shifted to obtain the first shifted result; The first shift result and the second set of products are accumulated.

13. The data processing method according to claim 12, characterized in that, The calculation of the first group product and the second group product specifically includes: Calculate the high-order product, which includes the product of the high N bits of the first multiplier and the high N bits of the first multiplicand, and the product of the high N bits of the second multiplier and the high N bits of the second multiplicand; Calculate the high-low bit product, which includes the product of the high N bits of the first multiplier and the low N bits of the first multiplicand, and the product of the high N bits of the second multiplier and the low N bits of the second multiplicand. Calculate the low-high bit product, which includes the product of the low N bits of the first multiplier and the high N bits of the first multiplicand, and the product of the low N bits of the second multiplier and the high N bits of the second multiplicand. Calculate the low-order product, which includes the product of the low N bits of the first multiplier and the low N bits of the first multiplicand, and the product of the low N bits of the second multiplier and the low N bits of the second multiplicand. The step of accumulating the first set of products and the second set of products includes: The high-order product, the high-low-order product, the low-high-order product, and the low-order product are summed respectively.

14. The data processing method according to claim 13, characterized in that, Also includes: The result of the high-order multiplication accumulation is shifted left by N bits to obtain the second shifted result; The second shift result and the result of the multiplication and summation of the high and low bits are accumulated; The result of accumulating the second shift result and the result of multiplying and accumulating the high and low bits is shifted left by N bits to obtain the third shift result; The result of the accumulation of the low and high bits is shifted left by N bits to obtain the fourth shift result; The third shift result, the fourth shift result, and the low-order product are accumulated.

15. The data processing method according to claim 14, characterized in that, Also includes: The fourth shift result and the product of the least significant bits are accumulated; The accumulation of the third shift result, the fourth shift result, and the low-order product includes: The results of accumulating the third shift result, the fourth shift result, and the product of the least significant bits are then accumulated.

16. The data processing method according to claim 12, characterized in that, Also includes: Output the high N bits and low N bits of the first multiplier, and the high N bits and low N bits of the second multiplier.

17. The data processing method according to claim 16, characterized in that, Also includes: Construct a first association relationship, which includes the association relationship between the high N bits of the first multiplier and the first multiplicand, the association relationship between the low N bits of the first multiplier and the first multiplicand, the association relationship between the high N bits of the second multiplier and the second multiplicand, and the association relationship between the low N bits of the second multiplier and the second multiplicand.

18. The data processing method according to any one of claims 13 to 15, characterized in that, Also includes: Output the high N and low N bits of the first multiplier, the high N and low N bits of the second multiplier, the high N and low N bits of the first multiplicand, and the high N and low N bits of the second multiplicand.

19. The data processing method according to claim 18, characterized in that, Also includes: Construct a second association relationship, which includes the association relationship between the high N bits of the first multiplier and the high N bits of the first multiplicand, the association relationship between the high N bits of the first multiplier and the low N bits of the first multiplicand, the association relationship between the low N bits of the first multiplier and the high N bits of the first multiplicand, the association relationship between the low N bits of the first multiplier and the low N bits of the first multiplicand, the association relationship between the high N bits of the second multiplier and the low N bits of the second multiplicand, the association relationship between the low N bits of the second multiplier and the high N bits of the second multiplicand, and the association relationship between the low N bits of the second multiplier and the low N bits of the second multiplicand.

20. The data processing method according to any one of claims 12 to 17, characterized in that, The first multiplier and the second multiplier are feature layer data, and the first multiplicand and the second multiplicand are convolution kernel data, or the first multiplier and the second multiplier are convolution kernel data, and the first multiplicand and the second multiplicand are feature layer data.

21. A field-programmable gate array (FPGA), characterized in that, The FPGA includes the data processing apparatus described in any one of claims 1 to 11.

Images