An acceleration system and method for deconvolution calculation in neural network

Abstract

The present invention provides an acceleration system and method for deconvolution calculation in a neural network. The present invention uses the proposed specific rule to redistribute the weights stored in the processing unit PE, so that the accelerator supports the deconvolution calculation of the pixel value at a specific position under the same calculation mode in cooperation with the data input module within one calculation cycle, thereby improving the deconvolution calculation speed and the utilization rate of the processing unit PE, reducing the idle rate of hardware resources, and the key is that the data input buffer time of the data input module is also optimized; in addition, the results output by different processing units PE in the same calculation cycle can be effectively fused through the result fusion module, so that the hardware working performance of the accelerator is optimized.

Description

Technical Field

[0001] The present invention relates to the field of hardware acceleration of deconvolution computing tasks in neural networks, and more specifically, to an acceleration system and method for deconvolution computing in neural networks. Background Art

[0002] Deconvolution, also known as transposed convolution, is a kind of inverse operation of convolution in a broad sense. It is a common method of upsampling and is currently a key task in deep neural networks. Compared with conventional convolution calculations, deconvolution calculations produce more additional operations, which leads to performance and energy consumption bottlenecks. Storage-in-one computing (PIM) is a new paradigm for deploying deep neural networks. PIM based on resistive random access memory (ReRAM) is a powerful and effective method that has been widely used in conventional convolution calculation tasks, but has not been widely used in deconvolution calculation tasks due to the high complexity of the calculation.

[0003] In order to further apply ReRAM to the deployment of deep neural networks, previous studies have designed a deconvolution accelerator RED based on ReRAM variable resistive memory, which eliminates redundancy and additional operations to achieve the result of accelerating the deconvolution calculation task. The document "Ziru Li, et al. RED: A ReRAM-Based Efficient Accelerator for Deconvolutional Computation" makes full use of the different computing modes of deconvolution calculation, designs a new input buffer, and accelerates the deconvolution calculation in deep neural networks by providing pixel-level mapping and zero-skipping data stream input. Ideally, the deconvolution in the same computing mode of this method can be executed in parallel in one cycle. However, when the convolution kernel is large, the calculation amount of the same computing mode needs to be divided into two parts, each of which is allocated a computing cycle, which doubles the time required for deconvolution. In addition, since only PEs in the same computing mode are used in each cycle, the utilization rate of PEs in this method is not high.

[0004] The prior art discloses a patent for a neural network convolution calculation acceleration method based on the Kmeans algorithm. The patent adds a Kmeans clustering layer to the neural network convolution, and the input data after passing through the Kmeans clustering layer is clustered into fixed k categories; the multiplication of these k categories of data and the convolution kernel is calculated in the convolution layer, and the result is made into a lookup table. The result of the multiplication of all the original input data and the convolution kernel is obtained through this lookup table. However, the patent does not design a series of rules to redistribute the weights contained in each PE, so that the deconvolution calculation of the same calculation mode can still be executed in parallel within one cycle under any circumstances, and the calculation speed of the deconvolution is greatly improved; at the same time, the deconvolution calculation is performed by the redistributed PE, and the buffer time of the data input is optimized; and the calculation results of different processing units PE are well integrated through the result fusion module. Summary of the invention

[0005] The present invention provides an acceleration system for deconvolution calculation in a neural network, which solves the problem of low utilization rate of processing units PE and long deconvolution calculation time in deconvolution accelerator RED calculation based on ReRAM.

[0006] Another object of the present invention is to provide an acceleration method for the above-mentioned acceleration system for deconvolution calculation in a neural network.

[0007] In order to achieve the above technical effects, the technical solution of the present invention is as follows:

[0008] An acceleration system for deconvolution calculation in a neural network, comprising:

[0009] The original feature map module is used to store the original feature map to be deconvolution calculated;

[0010] The data input module is used to transmit the pixel values ​​to be calculated in the feature map to the next module in batches according to the deconvolution calculation mode;

[0011] The deconvolution calculation module is used to redistribute the weights contained in each processing unit PE according to a specific rule, thereby accelerating and completing the deconvolution calculation;

[0012] The result fusion module is used to fuse the deconvolution results of different computing modes on the same output feature map.

[0013] Furthermore, the deconvolution calculation module is composed of a plurality of processing units PE that map weights according to the proposed rule, receives pixel values ​​of specific positions of feature maps under the same calculation mode in batches from the data input module, distributes pixel values ​​at different positions to different pre-set calculation units PE through the data path, and calculates the deconvolution results of all pixel values ​​at specific positions of the same calculation mode in parallel within one calculation cycle.

[0014] Furthermore, the result fusion module is a deconvolution calculation result processing device, which gathers the deconvolution results of the deconvolution calculation module under the same calculation mode into the same adder through the data path for result fusion, and then stores them in the specific position of the output feature map in the order of the calculation mode, wherein the number of adders is equal to the number of types of deconvolution calculation modes, that is, there are S 2 An adder.

[0015] Furthermore, the calculation mode of the deconvolution is:

[0016] Calculation mode 1 only performs deconvolution calculations on the pixel values ​​at positions (0,0), (0,2), (2,0), and (2,2); calculation mode 2 only performs deconvolution calculations on the pixel values ​​at positions (0,1) and (2,1); calculation mode 3 only performs deconvolution calculations on the pixel values ​​at positions (1,0) and (1,2); and calculation mode 4 only performs deconvolution calculations on the pixel values ​​at positions (1,1).

[0017] Furthermore, the specific rule is:

[0018] First, the weight of each position of the convolution kernel is assigned to different PEs respectively, and the PEs in the same calculation mode are arranged in the same row from left to right and from top to bottom according to the position of the convolution kernel, that is, the first row includes PE1(0,0), PE 2(0,2), PE3(2,0), PE 4(2,2), the second row includes PE 5(0,1), PE 6(2,1), the third row includes PE 7(1,0), PE 8(1,2), and the fourth row includes PE 9(1,1); then the PEs in the same row are merged in pairs. If the number of PEs is an odd number, the weight of the last PE in the row remains unchanged, that is, the first row includes PE1(0,0)+(0,2), PE 2(0,2)+(2,2), the second row includes PE 3(0,1)+(2,1), the third row includes PE4(1,0)+(1,2), and the fourth row includes PE 5(1,1); finally, the weights of all PEs are redistributed. The idle PEs in the same column but different rows on the original data path that have not been reallocated should be selected by column. If it does not meet the requirements, the PEs in different columns should be selected or the reallocated PEs should be restored and reallocated. The second half of the PE should be selected first. If PE 1=(0,0)+(0,2), the weight allocation of the position (0,2) should be changed first. If it does not meet the requirements, the weight of the first half of the PE should be selected for reallocation.

[0019] A method for accelerating deconvolution calculation in a neural network, comprising the following steps:

[0020] S1: The original feature map module stores the original feature map to be deconvolved;

[0021] S2: The data input module transmits the pixel values ​​to be calculated in the feature map to the next module in batches according to the deconvolution calculation mode;

[0022] S3: The deconvolution calculation module redistributes the weights contained in each processing unit PE according to a specific rule, thereby accelerating and completing the deconvolution calculation;

[0023] S4: The result fusion module fuses the deconvolution results of different computing modes on the same output feature map.

[0024] Furthermore, the deconvolution calculation module is composed of a plurality of processing units PE that map weights according to the proposed rule, receives pixel values ​​of specific positions of feature maps under the same calculation mode in batches from the data input module, distributes pixel values ​​at different positions to different pre-set calculation units PE through the data path, and calculates the deconvolution results of all pixel values ​​at specific positions of the same calculation mode in parallel within one calculation cycle.

[0025] Furthermore, the result fusion module is a deconvolution calculation result processing device, which gathers the deconvolution results of the deconvolution calculation module under the same calculation mode into the same adder through the data path for result fusion, and then stores them in the specific position of the output feature map in the order of the calculation mode, wherein the number of adders is equal to the number of types of deconvolution calculation modes, that is, there are S 2 An adder.

[0026] Furthermore, the deconvolution calculation module is a set of calculation units PEs that are weighted based on a specific rule, and the deconvolution calculation module is constructed with marginal additional cost. The pixel values ​​to be calculated in batches under the same calculation mode are used as input, and the calculation results are output in a deconvolution manner within one calculation cycle.

[0027] Furthermore, the result fusion module supports the aggregation and fusion of deconvolution results under the same calculation mode onto the output feature map, wherein the number of adders in the module should be equal to the number of deconvolution calculation modes, that is, there are S 2 An adder.

[0028] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0029] The present invention aims at the problem of low utilization rate of processing unit PE and long deconvolution calculation time in the existing ReRAM-based deconvolution accelerator RED calculation. By using the proposed specific rule to redistribute the weights stored in the processing unit PE, the accelerator supports the deconvolution calculation of pixel values ​​at specific positions under the same calculation mode in cooperation with the data input module within one calculation cycle, thereby improving the deconvolution calculation speed and the utilization rate of the processing unit PE, reducing the idle rate of hardware resources, and most importantly, the data input buffer time of the data input module is also optimized. In addition, the results output by different processing units PE in the same calculation cycle can be effectively fused through the result fusion module, so that the hardware working performance of the accelerator is optimized, which is a more reliable and efficient accelerator device and design method. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a system structure block diagram of the present invention;

[0031] Figure 2 Schematic diagram of four calculation modes of deconvolution calculation involved in the present invention;

[0032] Figure 3 A schematic diagram of a mapping method of a convolution kernel to a processing unit PE proposed in the present invention;

[0033] Figure 4 This is a structural block diagram of a deconvolution calculation module for reallocating PE weights according to a specific rule proposed by the present invention;

[0034] Figure 5 This is a structural block diagram of the result fusion module proposed in the present invention. DETAILED DESCRIPTION

[0035] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0036] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0037] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0038] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0039] Example 1

[0040] like Figure 1 As shown, an acceleration system for deconvolution calculation in a neural network includes:

[0041] The original feature map module is used to store the original feature map to be deconvolution calculated;

[0042] The data input module is used to transmit the pixel values ​​to be calculated in the feature map to the next module in batches according to the deconvolution calculation mode;

[0043] The deconvolution calculation module is used to redistribute the weights contained in each processing unit PE according to a specific rule, thereby accelerating and completing the deconvolution calculation;

[0044] The result fusion module is used to fuse the deconvolution results of different computing modes on the same output feature map.

[0045] The deconvolution calculation module is composed of multiple processing units PE that map weights according to the proposed rule. It receives pixel values ​​of specific positions of the feature map under the same calculation mode in batches from the data input module, and distributes the pixel values ​​at different positions to different pre-set calculation units PE through the data path. The deconvolution results of all pixel values ​​at specific positions of the same calculation mode are calculated in parallel within one calculation cycle.

[0046] The result fusion module is a deconvolution calculation result processing device, which gathers the deconvolution results of the same calculation mode of the deconvolution calculation module into the same adder through the data path for result fusion, and then stores them in the specific position of the output feature map in the order of the calculation mode. The number of adders is equal to the number of types of deconvolution calculation modes, that is, there are S 2 An adder.

[0047] The calculation mode of deconvolution is:

[0048] Calculation mode 1 only performs deconvolution calculations on the pixel values ​​at positions (0,0), (0,2), (2,0), and (2,2); calculation mode 2 only performs deconvolution calculations on the pixel values ​​at positions (0,1) and (2,1); calculation mode 3 only performs deconvolution calculations on the pixel values ​​at positions (1,0) and (1,2); and calculation mode 4 only performs deconvolution calculations on the pixel values ​​at positions (1,1).

[0049] The specific rules are:

[0050] First, the weight of each position of the convolution kernel is assigned to different PEs respectively, and the PEs in the same calculation mode are arranged in the same row from left to right and from top to bottom according to the position of the convolution kernel, that is, the first row includes PE1(0,0), PE 2(0,2), PE3(2,0), PE 4(2,2), the second row includes PE 5(0,1), PE 6(2,1), the third row includes PE 7(1,0), PE 8(1,2), and the fourth row includes PE 9(1,1); then the PEs in the same row are merged in pairs. If the number of PEs is an odd number, the weight of the last PE in the row remains unchanged, that is, the first row includes PE1(0,0)+(0,2), PE 2(0,2)+(2,2), the second row includes PE 3(0,1)+(2,1), the third row includes PE4(1,0)+(1,2), and the fourth row includes PE 5(1,1); finally, the weights of all PEs are redistributed. The idle PEs in the same column but different rows on the original data path that have not been reallocated should be selected by column. If it does not meet the requirements, the PEs in different columns should be selected or the reallocated PEs should be restored and reallocated. The second half of the PE should be selected first. If PE 1=(0,0)+(0,2), the weight allocation of the position (0,2) should be changed first. If it does not meet the requirements, the weight of the first half of the PE should be selected for reallocation.

[0051] Example 2

[0052] A method for accelerating deconvolution calculation in a neural network, comprising the following steps:

[0053] S1: The original feature map module stores the original feature map to be deconvolved;

[0054] S2: The data input module transmits the pixel values ​​to be calculated in the feature map to the next module in batches according to the deconvolution calculation mode;

[0055] S3: The deconvolution calculation module redistributes the weights contained in each processing unit PE according to a specific rule, thereby accelerating and completing the deconvolution calculation;

[0056] S4: The result fusion module fuses the deconvolution results of different computing modes on the same output feature map.

[0057] Furthermore, the deconvolution calculation module is composed of a plurality of processing units PE that map weights according to the proposed rule, receives pixel values ​​of specific positions of feature maps under the same calculation mode in batches from the data input module, distributes pixel values ​​at different positions to different pre-set calculation units PE through the data path, and calculates the deconvolution results of all pixel values ​​at specific positions of the same calculation mode in parallel within one calculation cycle.

[0058] The result fusion module is a deconvolution calculation result processing device, which gathers the deconvolution results of the same calculation mode of the deconvolution calculation module into the same adder through the data path for result fusion, and then stores them in the specific position of the output feature map in the order of the calculation mode. The number of adders is equal to the number of types of deconvolution calculation modes, that is, there are S 2 An adder.

[0059] The deconvolution calculation module is a set of computing units PEs that are weighted based on a specific rule, and is constructed with marginal extra cost. The pixel values ​​to be calculated in batches under the same calculation mode are used as input, and the calculation results are output in a deconvolution manner within one calculation cycle.

[0060] The result fusion module supports the aggregation and fusion of deconvolution results under the same calculation mode onto the output feature map. The number of adders in this module should be equal to the number of deconvolution calculation modes. That is, there are S 2 An adder.

[0061] Example 3

[0062] like Figure 1 As shown, an acceleration system for deconvolution calculation in a neural network includes:

[0063] The original feature map module is used to store the original feature map to be deconvolution calculated;

[0064] The data input module is used to transmit the pixel values ​​to be calculated in the feature map to the next module in batches according to the deconvolution calculation mode;

[0065] The deconvolution calculation module is used to redistribute the weights contained in each processing unit PE according to a specific rule, thereby accelerating and completing the deconvolution calculation;

[0066] The result fusion module is used to fuse the deconvolution results of different computing modes on the same output feature map.

[0067] The deconvolution calculation module is composed of multiple processing units PE that map weights according to the proposed rule. It receives pixel values ​​of specific positions of the feature map under the same calculation mode in batches from the data input module, and distributes the pixel values ​​at different positions to different pre-set calculation units PE through the data path. The deconvolution results of all pixel values ​​at specific positions of the same calculation mode are calculated in parallel within one calculation cycle.

[0068] The result fusion module is a deconvolution calculation result processing device, which gathers the deconvolution results of the same calculation mode of the deconvolution calculation module into the same adder through the data path for result fusion, and then stores them in the specific position of the output feature map in the order of the calculation mode. The number of adders is equal to the number of types of deconvolution calculation modes, that is, there are S 2 An adder.

[0069] The deconvolution calculation mode is based on the input feature map size I = 3 (such as Figure 2 a), convolution kernel size K = 3 (as shown in Figure 2 b) and the deconvolution movement stride S as an example, assuming that the input feature map is processed by interpolating 0 values, the following will be obtained Figure 2 c shows the interpolation feature map, where the vertical line filled part is the valid area to be calculated. Based on this, the process of sliding the convolution kernel on the interpolation feature map with different starting points will produce the following Figure 2 d shows the four calculation modes, where the diagonal filled part of each calculation mode is the effective calculation area of ​​the convolution kernel. Calculation mode 1 only performs deconvolution calculations on the positions (0,0), (0,2), (2,0), and (2,2). Calculation mode 2 only performs deconvolution calculations on the positions (0,1) and (2,1). Calculation mode 3 only performs deconvolution calculations on the positions (1,0) and (1,2). Calculation mode 4 only performs deconvolution calculations on the position (1,1). It should be noted that in the actual process, it is not necessary to interpolate the original feature map to 0. Figure 2 The interpolated feature map shown in c is for illustration only. Figure 2 d shows the four calculation modes.

[0070] The method of mapping the convolution kernel to the processing unit PE supports mapping the weights of two pixel value positions to be calculated to the same PE. Figure 3 As shown, take the input deconvolution kernel size K = 3, the number of M and the number of channels C as an example. Based on this, PE only stores one pixel value position ( Figure 3 When the weights of the cross-line part in a are mapped to the PE, the weights of all channels at the same position of the same convolution kernel are arranged in rows, and the weights of all channels at the same position of different convolution kernels are arranged in columns. Finally, a PE with a size of C rows and M columns is obtained. Figure 3 a; PE stores two pixel value locations ( Figure 3 b) is equivalent to merging two PEs that only store one pixel value position in the channel direction, that is, the size of the merged PE is 2C rows and M columns. Figure 3 As shown in b, channels 0~C and C+1~2C are the first half and the second half of PE respectively, which store the weights of all channels at two positions of the convolution kernel.

[0071] The specific rule described above supports redistributing the weights contained in each PE. Taking the input feature map size I=3, the deconvolution kernel size K=3 and the deconvolution movement stride S=2 as an example, the corresponding requirement is computing units PE, and S 2 =4 calculation modes. Based on this, the specific content of this rule is to first assign the weight of each position of the convolution kernel to different PEs, and arrange the PEs in the same calculation mode in the same row from left to right and from top to bottom according to the position of the convolution kernel, that is, the first row includes PE 1(0,0), PE 2(0,2), PE3(2,0), PE 4(2,2), the second row includes PE 5(0,1), PE 6(2,1), the third row includes PE 7(1,0), PE 8(1,2), and the fourth row includes PE 9(1,1); then merge the PEs in the same row in pairs. If the number of PEs is an odd number, the weight of the last PE in the row remains unchanged, that is, the first row includes PE 1(0,0)+(0,2), PE 2(0,2)+(2,2), the second row includes PE 3(0,1)+(2,1), the third row includes PE 4(1,0)+(1,2), and the fourth row includes PE 5(1,1); finally, the weights of all PEs are redistributed. The idle PEs in the same column but different rows on the original data path that have not been reallocated should be selected by column. If it does not meet the requirements, the PEs in different columns should be selected or the reallocated PEs should be restored and reallocated. The second half of the PE should be selected first. For example, PE 1 = (0,0) + (0,2), the weight allocation of the position (0,2) should be changed first. If it does not meet the requirements, the weight of the first half of the PE should be selected for reallocation. Based on this, the following is finally obtained: Figure 4 The reallocation results shown are that the first row includes PE 1(0,0)+(1,2) and PE 2(2,2), the second row includes PE 3(0,1)+(2,0), the third row includes PE 4(1,0)+(0,2), and the fourth row includes PE5(1,1)+(2,1).

[0072] The deconvolution calculation module is a set of computing units PE that are weighted based on a specific rule, and the deconvolution calculation module is constructed at a marginal additional cost. The pixel values ​​to be calculated in batches under the same calculation mode are used as input, and the calculation results are output in a deconvolution manner within a calculation cycle. Based on this, we use a simple example of calculating deconvolution in ReRAM to illustrate, where the input feature map size I = 3, the deconvolution kernel size K = 3, and the deconvolution movement stride S = 2, corresponding to the required computing units PE, and S 2 = 4 computing modes. The five computing units PE are arranged into corresponding deconvolution computing modules by specific rules. Figure 4As shown in the figure, after that, the data input module transmits the pixel values ​​of the specific positions to be calculated in the feature map to the corresponding calculation unit PE in batches according to the deconvolution calculation mode. The PE that receives the pixel values ​​performs parallel calculations in one calculation cycle to obtain the deconvolution results; the deconvolution results in the same calculation cycle are selected and stored in the result fusion module as the input for the next step. Based on this, the deconvolution calculation speed is greatly improved, the utilization rate of the processing unit PE is improved, and the data input buffer time of the data input module is optimized.

[0073] The result fusion module supports the aggregation and fusion of the deconvolution results under the same calculation mode onto the output feature map. The number of adders in this module should be equal to the number of deconvolution calculation modes. That is, there are S 2 Assume that the input feature map size I = 3, the deconvolution kernel size K = 3, and the deconvolution movement stride S = 2, and take deconvolution calculation mode 1 as an example, Figure 5 PE 1 to PE 5 are simplified schematic diagrams of the deconvolution calculation module; Adder, the data path between Adder and the deconvolution calculation module, the data path between Adders, the output feature map, and the data path between the output feature map and Adder are schematic diagrams of the result fusion module; wherein, Figure 5 The Adders arranged from top to bottom correspond to calculation modes one, two, three and four respectively. Based on this, in calculation cycle one (Cycle 1), PE 1, PE4, PE 2 and PE 3 will parallelly calculate the deconvolution results of I(1), I(2), I(4) and I(5) in calculation mode one, and then through the data path between Adder and the deconvolution calculation module and the data path between Adder, the calculation results of the four PEs are gathered to the adder corresponding to the same calculation mode for result fusion, and then stored in the specific position of the output feature map in the order of the calculation mode (wherein, the result of calculation mode one is stored in the upper left corner of the sliding window of the output feature map, the result of calculation mode two is stored in the upper right corner of the sliding window of the output feature map, the result of calculation mode three is stored in the lower left corner of the sliding window of the output feature map, and the result of calculation mode four is stored in the lower right corner of the sliding window of the output feature map. If and only if the four positions of the sliding window are filled with data at this moment, the sliding window moves to the next position). Based on this, the hardware working calculation performance of the accelerator is optimized.

[0074] The same or similar reference numerals correspond to the same or similar components;

[0075] The positional relationships described in the drawings are only for illustrative purposes and should not be construed as limiting the present patent.

[0076] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An acceleration system for deconvolution calculation in a neural network, characterized in that: include: The original feature map module is used to store the original feature map to be deconvolution calculated; The data input module is used to transmit the pixel values ​​to be calculated in the feature map to the next module in batches according to the deconvolution calculation mode; The deconvolution calculation module is used to redistribute the weights contained in each processing unit PE according to a specific rule, thereby accelerating and completing the deconvolution calculation; the specific rule is: First, the weight of each position of the convolution kernel is assigned to different PEs respectively, and the PEs in the same computing mode are arranged in the same row from left to right and from top to bottom according to the position of the convolution kernel. Then, the PEs in the same row are merged in pairs. If the number of PEs is an odd number, the weight of the last PE in the row remains unchanged. Finally, the weights of all PEs are redistributed, and the idle and unallocated PEs in the same column but different rows on the original data path are preferentially selected by column. If they do not meet the requirements, PEs in different columns are selected or the reallocated PEs are restored and reallocated. The result fusion module is used to fuse the deconvolution results of different computing modes on the same output feature map.

2. The acceleration system for deconvolution calculation in a neural network according to claim 1, characterized in that: The deconvolution calculation module is composed of multiple processing units PE that map weights according to the proposed rule. It receives pixel values ​​at specific positions of the feature map under the same calculation mode in batches from the data input module, and distributes the pixel values ​​at different positions to different pre-set calculation units PE through the data path. The deconvolution results of all pixel values ​​at specific positions of the same calculation mode are calculated in parallel within one calculation cycle.

3. The acceleration system for deconvolution calculation in a neural network according to claim 2, characterized in that: The result fusion module is a deconvolution calculation result processing device, which aggregates the deconvolution results under the same calculation mode of the deconvolution calculation module into the same adder through a data path for result fusion, and then stores them in the specific position of the output feature map in the order of the calculation modes, wherein the number of adders is equal to the number of types of deconvolution calculation modes.

4. The acceleration system for deconvolution calculation in a neural network according to claim 3, characterized in that: The calculation mode of the deconvolution is: Calculation mode 1 only performs deconvolution calculations on the pixel values ​​at positions (0,0), (0,2), (2,0), and (2,2); calculation mode 2 only performs deconvolution calculations on the pixel values ​​at positions (0,1) and (2,1); calculation mode 3 only performs deconvolution calculations on the pixel values ​​at positions (1,0) and (1,2); and calculation mode 4 only performs deconvolution calculations on the pixel values ​​at positions (1,1).

5. The acceleration system for deconvolution calculation in a neural network according to claim 4, characterized in that: The specific rule is that the first row includes PE 1(0,0), PE 2(0,2), PE 3(2,0), PE 4(2,2), the second row includes PE 5(0,1), PE 6(2,1), the third row includes PE 7(1,0), PE 8(1,2), and the fourth row includes PE 9(1,1); then the PEs in the same row are merged in pairs. If the number of PEs is an odd number, the weight of the last PE in the row remains unchanged, that is, the first row includes PE 1(0,0)+(0,2), PE 2(0,2)+(2,2), the second row includes PE 3(0,1)+(2,1), the third row includes PE 4(1,0)+(1,2), and the fourth row includes PE 5(1,1); finally, the weights of all PEs are redistributed. The idle PEs in the same column but different rows on the original data path that have not been reallocated should be selected by column. If it does not meet the requirements, the PEs in different columns should be selected or the reallocated PEs should be restored and reallocated. The second half of the PE should be selected first. If PE 1=(0,0)+(0,2), the weight allocation of the position (0,2) should be changed first. If it does not meet the requirements, the weight of the first half of the PE should be selected for reallocation.

6. An acceleration method for an acceleration system for deconvolution calculation in a neural network as claimed in claim 1, characterized in that: The following steps are involved: S1: The original feature map module stores the original feature map to be deconvolved; S2: The data input module transmits the pixel values ​​to be calculated in the feature map to the next module in batches according to the deconvolution calculation mode; S3: The deconvolution calculation module redistributes the weights contained in each processing unit PE according to a specific rule, thereby accelerating and completing the deconvolution calculation; S4: The result fusion module fuses the deconvolution results of different computing modes on the same output feature map.

7. The acceleration method for deconvolution calculation in a neural network according to claim 6, characterized in that: The deconvolution calculation module is composed of multiple processing units PE that map weights according to the proposed rule. It receives pixel values ​​at specific positions of the feature map under the same calculation mode in batches from the data input module, and distributes the pixel values ​​at different positions to different pre-set calculation units PE through the data path. The deconvolution results of all pixel values ​​at specific positions of the same calculation mode are calculated in parallel within one calculation cycle.

8. The acceleration method for deconvolution calculation in a neural network according to claim 6, characterized in that: The result fusion module is a deconvolution calculation result processing device, which aggregates the deconvolution results under the same calculation mode of the deconvolution calculation module into the same adder through a data path for result fusion, and then stores them in the specific position of the output feature map in the order of the calculation modes, wherein the number of adders is equal to the number of types of deconvolution calculation modes.

9. The acceleration method for deconvolution calculation in a neural network according to claim 8, characterized in that: The deconvolution calculation module is a set of calculation units PE to which weights are assigned based on a specific rule. The deconvolution calculation module is constructed with marginal additional cost, and batches of pixel values ​​to be calculated under the same calculation mode are used as input, and the calculation results are output in a deconvolution manner within one calculation cycle.

10. The acceleration method for deconvolution calculation in a neural network according to claim 9, characterized in that: The result fusion module supports aggregating and fusing the deconvolution results under the same calculation mode onto the output feature map, wherein the number of adders in the module should be equal to the number of types of deconvolution calculation modes.

Images