Deconvolution processing method and device and electronic equipment

Abstract

The invention provides a deconvolution processing method and apparatus, and an electronic device. The method comprises the steps of obtaining an original feature map and an original weight matrix; performing up-sampling on the original feature map in a mode of copying original feature points in rows and columns to obtain a new feature map; performing rearrangement and row-column copying on the original weight matrix to obtain a new weight matrix of which the size is the same as that of the new feature map; and performing convolution operation based on the new feature map and the new weight matrix to obtain an output feature map. Through rearrangement and copying operation, the new weight matrix can be matched with the size of the feature map after up-sampling. By pre-calculating and storing a new weight matrix, repeated rearrangement and copy operations in each convolution operation can be avoided. The calculation efficiency is improved, and the whole processing process is accelerated. As the eigenvalues are matched with the number of the weights, only one convolution cycle is needed, the calculation performance of deconvolution is improved, and the hardware processing power consumption is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a deconvolution processing method, device and electronic equipment. Background Art

[0002] With the in-depth application of AI in various industries, AI-based Bayer Noise Reduce (AIBNR) has gradually moved from theory to application. Since AIBNR processes real-time images or videos, it places extreme demands on hardware processing performance. Currently, application-specific integrated circuits (ASICs) have appeared to accelerate AIBNR processing. Deconvolution, as a common operator in AIBNR, can shorten hardware processing latency by improving its computing performance, thereby further improving AIBNR processing performance while saving hardware processing power consumption. Therefore, it is extremely important to improve the computing performance of deconvolution. Summary of the invention

[0003] In order to improve the computational performance of deconvolution, the present invention provides a deconvolution processing method, device and electronic device. The technical solution proposed by the present invention is as follows: In a first aspect, the present invention provides a deconvolution processing method, comprising: Get the original feature map and original weight matrix; The original feature map is upsampled by copying the original feature points in rows and columns to obtain a new feature map; The original weight matrix is ​​rearranged and rows and columns are copied to obtain a new weight matrix; wherein the size of the new weight matrix is ​​the same as the size of the new feature map; A convolution operation is performed based on the new feature map and the new weight matrix to obtain the output feature map.

[0004] Optionally, upsampling the original feature map by copying the original feature points in rows and columns to obtain a new feature map includes: Copy each of the original feature points in the row direction to obtain copy feature points of the original feature points in the row direction of the original feature map, wherein the copy feature points and the original feature points are target feature points; The target feature point is copied in the column direction of the original feature map to obtain a new feature map; wherein the number of copies in the row direction and the column direction is the same.

[0005] Optionally, the rearranging and row-column copying of the original weight matrix to obtain a new weight matrix includes: Rearrange the weights in the original weight matrix to obtain a rearranged weight matrix; The rearranged weight matrix is ​​taken as a whole, and rows and columns are copied to obtain a new weight matrix; wherein the new weight matrix is ​​consistent with the size of the new feature map.

[0006] Optionally, the weights in the original weight matrix are arranged in order; and the weights in the original weight matrix are rearranged to obtain a rearranged weight matrix, including: The weights in the original weight matrix are rearranged in the reverse order of the corresponding sequence to obtain a rearranged weight matrix.

[0007] Optionally, the rearrangement of the weights in the original weight matrix to obtain a rearranged weight matrix includes: Swap the diagonal elements of the original weight matrix to obtain the rearranged weight matrix.

[0008] Optionally, performing a convolution operation based on the new feature map and the new weight matrix to obtain an output feature map includes: The convolution kernel is slid on the new feature map according to a preset step size, and the elements at each corresponding position in the new feature map and the new weight matrix are multiplied to obtain the corresponding output, and all the outputs are added together to obtain the output value to obtain the output feature map.

[0009] In a second aspect, the present invention further provides a deconvolution processing device, comprising the following modules: An acquisition module is used to obtain the original feature map and the original weight matrix; An upsampling module is used to upsample the original feature map by copying the original feature points in rows and columns to obtain a new feature map; A copy module is used to rearrange and copy the rows and columns of the original weight matrix to obtain a new weight matrix; The convolution module is used to perform a convolution operation based on the new feature map and the new weight matrix to obtain an output feature map.

[0010] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the deconvolution processing method as described in the first aspect above is implemented.

[0011] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the deconvolution processing method as described in the first aspect above.

[0012] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the deconvolution processing method as described in the first aspect above.

[0013] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows: The deconvolution processing method, device and electronic device provided by the present invention obtain a new feature map by upsampling the original feature map by copying the original feature points in rows and columns, and obtain a new weight matrix with the same size as the new feature map by rearranging and copying the original weight matrix in rows and columns. Through the rearrangement and copying operations, the new weight matrix can match the size of the upsampled feature map. This ensures that the convolution operation can be carried out normally and avoids errors or performance degradation caused by size mismatch. Since the new weight matrix is ​​obtained by copying the original weight matrix, they maintain the feature extraction ability of the original weight matrix. This helps to maintain the consistency of features in the convolution operation, thereby improving the stability and reliability of the entire system. Since the eigenvalues ​​match the number of weights, only one convolution cycle is required, which improves the computational performance of the deconvolution and reduces the power consumption of hardware processing. By pre-calculating and storing the new weight matrix, it is possible to avoid repeated rearrangement and copying operations in each convolution operation. This improves computational efficiency and speeds up the entire processing process.

[0014] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0015] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1a This is a schematic diagram of the deconvolution processing flow in the current ASIC.

[0018] Figure 1b It is a schematic diagram of the division of feature points in the feature map.

[0019] Figure 2 It is a schematic flow chart of the deconvolution processing method provided by the present invention.

[0020] Figure 3 It is a schematic diagram of the deconvolution processing flow provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the deconvolution processing device provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Aiming at the hardware acceleration problem of specific deconvolution in AIBNR, a new feature map upsampling and weight replication scheme is proposed to significantly improve the hardware processing performance while reducing the hardware processing power consumption.

[0025] It should be noted that the specific deconvolution refers to the deconvolution with a kernel of 2x2, a stride of 1, top padding of 1, and left padding of 1. The deconvolution statement is used to replace the specific deconvolution statement. Padding is to add extra zero values ​​to the edge of the input data to maintain the size of the output image after the convolution operation. The top padding and left padding are both 1, which means adding a row and a column of zero values ​​to the top and left of the input data to maintain the spatial dimension during the convolution process and prevent the information from decreasing too quickly at the edge.

[0026] The deconvolution process in the current ASIC is as follows: 1) Preprocessing: Read a certain amount of original feature maps (4H*8W, 4 rows and 8 columns, a total of 32 feature points) and weights (2H*2W, 4 weights).

[0027] 2) Preprocessing: Fill the feature map with zero values ​​( Figure 1a The gray filling in the figure) gives a new feature map (9H*17W, 9 rows and 17 columns, a total of 153 feature points).

[0028] 3) Cycle 0: Select the 8*16 feature points in the upper left corner of the feature map in 2), select the first weight value in the weight in 1) and copy it in rows and columns (8H*16W, 128 weights) to get the new weight, and multiply and add the feature value and weight value.

[0029] 4) Cycle 1: Select the 8*16 feature points in the upper right corner of the feature map in 2), select the second weight value in the weight in 1) and copy it in rows and columns (8H*16W, 128 weights) to get the new weight, and multiply and add the feature value and weight value.

[0030] 5) Cycle 2: Select the 8*16 feature points in the lower left corner of the feature map in 2), select the third weight value in the weight in 1) and copy it in rows and columns (8H*16W, 128 weights) to get the new weight, and multiply and add the feature value and weight value.

[0031] 6) Cycle 3: Select the 8*16 feature points in the lower right corner of the feature map in 2), select the 4th weight value in the weight in 1) and copy it in rows and columns (8H*16W, 128 weights) to get the new weight, and multiply and add the feature value and weight value.

[0032] 7) End.

[0033] The division of the above 8*16 feature points in the upper left corner, 8*16 feature points in the upper right corner, 8*16 feature points in the upper right corner, and 8*16 feature points in the lower left corner in the feature map is referenced Figure 1b shown.

[0034] According to the above description, in the specific deconvolution scenario (Kernel 2x2, Stride 1, Top padding 1, Leftpadding 1), a certain amount of original feature maps (4H*8W, 4 rows and 8 columns, a total of 32 Pix) and weights (1H*1W, 1 weight) are read. The traditional method fills the feature map with zero values ​​to obtain a new feature map (conventional convolution, after filling, it is 9H*17W, 9 rows and 17 columns, a total of 153 Pix, but filling 0 will cause waste, 0 multiplied by any number is 0, which is equivalent to an invalid calculation and will not affect the accumulation result). Each weight is copied as the minimum unit, and copied into 4 groups of 8H*16W weight matrices to obtain new weights. The processed feature map is multiplied by each weight matrix, and then self-accumulated to complete the hardware acceleration processing of the specific deconvolution scenario. It takes 4 cycles to complete all calculations.

[0035] The existing solution is implemented completely according to the deconvolution algorithm process. There is no problem in terms of function, but there are great disadvantages in performance and power consumption. Under this solution, it takes 4 cycles to output the points of the target feature map, but in these 4 cycles, the feature points in 3 cycles are actually zero values. This means that the calculation results of these 3 cycles will not affect the final calculation results, but the calculation process does generate power consumption. In order to more intuitively reflect the process, it is now assumed that the hardware output parallelism is only 1 point (the above example Figure 1ais 8*16 points), at this time the input feature map is 4 points, and the weight is still 4 values: The new input feature map {F00(0), F01(0), F10(0), F11(32)} after filling the original feature map with zero values; 4 weight values ​​{W00(4), W01(6), W10(8), W11(5)}; According to the deconvolution algorithm, the dot product of the feature point and the weight is: F00(0)* W00(4)+ F01(0)* W01(6)+ F10(0)* W10(8)+ F11(32)* W11(5). This process requires 4 cycles, cycle 0 calculates F00(0)* W00(4), cycle 1 calculates F01(0)* W01(6), cycle 2 calculates F10(0)* W10(8), and cycle 3 calculates F11(32)* W11(5).

[0036] Obviously, the calculation outputs of cycle 0 / cycle 1 / cycle 2 are all 0, and cycle 4 outputs 160 (32*5), which is the final value. In this process, cycle 0 / cycle 1 / cycle 2 are all redundant calculations, which reduces performance and wastes power consumption. In order to overcome this defect, the present invention optimizes the copying method of feature maps and weights in the preprocessing stage in the traditional calculation process.

[0037] Combine the following Figure 2-Figure 5 The present invention describes the image processing deconvolution processing method, device and electronic device.

[0038] Reference Figure 2 As shown, the method includes the following: Step S101, obtaining an original feature map and an original weight matrix; The original feature map is the input or intermediate layer output in the convolutional neural network, which usually contains the spatial information of the image or feature data. These weights in the original weight matrix are the parameters in the convolution kernel, which are used to extract features in the convolution operation.

[0039] Step S102: up-sample the original feature map by copying the original feature points in rows and columns to obtain a new feature map.

[0040] Specifically, upsampling is achieved by copying the original feature points in the original feature map in rows and columns. After upsampling, the size of the original feature map is increased so that it can be convolved with the new weight matrix in the subsequent steps. The above deconvolution process can be applied to image data processing or accelerated processing of graphics processing equipment.

[0041] Step S103: rearrange the original weight matrix and copy its rows and columns to obtain a new weight matrix of the same size as the new feature map.

[0042] The original weight matrix is ​​rearranged according to the upsampling method and target size. In order to match the size of the new feature map, the weight matrix is ​​copied row and column to maintain the consistency of feature extraction in the new convolution operation. By adjusting the size of the new weight matrix to the same size as the new feature map, it is ensured that the convolution operation can proceed normally and extract features corresponding to the original feature map.

[0043] Step S104: Perform a convolution operation based on the new feature map and the new weight matrix to extract features and obtain output values. The result of the convolution operation contains higher-level feature information, which can be used for subsequent classification, regression or other tasks.

[0044] The present invention uses upsampling and weight rearrangement, and deconvolution can flexibly adjust the size and shape of the output feature map to adapt to different application scenarios. Moreover, through the upsampling process, the size of the original feature map is increased, thereby improving the resolution of the feature map. This is beneficial for subsequent feature extraction and image processing tasks, because higher resolution can provide more detailed information. The calculation delay that originally required 4 cycles is optimized to only 1 cycle, which improves the computing performance by 75% and reduces power consumption by more than 75%. This helps to improve the accuracy of image processing while improving computing performance. Deconvolution can be seamlessly integrated into the existing convolutional neural network architecture for building an end-to-end upsampling-downsampling network. This helps to achieve more accurate feature extraction and image restoration in tasks such as image segmentation and image reconstruction. In image reconstruction tasks (such as super-resolution reconstruction and deblurring), deconvolution can improve the quality of reconstructed images by gradually restoring the detailed information of the image.

[0045] In an optional embodiment, the above step S102 up-sampling the original feature map by copying the original feature points in rows and columns to obtain a new feature map includes: S1021, copying each of the original feature points in the row direction to obtain copy feature points of the original feature points in the row direction of the original feature map, wherein the copy feature points and the original feature points are target feature points; First, it is necessary to determine the number of times the feature point is copied in the row direction and the column direction. In the method of the present invention, the number of copies is consistent in the row direction and the column direction. For each original feature point in the original feature map, each of the original feature points is copied in the row direction, for example, the feature point is copied to the right or left of the original feature point. The number of copies determines the number of new feature points in the row direction. For example, if the number of copies is 1, an identical feature point will be added to the right of each original feature point, resulting in 2 feature points in the row direction.

[0046] S1022. Copy the target feature points in the column direction of the original feature map to obtain a new feature map.

[0047] The feature map copied in the row direction is copied in the column direction. The number of copies is the same as the number of copies in the row direction. In this way, each feature point added in the row direction will also be copied in the column direction, thereby forming a new feature map. For example, if the number of copies is 1, 2 feature points in the row direction are obtained, and the above 2 feature points are copied in the column direction to obtain 4 feature points in total, two feature points in each row and column direction. This operation is performed on all feature points of the original feature map to obtain a new feature map.

[0048] It should be noted that the original feature map is upsampled by copying the original feature points in rows and columns to obtain a new feature map. The copying method is not limited to copying the original feature points in the original feature map in rows and then in columns. You can also choose to copy in the column direction first and then in the row direction.

[0049] Through the above steps, each feature point in the original feature map will be replicated in the row and column directions to construct a new feature map with an increased size. The size of the new feature map will be (number of replications + 1) × (number of replications + 1) times the size of the original feature map.

[0050] The present invention increases the size of the original feature map through the upsampling process, thereby improving the resolution of the feature map. This is beneficial for subsequent feature extraction and image processing tasks, because higher resolution can provide more detailed information. Since the upsampling process is implemented by copying feature points, the feature points in the new feature map are consistent with the feature points in the original feature map. This helps to maintain the consistency of features in subsequent processing, thereby improving the stability and reliability of the entire system. Compared with other upsampling methods (such as bilinear interpolation, bicubic interpolation, etc.), the upsampling method of copying feature points has a simpler calculation process. There is no need to perform complex interpolation calculations, so computing resources can be saved and processing speed can be accelerated. Moreover, this upsampling method is flexible and can adapt to different application scenarios. By adjusting the number of copies, the size and resolution of the new feature map can be flexibly controlled to meet different image processing requirements.

[0051] In an optional embodiment, the above step S103 of rearranging and copying rows and columns of the original weight matrix to obtain a new weight matrix includes: S1031. Rearrange the weights in the original weight matrix to obtain a rearranged weight matrix.

[0052] In order to match the size of the feature map after subsequent upsampling and the requirements of subsequent convolution operations, the weights in the original weight matrix are rearranged to obtain a rearranged weight matrix.

[0053] S1032: Take the rearranged weight matrix as a whole, and perform row and column replication to obtain a new weight matrix.

[0054] The rearranged weight matrix is ​​copied row and column as a whole to obtain a new weight matrix, and the new weight matrix is ​​consistent with the size of the new feature map. Therefore, as long as the rearranged weight matrix can be copied into a new weight matrix consistent with the size of the new feature map by row and column copying (that is, the number of rows and columns of the new weight matrix is ​​the same as the number of rows and columns of the new feature map), this embodiment does not limit whether to copy in the row direction first and then in the column direction, or to copy in the column direction first and then in the row direction. If the copying in the row direction is performed first and then in the column direction, the rearranged weight matrix is ​​taken as a whole, and the rearranged weight matrix is ​​copied in the row direction to obtain a preset number of weight matrices in the row direction; wherein the preset number is consistent with the number of columns of the original feature map.

[0055] The rearranged weight matrix is ​​copied as a whole in the row direction. The number of copies is preset and consistent with the number of columns of the original feature map. If the original feature map has N columns, it will be copied N-1 times in the row direction (assuming that the copied entity already represents a column), thus obtaining N identical weight matrices (arranged in the row direction).

[0056] Then, the preset number of weight matrices are copied in the column direction to obtain a new weight matrix; wherein, after the copying, the number of the rearranged weight matrices in the column direction is consistent with the number of rows of the original feature map.

[0057] The N weight matrices obtained above are copied in the column direction. The number of copies is also preset and consistent with the number of rows of the original feature map. If the original feature map has M rows, it is copied M-1 times in the column direction to obtain M×N weight matrices (forming a two-dimensional weight matrix). Through the above steps, a new weight matrix is ​​obtained, whose size matches the size of the upsampled feature map. This new weight matrix will be used in subsequent convolution operations to extract features and generate output values.

[0058] The present invention uses rearrangement and copying operations to enable the new weight matrix to match the size of the feature map after upsampling. This ensures that the convolution operation can be carried out normally and avoids errors or performance degradation caused by size mismatch. Since the new weight matrix is ​​obtained by copying the original weight matrix, they maintain the feature extraction capability of the original weight matrix. This helps to maintain the consistency of features in the convolution operation, thereby improving the stability and reliability of the entire system. By pre-calculating and storing the new weight matrix, repeated rearrangement and copying operations in each convolution operation can be avoided. This improves computational efficiency and speeds up the entire processing process. By adjusting the number of rearrangements and copies, different feature map sizes and convolution operation requirements can be flexibly adapted. In addition, the method can also be combined with other upsampling methods or convolution operations to build a more complex image processing system.

[0059] In an optional embodiment, the original weight matrix in the above step S1031 is a two-dimensional array or a matrix with two rows and two columns, in which the weights are arranged in a certain order (such as from left to right, from top to bottom). These weights are used for subsequent convolution operations.

[0060] The above step S1031 re-arranges the weights in the original weight matrix to obtain a re-arranged weight matrix, including: The weights in the original weight matrix are rearranged in the reverse order of the corresponding sequence to obtain a rearranged weight matrix.

[0061] Reverse ordering means arranging the weights in the original weight matrix in the reverse order of their original order. For example, if the original weight matrix is ​​[w1, w2, w3, ..., wn], the reverse ordering weight matrix will be [wn,wn-1, wn-2, ..., w1].

[0062] The process of performing reverse order rearrangement is: First, determine the starting and ending positions of the original weight matrix. Then, starting from the ending position, copy the weights to the new positions one by one until the starting position is reached. The order of copying is in the reverse order of the original weights. Finally, a new weight matrix, the rearranged weight matrix, is obtained. The rearranged weight matrix can be used for subsequent convolution operations. By adjusting the order of weight arrangement, the result of the convolution operation can be affected, thereby changing the output characteristics of the network.

[0063] The rearranged weight matrix can change the way the convolution operation extracts features from the input feature map. This helps the network learn different feature representations, thereby improving the generalization ability of the model. By changing the order of weights, the robustness of the model to the input data can be increased. This is because different weight arrangements will cause the network to respond differently to the input data, thus reducing the risk of overfitting to a certain extent. Reverse order rearrangement is easy to implement and debug, does not require additional computing resources, and does not significantly increase the training time of the model.

[0064] In an alternative embodiment, referring to Figure 3 As shown, the original weight matrix {W00(4), W01(6), W10(8), W11(5)} includes a first weight W00(4), a second weight W01(6), a third weight W10(8) and a fourth weight W11(5) arranged in sequence; the weights in the original weight matrix are rearranged to obtain a rearranged weight matrix, including: Swapping the diagonal elements of the original weight matrix yields the rearranged weight matrix, i.e., from {W00(4), W01(6), W10(8), W11(5)} -> {W11(5), W10(8), W01(6), W00(4)}.

[0065] In an optional embodiment, the convolution operation based on the new feature map and the new weight matrix described in step S104 to obtain the output feature map includes: S1041. Slide the convolution kernel on the new feature map according to a preset step size, and perform multiplication and addition operations on the elements at each corresponding position in the new feature map and the new weight matrix to obtain an output value, until the entire new feature map is traversed to obtain an output feature map.

[0066] The convolution kernel is the key element in the convolution operation, which determines the type of features to be extracted from the feature map. The stride is the step size of the convolution kernel on the feature map, which affects the size of the output feature map. Multiply each element of the convolution kernel (i.e., the weight in the new weight matrix) with the element at the corresponding position in the feature map. This step usually involves matrix multiplication or element-by-element multiplication, depending on the dimensions of the convolution kernel and the feature map. Add all the multiplication results to get the output value at that position. This step is the core of the convolution operation, which extracts features by weighted summing of local features. Slide the convolution kernel on the feature map according to the preset stride, and repeat the multiplication and addition operations until the entire feature map is traversed to obtain the output feature value. In this way, each position will get an output value, which will constitute the output feature map. The output values ​​of all positions are combined to form the output feature map. The size of the output feature map depends on the size of the new feature map mentioned above, the size of the convolution kernel, and the stride.

[0067] In an alternative embodiment, referring to Figure 3 As shown, the step size can be set to 2 to finally obtain an output feature map of 4H*8W.

[0068] The present invention can extract useful features from the feature map through convolution operation, which are crucial for subsequent image analysis and processing. By adjusting the parameters of the convolution kernel, different types of features such as edges, textures, shapes, etc. can be extracted. By accurately controlling the parameters and step size of the convolution kernel, useful features can be extracted and the amount of calculation can be reduced, thereby providing strong support for subsequent image analysis and processing.

[0069] Reference Figure 3 As shown, the process of applying the deconvolution processing method provided by the present invention to process an original feature map is as follows: 1) Preprocessing: Read a certain amount of original feature maps (4H*8W, 4 rows and 8 columns, a total of 32 feature points) and original weight matrices (2H*2W, 4 weights)

[0070] 2) Preprocessing: Upsample the original feature map to obtain a new feature map (8H*16W, 8 rows and 16 columns, a total of 128 feature points), that is, copy the feature points in the original feature map, but the copied points are not 0. This process specifically includes: copying the feature points on the right side of the original feature points in the row direction to obtain 2 feature points in the row direction, copying the above 2 feature points in the column direction to obtain 4 feature points, two feature points in each row and column direction, and perform this operation on all feature points of the original feature map.

[0071] The original weight matrix is ​​copied row and column to obtain a new weight matrix (8H*16W, 8 rows and 16 columns, a total of 128 weights). This process specifically includes: first, rearrange the 4 weights internally, {W00(4), W01(6), W10(8), W11(5)} -> {W11(5), W10(8), W01(6), W00(4)}, then take the set of 4 weights as a whole, copy the set 7 times on the right side of the set in the row direction (assuming that the copied entity already represents a column), and obtain 2*16 feature points in the row direction, and copy the above 2*16 weights 3 times in the column direction, and obtain a total of 8*16 weights in the row and column directions to obtain a new weight matrix.

[0072] 3) Cycle 0: Multiply and add the eigenvalues ​​in the new feature map and the weights in the new weight matrix.

[0073] The deconvolution processing device provided by the present invention is described below. The deconvolution processing device described below and the deconvolution processing method described above can be referred to each other.

[0074] The deconvolution processing device provided by the present invention refers to Figure 4 As shown, it includes the following modules: An acquisition module 201 is used to acquire an original feature map and an original weight matrix; An upsampling module 202 is used to upsample the original feature map by copying the original feature points in rows and columns to obtain a new feature map; A copy module 203 is used to rearrange and copy the rows and columns of the original weight matrix to obtain a new weight matrix; The convolution module 204 is used to perform a convolution operation based on the new feature map and the new weight matrix to obtain an output feature map.

[0075] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the deconvolution processing method.

[0076] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0077] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the deconvolution processing methods provided by the above methods.

[0078] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the deconvolution processing method provided by the above methods.

[0079] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0080] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deconvolution processing method, characterized in that: include: Get the original feature map and original weight matrix; The original feature map is upsampled by copying the original feature points in rows and columns to obtain a new feature map; The original weight matrix is ​​rearranged and rows and columns are copied to obtain a new weight matrix; wherein the size of the new weight matrix is ​​the same as the size of the new feature map; A convolution operation is performed based on the new feature map and the new weight matrix to obtain the output feature map.

2. The deconvolution processing method according to claim 1, characterized in that: The upsampling of the original feature map by copying the original feature points in rows and columns to obtain a new feature map includes: Copy each of the original feature points in the row direction to obtain copy feature points of the original feature points in the row direction of the original feature map, wherein the copy feature points and the original feature points are target feature points; The target feature point is copied in the column direction of the original feature map to obtain a new feature map; wherein the number of copies in the row direction and the column direction is the same.

3. The deconvolution processing method according to claim 1, characterized in that: The new weight matrix is ​​obtained by rearranging and copying rows and columns of the original weight matrix, including: Rearrange the weights in the original weight matrix to obtain a rearranged weight matrix; The rearranged weight matrix is ​​taken as a whole, and rows and columns are copied to obtain a new weight matrix; wherein the new weight matrix is ​​consistent with the size of the new feature map.

4. The deconvolution processing method according to claim 3, characterized in that: The weights in the original weight matrix are arranged in order; the weights in the original weight matrix are rearranged to obtain a rearranged weight matrix, including: The weights in the original weight matrix are rearranged in the reverse order of the corresponding sequence to obtain a rearranged weight matrix.

5. The deconvolution processing method according to claim 3, characterized in that: The step of rearranging the weights in the original weight matrix to obtain a rearranged weight matrix includes: Swap the diagonal elements of the original weight matrix to obtain the rearranged weight matrix.

6. The deconvolution processing method according to claim 1, characterized in that: The convolution operation is performed based on the new feature map and the new weight matrix to obtain the output feature map, including: The convolution kernel is slid on the new feature map according to a preset step size, and a multiplication and addition operation is performed on the elements at each corresponding position in the new feature map and the new weight matrix to obtain an output value, until the entire new feature map is traversed to obtain an output feature map.

7. A deconvolution processing device, characterized in that: include: An acquisition module is used to obtain the original feature map and the original weight matrix; An upsampling module is used to upsample the original feature map by copying the original feature points in rows and columns to obtain a new feature map; A copy module is used to rearrange and copy the rows and columns of the original weight matrix to obtain a new weight matrix; The convolution module is used to perform a convolution operation based on the new feature map and the new weight matrix to obtain an output feature map.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the deconvolution processing method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the deconvolution processing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the deconvolution processing method according to any one of claims 1 to 6 is implemented.

Images