A method and device for accelerating calculation of dilated convolution
By decomposing the expansion convolution operation into sub-operation parallel processing, the problems of high complexity and poor versatility in the prior art are solved, and efficient image processing and information retention are achieved.
Patent Information
- Application Number
- CN202010659646.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-08
- Publication Date
- 2024-12-10
- Estimated Expiration
- 2040-07-08
AI Technical Summary
In the prior art, the implementation of expansion convolutional operation is poor in general and has high complexity, which leads to the problem of loss of information in image processing and the inability to reconstruct small object images. The acceleration calculation method is complex and difficult to compatible with volumes of different sizes. Accumulated core.
The expansion convolution operation of R×S is decomposed into S sub-expansion convolution operations of S R×1, and the weight value is cached in parallel to the calculation unit. The accumulation is performed between the calculation units through sliding window operation and multiplication operation, and the sub-expansion volume is superimposed. The intermediate result of the product operation is used to obtain the convolution result.
High-versatility and low-complexity expansion convolution accelerated calculations are realized, which reduces hardware resources and computing power consumption, avoids the problems of information loss and the inability to reconstruct small object images.
Smart Images
Figure CN113989169B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of deep learning, and specifically relates to a method and device for accelerating calculation of dilated convolution. Background Art
[0002] This section is intended to provide a background or context to embodiments of the invention that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.
[0003] Convolutional Neural Networks (CNN) is a deep feedforward artificial neural network that has been applied in many fields, such as image recognition. Convolutional neural networks perform relatively complex calculations during image data processing, including convolution calculations, batch normalization calculations, activation calculations, etc.
[0004] When using CNN for image processing, it is usually necessary to perform multiple convolution and pooling operations to increase the receptive field of the model. Pooling can reduce the size of the image, and then using the convolution kernel can increase the receptive field; the feature map after convolution and pooling will be relatively small, and can be passed to the fully connected network for classification. However, when performing image segmentation, it is necessary to make predictions for each pixel, so the reduced-size feature map must be converted back to the original image size through upsampling methods (such as deconv deconvolution) before prediction. The main problems in this process are: (1) Information loss. The pooling operation is irreversible. Information is lost when the image size is restored by upsampling the feature map. (2) It is impossible to reconstruct small object images. If an object occupies 4×4 pixels, the information of the object cannot be reconstructed after 4 pooling operations. Therefore, in order to avoid using operations such as pooling to expand the receptive field, dilated convolution is proposed in "MULTI-SCALE CONTEXT AGGREGATION BY DILATED CONVOLUTIONS". The dilated convolution operation expands the convolution kernel to a specified size during the convolution process and fills the unoccupied areas of the original convolution kernel with zeros.
[0005] In the prior art, in order to accelerate the convolution calculation of image data to obtain image processing results, a common method is to use the Im2col function to optimize the convolution operation. In the CNN learning and training process, the entire image is usually not processed at once, but the image is first divided into multiple small blocks (patches), and each patch needs to be rearranged through the Im2col process to expand the three-dimensional patch into a one-dimensional vector, and then the convolution operation can be converted into a two-dimensional matrix multiplication.
[0006] In the above scheme, when calculating a dilated convolution, it is necessary to access multiple rows and columns of data at the same time, such as Figure 1 As shown in the figure, taking a dilated convolution with a size of 3*3 and a dilation rate of 2 as an example, the 9 numbers required for a dilated convolution are distributed in 3 rows and 3 columns. It should be understood that data reading can only ensure the bandwidth of data reading for continuous data. If the above 9 numbers need to be accessed at the same time, the internal memory layout needs to be specially designed, such as increasing the parallelism of memory access by internal memory segmentation. However, since NPU usually needs to support convolutions of different sizes, if it is necessary to achieve universality for different convolution kernels, the memory needs to be divided into many small blocks to be compatible with various settings. This will increase the area of the internal memory on the one hand, and increase the complexity of the data access logic on the other hand.
[0007] Therefore, designing a dilated convolution acceleration calculation method with high versatility and low complexity is a technical problem that needs to be solved urgently. Summary of the invention
[0008] In view of the problem that the implementation of the dilated convolution operation in the prior art is poor in versatility and high in complexity, the embodiment of the present invention proposes a dilated convolution acceleration calculation method and device. The method and device can solve the above problems.
[0009] The following solutions are provided in the embodiments of the present invention.
[0010] In a first aspect, a method for accelerating calculation of dilated convolution is provided, the method comprising: decomposing an R×S dilated convolution operation into S R×1 sub-dilated convolution operations, wherein R is a height of a convolution kernel and S is a width of a convolution kernel; for each R×1 sub-dilated convolution operation, caching a plurality of weight values in parallel to a plurality of computing units in a computing unit array; determining from input image data a plurality of input data streams corresponding to the plurality of weight values respectively, and inputting the plurality of input data streams in parallel to the plurality of computing units; within the plurality of computing units, each performing a sliding window operation and a multiplication operation based on the cached weight values and the received input data streams, and performing an accumulation operation between the plurality of computing units to output an intermediate result of the sub-dilated convolution operation; and superimposing the intermediate results of the S R×1 sub-dilated convolution operations to obtain a convolution result of the dilated convolution operation.
[0011] In a possible implementation, required data is read from the input image data according to the dilation rate and convolution step size of the dilated convolution operation to splice together the input data stream corresponding to each weight value.
[0012] In a possible implementation, multiple rows of data in the input image data are read according to the dilation rate and convolution step size of the dilated convolution operation to splice together to form an input data stream corresponding to each weight value.
[0013] In a possible implementation, the sliding step size of the sliding window operation is determined according to the convolution step size of the dilated convolution operation.
[0014] In a possible implementation, superimposing the intermediate results of the S R×1 sub-dilated convolution operations includes: superimposing the intermediate results of the S R×1 sub-dilated convolution operations in real time during the S R×1 sub-dilated convolution operations, or superimposing multiple intermediate results after the S R×1 sub-dilated convolution operations are completed.
[0015] In a second aspect, a dilated convolution acceleration computing device is provided, the device comprising: a logic control unit and a computing unit array; wherein the logic control unit is configured to: decompose an R×S dilated convolution operation into S R×1 sub-dilated convolution operations, wherein R is a convolution kernel height and S is a convolution kernel width; for each R×1 sub-dilated convolution operation, cache multiple weight values in parallel to multiple computing units in the computing unit array; determine multiple input data streams corresponding to the multiple weight values respectively, and input the multiple input data streams in parallel to the multiple computing units; the computing unit array is configured to: within the multiple computing units, each performs a sliding window operation and a multiplication operation based on the cached weight values and the received input data streams, and performs an accumulation operation between the multiple computing units to output an intermediate result of the sub-dilated convolution operation; superimpose the intermediate results of the S R×1 sub-dilated convolution operations to obtain a convolution result of the dilated convolution operation.
[0016] In a possible implementation, the logic control unit is configured to read required data from the input image data according to the dilation rate and convolution step of the dilated convolution operation to splice together the input data stream corresponding to each weight value.
[0017] In a possible implementation, the logic control unit is configured to read multiple rows of data in the input image data according to the dilation rate and convolution step of the dilated convolution operation to splice together the input data stream corresponding to each weight value.
[0018] In a possible implementation, the sliding step size of the sliding window operation is determined according to the convolution step size of the dilated convolution operation.
[0019] In a possible implementation, the computing unit array is configured to: superimpose intermediate results of the S R×1 sub-dilated convolution operations in real time during the S R×1 sub-dilated convolution operations, or superimpose multiple intermediate results after the S R×1 sub-dilated convolution operations are completed.
[0020] At least one of the above-mentioned technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: by decomposing an R×S dilated convolution operation into S R×1 sub-dilated convolution operations, the multiple weight values obtained by the decomposition are cached in parallel in a column of computing units, and each computing unit performs sliding window operations and multiplication operations on the input data stream based on the cached weight values, and performs accumulation operations between the computing units, thereby realizing accelerated dilated convolution operations. Through this solution, there is no need to implement the function of Im2col separately, which reduces the complexity.
[0021] It should be understood that the above description is only an overview of the technical solution of the present invention, so that the technical means of the present invention can be more clearly understood and implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described below by way of example. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The advantages and benefits described herein and other advantages and benefits will be apparent to those of ordinary skill in the art by reading the detailed description of the exemplary embodiments below. The accompanying drawings are only used for the purpose of illustrating exemplary embodiments and are not to be considered as limiting the present invention. Moreover, the same reference numerals are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0023] Figure 1 It is a schematic diagram of a dilated convolution operation based on Im2col in the prior art;
[0024] Figure 2 It is a structural schematic diagram of a convolutional neural network computing device;
[0025] Figure 3 A schematic diagram of a flow chart of a method for accelerating calculation of dilated convolution according to an embodiment of the present invention;
[0026] Figure 4a A schematic diagram of a dilated convolution operation. Figure 4b Another schematic diagram of the dilated convolution operation shown in Figure a;
[0027] Figure 5 A schematic diagram of decomposing an R×S dilated convolution operation into S R×1 sub-dilated convolution operations according to an embodiment of the present invention;
[0028] Figure 6Another schematic diagram of decomposing an R×S dilated convolution operation into S R×1 sub-dilated convolution operations according to an embodiment of the present invention;
[0029] Figure 7 A schematic diagram of performing dilated convolution using a computing unit (PE) having a cached weight value according to an embodiment of the present invention;
[0030] Figure 8 A schematic diagram of performing a sliding window operation using a computing unit (PE) having a cached weight value according to an embodiment of the present invention;
[0031] Fig. 9 Schematic diagram of the structure of a dilated convolution accelerated computing device according to an embodiment of the present invention.
[0032] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION
[0033] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0034] In the present invention, it should be understood that terms such as “including” or “having” are intended to indicate the existence of features, numbers, steps, behaviors, components, parts or a combination thereof disclosed in the specification, and are not intended to exclude the possibility of the existence of one or more other features, numbers, steps, behaviors, components, parts or a combination thereof.
[0035] It should also be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0036] Figure 2 A structural schematic diagram 20 of a dilated convolution acceleration computing device is shown. It includes: a computing platform 21 and an external memory 22, wherein the computing platform 21 includes at least a computing unit (PE) array 211 for performing dilated convolution operations and an internal memory 212, wherein the external memory 22 usually uses a low-cost storage medium, which usually has limited bandwidth and high power consumption for reading and writing. The internal memory usually uses a storage medium with a faster access speed, such as SRAM, which has a higher bandwidth and a lower read and write cost, but usually has a higher cost, so the capacity is generally limited.
[0037] Figure 3FIG. 3 is a flow chart of a method 300 for accelerating calculation of dilated convolution according to an embodiment of the present invention. Figure 2 The dilated convolution acceleration computing device shown is Figure 3 Various aspects of the dilated convolution acceleration calculation method 300 are described in detail.
[0038] like Figure 3 As shown, the method 300 may include:
[0039] Step 301: Decompose the R×S dilated convolution operation into S R×1 sub-dilated convolution operations, where R is the height of the convolution kernel and S is the width of the convolution kernel;
[0040] Step 302: for each R×1 sub-dilated convolution operation, cache multiple weight values in parallel to multiple computing units in the computing unit array;
[0041] Step 303: determining multiple input data streams corresponding to multiple weight values respectively from the input image data, and inputting the multiple input data streams into multiple computing units in parallel;
[0042] Step 304: within the plurality of computing units, each performs a sliding window operation and a multiplication operation based on the cached weight value and the received input data stream, and performs an accumulation operation between the plurality of computing units to output an intermediate result of the sub-dilated convolution operation;
[0043] Step 305: superimpose the intermediate results of the S R×1 sub-dilated convolution operations to obtain a convolution result of the dilated convolution operation.
[0044] Figure 4a and Figure 4b An example of a dilated convolution operation is shown. Figure 4a As shown, the input image data of the dilated convolution operation is composed of several pixel values D abc The matrix data is composed of, where subscript a represents the channel number, subscript b represents the row number, and takes values of 0, 1, ..., 6; subscript c represents the column number, and takes values of 0, 1, ..., 6, that is, its size is 7 (columns) * 7 (rows). The input image data can be obtained according to the real image input to the neural network model, and is usually stored in the external memory 22 row by row and channel by channel in sequence. The convolution kernel of the dilated convolution layer is composed of several weight values W edf The convolution kernel size is 3 (columns) * 3 (rows), and the convolution kernel can be obtained by training a number of image sample data and is usually stored in an external memory 22. Figure 4bAs shown in the figure, the dilated convolution result is the output data obtained by the convolution kernel performing dilated convolution on the input image data, where P xy The subscript x represents the row number, which takes values of 0, 1, and 2; the subscript y represents the column number, which takes values of 0, 1, and 2; that is, its size is 3 (columns) * 3 (rows). In this embodiment, the dilated convolution operation implemented is an operation of mapping the dilated convolution result from the input image data and the weight matrix. In this embodiment, the calculation of channel 0 is used as an example for description. It is understood by those skilled in the art that other channels can be calculated according to the calculation logic of channel 0.
[0045] by Figure 4a and Figure 4b The embodiment of the present invention is described in detail by taking the dilated convolution operation shown as an example.
[0046] Assume that the dilation rate of the dilated convolution operation is λ = 2 and the convolution step size is s = 1. First, we can Figure 4a and Figure 4b The 3×3 dilated convolution operation shown is decomposed into three 3×1 sub-dilated convolution operations. Figure 5 As shown in , the 3×3 convolution kernel is split into three 3×1 sub-convolution kernels: the first sub-convolution kernel, the second sub-convolution kernel, and the third sub-convolution kernel, and then Figure 4b The dilated convolution operation shown is split into Figure 6 Three groups of sub-dilated convolution operations are shown.
[0047] Next, for each 3×1 sub-expanded convolution operation, as Figure 7 As shown, the convolution calculation of channel 0 is described as an example. First, the three weight values (W 000 ,W 010 ,W 020 ) are cached in parallel to the three computing units in the computing unit array. Next, the three weight values (W 000 ,W 010 ,W 020 ) and input the 3 input data streams into 3 computing units in parallel.
[0048] In some possible implementations, multiple lines of data in the input image data can be determined according to the dilation rate λ=2 and the convolution step size s=1 of the dilated convolution operation, and the required data can be read from the multiple lines of data in sequence to form an input data stream corresponding to each weight value. For example, the input cache weight value W 000 The data flow of the computing unit can be: 000 ,D 001 ,D 002 ,D 010 ,D 011 ,D012 ,D 020 ,D 021 ,D 022 ), that is: (D 000 ,...,D 00(W-2λ-1) ,D 010 ,...,D 01(W-2λ-1) ,D 0(H-2λ-1)0 ,...,D 0(H-2λ-1)(W-2λ-1) ), where W is the width of the input image data, H is the height of the input image data, W = 7, H = 7. Accordingly, it can be deduced in turn that the input buffer has a weight value W 010 and W 020 Next, within the three computing units, a sliding window operation and a multiplication operation are performed based on the cached weight value and the received input data stream, and an accumulation operation is performed between the multiple computing units to output an intermediate result of the sub-expanded convolution operation. For example, within the computing unit, a sliding window operation is performed on the input data stream with the cached weight value as a sliding window, wherein in each sliding window operation, a multiplication operation is performed on the data in the window to obtain a multiplication result, and after each multiplication operation, an accumulation operation is performed on the multiplication results of a column of computing units to obtain an intermediate result corresponding to a pixel position.
[0049] In a possible implementation, superimposing the intermediate results of S R×1 sub-dilated convolution operations includes: superimposing the intermediate results of the S R×1 sub-dilated convolution operations in real time during the S R×1 sub-dilated convolution operations, or superimposing multiple intermediate results after the S R×1 sub-dilated convolution operations are completed. For example, after the first intermediate result, the second intermediate result, and the third intermediate result are calculated using the computing unit array, the intermediate results of the three 3×1 sub-dilated convolution operations cached above are superimposed to obtain the convolution result of the dilated convolution operation. Optionally, superimposition can also be performed in real time during the calculation process of the computing unit, such as when P of the second intermediate result is calculated. 00″ Then add it to the first intermediate result P in real time 00′ Optionally, the superposition mode can be determined according to the size of the on-chip cache area to perform superposition in real time during the calculation process, or to perform superposition of multiple intermediate results after the calculation is completed.
[0050] In some other possible implementations, in order to further simplify the data reading logic, multiple rows of data in the input image data can be determined according to the dilation rate and convolution step of the dilated convolution operation, and the multiple rows of data can be spliced into an input data stream corresponding to each weight value. For example, when the dilation rate λ=2, the convolution step=1, the convolution kernel size is 3×3, and the input image data size is 7×7, the weight value W can be cached. 000The computing unit of the input image directly inputs the splicing combination of the first row, the second row and the third row of data of the input image data, that is, (D 000 ,...,D 006 ,D 010 ,...,D 016 ,D 020 ,...,D 026 ), and the sliding window area of the sliding window operation is controlled by the computing unit internally, for example, the sliding window operation can be performed only on the data stream with a column number not exceeding 2, and the data with a column number greater than 2 can be skipped.
[0051] like Figure 8 As shown, taking the above 3×3 dilated convolution operation as an example, based on the cached weight value W 000 The operation of the computing unit array is described by the computing unit, where at time sequence T1, the weight value W 000 Slide on the input data stream to D 000 , and perform multiplication and addition operations based on the weight values in the window and the input data: W 000 ×D 000 At the same time, other computing units (PEs) also perform synchronous sliding on their corresponding input data streams based on the cached weight segments, and perform multiplication and addition operations based on the weight values and the data in the window. For example, the weight value W 001 Slide to D 020 , weight value W 002 Slide to D 040 , and so on, the multiplication result of each computing unit (PE) at the timing T1 can be obtained, and then all the multiplication results of a column of computing units (PE) corresponding to the first sub-convolution kernel at the timing T1 can be added to obtain an output value P 00′ At time T2, the weight value W 000 Slide on the input data stream to D according to the convolution step size 001 , and so on, we can get Figure 6 Another output value P in 01′ , at timing T3 Figure 6 Another output value P in 02′ Finally, we can get Figure 6 For each value in the first intermediate result in , the first intermediate result is cached.
[0052] It should be understood that after the calculation involving the first sub-convolution kernel is completed, the cache value in the calculation unit array is updated to the multiple weight values of the second sub-convolution kernel, and the weights are obtained in a similar manner as described above. Figure 6 For each value in the second intermediate result in , and so on, the intermediate result corresponding to each sub-expanded convolution operation can be obtained. By superimposing the corresponding positions of the above multiple intermediate results, the convolution result of the expanded convolution operation can be obtained.
[0053] In another embodiment, multiple columns of computing units (PE) can be used to implement the dilated convolution operation of multiple convolution kernels in parallel. Furthermore, if the weight values at the same position in different convolution kernels correspond to the same input data stream, the weight values at the same position in different convolution kernels can be arranged in the same row of the computing unit (PE) array.
[0054] In some possible implementations, in step 304, the sliding step size of the sliding window operation is determined according to the convolution step size of the dilated convolution operation. Figure 8 In the sliding window operation shown, the sliding step size is the same as the convolution step size.
[0055] In this embodiment, by adopting the above method, there is no need to frequently read across rows or columns of the input image data continuously stored in a single storage direction on the external memory at the same time, and there is no need to specially design the memory layout. A highly versatile and low-complexity dilated convolution acceleration calculation method can be implemented. In addition, there is no need to additionally implement the Im2col function in the computing platform, which saves hardware resources and computing power consumption.
[0056] Based on the same or similar technical concept, an embodiment of the present invention further provides a dilated convolution acceleration computing device, which includes: a logic control unit 91 and a computing unit (PE) array 211; wherein,
[0057] The logic control unit is configured to: decompose an R×S dilated convolution operation into S R×1 sub-dilated convolution operations, where R is a convolution kernel height and S is a convolution kernel width; for each R×1 sub-dilated convolution operation, cache multiple weight values in parallel to multiple computing units in a computing unit array; determine multiple input data streams corresponding to the multiple weight values, respectively, and input the multiple input data streams in parallel to multiple computing units;
[0058] The computing unit array is configured to: perform sliding window operations and multiplication operations based on cached weight values and received input data streams within multiple computing units, and perform accumulation operations between multiple computing units to output intermediate results of sub-dilated convolution operations; superimpose intermediate results of S R×1 sub-dilated convolution operations to obtain a convolution result of the dilated convolution operation.
[0059] In some possible implementations, the logic control unit 91 is configured to read required data from the input image data according to the dilation rate and convolution step size of the dilated convolution operation to splice together the input data stream corresponding to each weight value.
[0060] In some other possible implementations, the logic control unit 91 is configured to read multiple rows of data in the input image data according to the dilation rate and convolution step size of the dilated convolution operation to splice together the input data stream corresponding to each weight value.
[0061] In some possible implementations, the sliding step size of the sliding window operation is determined according to the convolution step size of the dilated convolution operation.
[0062] In one possible implementation, the computing unit array is configured to: superimpose intermediate results of S R×1 sub-dilated convolution operations in real time during the S R×1 sub-dilated convolution operations, or superimpose multiple intermediate results after the S R×1 sub-dilated convolution operations are completed.
[0063] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, since it is basically similar to the method embodiment, its description is simplified, and the relevant parts can be referred to the partial description of the method embodiment.
[0064] The device and method provided in the embodiments of the present application correspond one to one, and therefore, the device also has similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device will not be repeated here.
[0065] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.
[0066] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.
Claims
1. A method for accelerating calculation of dilated convolution, characterized in that: The method comprises: Decompose the R×S dilated convolution operation into S R×1 sub-dilated convolution operations, where R is the height of the convolution kernel and S is the width of the convolution kernel; For each R×1 sub-expanded convolution operation, multiple weight values are cached in parallel to multiple computing units in the computing unit array; Determine from the input image data a plurality of input data streams respectively corresponding to the plurality of weight values, and input the plurality of input data streams into the plurality of computing units in parallel; Within the plurality of computing units, each performs a sliding window operation and a multiplication operation based on the cached weight value and the received input data stream, and performs an accumulation operation between the plurality of computing units to output an intermediate result of the sub-dilated convolution operation; The intermediate results of the S R×1 sub-dilated convolution operations are superimposed to obtain a convolution result of the dilated convolution operation.
2. The method according to claim 1, characterized in that According to the dilation rate and convolution step size of the dilated convolution operation, required data is read from the input image data to splice and form an input data stream corresponding to each weight value.
3. The method according to claim 1, characterized in that According to the dilation rate and convolution step size of the dilated convolution operation, multiple rows of data in the input image data are read to splice and form an input data stream corresponding to each weight value.
4. The method according to claim 1, characterized in that: The sliding step size of the sliding window operation is determined according to the convolution step size of the dilated convolution operation.
5. The method according to claim 1, characterized in that Superimposing the intermediate results of the S R×1 sub-dilated convolution operations includes: superimposing the intermediate results of the S R×1 sub-dilated convolution operations in real time during the S R×1 sub-dilated convolution operations, or superimposing multiple intermediate results after the S R×1 sub-dilated convolution operations are completed.
6. A dilated convolution accelerated computing device, characterized in that: The device comprises: a logic control unit and a computing unit array; wherein, The logic control unit is configured to: decompose an R×S dilated convolution operation into S R×1 sub-dilated convolution operations, wherein R is a convolution kernel height and S is a convolution kernel width; for each R×1 sub-dilated convolution operation, cache multiple weight values in parallel to multiple computing units in the computing unit array; determine multiple input data streams corresponding to the multiple weight values from the input image data, and input the multiple input data streams in parallel to the multiple computing units; The computing unit array is configured to: perform sliding window operations and multiplication operations within the multiple computing units based on the cached weight values and the received input data streams, and perform accumulation operations between the multiple computing units to output intermediate results of the sub-expanded convolution operations; and superimpose the intermediate results of the S R×1 sub-expanded convolution operations to obtain the convolution result of the expanded convolution operation.
7. The device according to claim 6, characterized in that The logic control unit is configured to: According to the dilation rate and convolution step size of the dilated convolution operation, required data is read from the input image data to splice and form an input data stream corresponding to each weight value.
8. The device according to claim 6, characterized in that The logic control unit is configured to: According to the dilation rate and convolution step size of the dilated convolution operation, multiple rows of data in the input image data are read to splice and form an input data stream corresponding to each weight value.
9. The device according to claim 6, characterized in that The sliding step size of the sliding window operation is determined according to the convolution step size of the dilated convolution operation.
10. The device according to claim 6, characterized in that The computing unit array is configured to: superimpose intermediate results of the S R×1 sub-dilated convolution operations in real time during the S R×1 sub-dilated convolution operations, or superimpose multiple intermediate results after the S R×1 sub-dilated convolution operations are completed.