Target classification method based on asymmetric PAD convolution calculation to achieve continuous data update
By asymmetric filling and splicing in the height or width direction of the image data, the problem of target identification errors in continuous update data is solved, the computing efficiency and storage utilization rate are improved, and efficient target classification of continuous update data is achieved.
Patent Information
- Application Number
- CN202111471155.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-03
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-03
AI Technical Summary
In the continuously updated image data, the prior art is difficult to achieve target classification, resulting in target recognition errors, low computing efficiency and insufficient storage space.
Asymmetric PAD convolution calculation method is adopted to realize the target classification of continuous update data by asymmetric filling and splicing in the height or width direction of image data, reducing repeated calculations and improving calculation efficiency.
It realizes target classification without waiting for all data sampling to be completed, reducing the target recognition error rate, improving computing efficiency, and reducing storage requirements.
Smart Images

Figure CN114255399B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of artificial intelligence and deep learning, and in particular to a method for implementing data target classification based on convolution calculation. Background Art
[0002] Convolutional neural networks are increasingly being used in image recognition and target classification problems, but the input image of the convolutional neural network needs to contain the complete target in order to perform target classification.
[0003] In industries such as color sorting and scanning, objects are usually transported using some continuously updated method, such as tracks, guide rails, and natural drops. Objects are captured and photographed by fixed cameras during the transport process, and the objects are transported continuously and uninterruptedly. Since the camera captures and photographs at fixed time intervals, there will inevitably be objects that can only be photographed partially. In order to use convolutional neural networks to classify objects, a stitching algorithm must be used to stitch objects that only capture a portion of the image. The effectiveness of the stitching algorithm will directly affect the actual classification results of the object. For different categories of objects, the stitching algorithms are generally different, and it is difficult to find a unified method to perfectly stitch all types of objects.
[0004] When using a convolutional neural network to classify objects, the calculation result data of each convolution calculation layer needs to be stored for use by the next layer. Each target classification calculation requires a complete object image. For continuously updated data, the object data must be fully sampled before the object can be classified. If the object data is incomplete, the classification result will be incorrect and the target recognition will be wrong.
[0005] When performing convolution calculations on images or feature maps, in order to obtain outputs of special sizes, it is generally necessary to pad the input with data (PAD). The specific process is to first pad the input to expand the input size, and then perform convolution calculations to properly process the edges of the input feature map. The padding value is generally 0 padding or edge data padding. The number of paddings is related to the size of the convolution kernel, and the padding values will be distributed above and below or at the left and right ends of the input. If no padding is performed, the size of the output feature map must be smaller than the size of the input feature map, and the edge information of the input feature map will be lost. Appropriate padding can keep the size of the output feature map the same as the size of the input feature map, and retain the edge data information of the input feature map as much as possible.
[0006] In summary, for continuously updated data, because the data is always growing, if it is not processed in time, the computer storage space will be insufficient. However, the existing stitching algorithm must wait until all the object data are sampled before the object can be classified. If the object data is incomplete, the classification result will be incorrect, resulting in target recognition errors. Summary of the invention
[0007] The technical problem to be solved by the present invention is how to achieve target classification on continuously updated images, thereby solving the problem of target recognition errors caused by image splicing.
[0008] The present invention solves the above technical problems through the following technical means: a target classification method based on asymmetric PAD convolution calculation to achieve continuous data update, comprising the following steps:
[0009] Step S1: Acquire continuously updated image data;
[0010] Step S2: inputting the continuously updated image data into a pre-trained convolutional neural network model to obtain a classification result of the continuously updated image data;
[0011] The step S2 further comprises:
[0012] S21, continuously intercepting image areas of a set size from continuously updated image data to obtain a plurality of first images;
[0013] S22, performing asymmetric filling in the height or width direction of the first image, if asymmetric filling is performed in the height direction, symmetric filling is performed in the width direction, if asymmetric filling is performed in the width direction, symmetric filling is performed in the height direction, to obtain a plurality of second images;
[0014] S23, continuously splicing the plurality of second images through the unfilled edges in the height or width direction;
[0015] S24, the spliced image output is subjected to convolution processing to obtain target classification features;
[0016] S25, inputting the target classification features into a feature classifier to determine the classification result of the target graphic.
[0017] As a further optimized technical solution, in step S22, there are two asymmetric filling methods in the height and width directions, namely left edge filling, right edge filling, upper edge filling and lower edge filling.
[0018] As a further optimized technical solution, asymmetric filling in the height direction adopts either of the following two methods:
[0019] Continuously intercept image areas with a width of Win and a height of Hin from continuously updated image data to obtain several first images, perform symmetrical filling in the width direction of the first image, with the left and right filling widths of Pad1 respectively, fill the upper edge of the height of the first image, and do not fill the lower edge, and the upper edge filling height is 2*Pad0, thereby obtaining a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0. Since the real data of the second image is at the lower edge, the lower edge of the second image in the height direction is infinitely spliced, thereby achieving the target classification of continuously updated data;
[0020] Alternatively, the lower edge of the height of the first image is filled, and the upper edge is not filled, and the lower edge is filled with a height of 2*Pad0, so that a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0 can be obtained. Since the real data of the second image is at the upper edge, the lower edge of the second image in the height direction is infinitely spliced, thereby achieving the target classification of continuously updated data.
[0021] As a further optimized technical solution, asymmetric filling in the width direction adopts either of the following two methods:
[0022] Continuously intercept image areas with a width of Win and a height of Hin from continuously updated image data to obtain several first images, and perform symmetrical filling in the height direction of the first image. The upper and lower filling widths are Pad0 respectively. Fill the left edge of the width of the first image, and do not fill the right edge. The filling width of the left edge is 2*Pad1, thereby obtaining a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0. Since the real data of the second image is at the right edge, the right edge of the second image in the width direction is infinitely spliced, thereby achieving the target classification of continuously updated data;
[0023] Alternatively, image areas with a width of Win and a height of Hin are continuously captured from continuously updated image data to obtain several first images, which are symmetrically filled in the height direction of the first image, with the upper and lower filling widths being Pad0 respectively. The right edge of the width of the first image is filled, the left edge is not filled, and the right edge is filled with a width of 2*Pad1, thereby obtaining a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0. Since the real data of the second image is at the left edge, the right edge of the second image is infinitely spliced in the width direction, thereby achieving target classification of continuously updated data.
[0024] As a further optimized technical solution, step S22 specifically includes:
[0025] Assume that the input data is updated downward in the height direction, assume that HK is an odd number, and the step length of the convolution kernel in the height direction is 1. This scheme is also applied to data updates in other directions, even-numbered H, and convolution kernel step lengths of other values. Let HK = 2*M+1. Every time N rows of input feature map data are updated, an inference calculation is performed on the deep learning model. The height of the input layer of the complete deep learning model is recorded as HF, and the width is recorded as WF. Here, N is not limited by the above HK.
[0026] When N lines of input feature map data are updated, the currently cached HF+2*MN lines of data are concatenated with the updated N lines of data, totaling HF+2*M lines of data, which are used as the input layer of the deep learning model for calculation. At this time, the height of the next layer of the convolutional neural network is still HF, and the top 2*M lines of data of the cached HF+2*MN lines of data are regarded as the asymmetric PAD of this calculation.
[0027] The process of obtaining the cached HF+2*MN rows of data is as follows: After the current network model inference is completed, the bottom HF+2*MN rows of data of the current cached HF+2*M data are retained, and the top N rows of data are discarded to obtain the cached HF+2*MN rows of data. When the next batch of N rows of data is updated, they are spliced together with the currently retained HF+2*MN rows of data to form HF+2*M rows of new data, which are sent to the deep learning model. At this time, the top 2*M rows of data in the retained HF+2*MN rows of data constitute the asymmetric PAD of this operation.
[0028] As a further optimized technical solution, in step 24, if the input feature data updated each time is not enough to complete a complete convolution operation, the convolution operation is performed multiple times. The intermediate result of each convolution calculation is stored in a specific location for cache. The intermediate result of the convolution here refers to the result obtained by performing a convolution operation on the input data, which is also the input of the subsequent layers of the deep learning model. The intermediate result of the convolution calculation of the bottom HF-N rows of this calculation is cached. At the same time, for the input data, only the bottom 2*M rows of data are retained, and all the data above are discarded.
[0029] After the next batch of N rows of data is updated, the retained 2*M rows of input data are spliced together with the updated N rows of input data, and only the convolution operation of the bottom N+2*M rows of input feature maps of the new feature map is calculated to obtain N rows of new intermediate results of convolution operation. These N rows of new convolution operation intermediate results are spliced with the above-retained HF-N rows of convolution operation intermediate results to form a complete convolution operation intermediate result, and the calculation of subsequent layers is performed. After the complete network model composed of this batch of new data is inferred, the bottom HF-N rows of convolution calculation intermediate results and the bottom 2*M rows of input data are also retained for reuse in the subsequent calculation of the convolution results of the updated data.
[0030] The advantages of the present invention are: the present invention adopts an asymmetric filling method to fill in one of the edges in the height or width direction. Since the real data is on one edge, the data can be infinitely spliced on this edge, and there is no need to wait for all the object data to be sampled before identifying the object, thereby achieving target classification of continuously updated data and solving the problem of target recognition errors caused by image splicing. The method of caching and splicing intermediate results of convolution calculations proposed in the present invention can reduce repeated calculations in multiple convolutional neural network reasoning processes, reduce the amount of calculations in each reasoning process, and improve calculation efficiency. Asymmetric convolution is introduced in the prior art, that is, the height and width of the convolution kernel are not equal, and the present invention is an asymmetric PAD. PAD refers to the padding around the input feature map during the convolution operation. The purpose of filling PAD is to make the output feature map meet a certain size requirement. If PAD is not filled, the output feature map will not meet the requirements of a certain size. Figure 1 It is smaller than the input feature map, which is completely different from the asymmetric convolution. The asymmetric PAD method of the present invention is mentioned for the first time. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is an example diagram of symmetric filling in the prior art;
[0032] Figure 2 This is an example diagram of asymmetric filling in which the upper edge in the height direction is filled and the lower edge is not filled in an embodiment of the present invention;
[0033] Figure 3 This is an example diagram of asymmetric filling in which the lower edge in the height direction is filled and the upper edge is not filled in an embodiment of the present invention;
[0034] Figure 4 This is an example diagram of asymmetric filling in which the left edge in the width direction is filled and the right edge is not filled in an embodiment of the present invention;
[0035] Figure 5 This is an example diagram of asymmetric filling in which the right edge in the width direction is filled and the left edge is not filled in an embodiment of the present invention;
[0036] Figure 6 4 is a diagram of a single-channel convolution calculation process in an embodiment of the present invention. DETAILED DESCRIPTION
[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0038] The target classification method based on asymmetric PAD convolution calculation to realize continuous data updating provided by the present invention comprises the following steps:
[0039] Step S1: Acquire continuously updated image data;
[0040] Step S2: input the above-mentioned continuously updated image data into a pre-trained convolutional neural network model to obtain a classification result of the continuously updated image data.
[0041] The step S2 further comprises:
[0042] S21, continuously intercepting image areas of a set size from continuously updated image data to obtain a plurality of first images;
[0043] S22, performing asymmetric filling in the height or width direction of the first image, if asymmetric filling is performed in the height direction, symmetric filling is performed in the width direction, if asymmetric filling is performed in the width direction, symmetric filling is performed in the height direction, to obtain a plurality of second images;
[0044] S23, continuously splicing the plurality of second images through the unfilled edges in the height or width direction;
[0045] S24, the spliced image output is subjected to convolution processing to obtain target classification features;
[0046] S25, inputting the target classification features into a feature classifier to determine the classification result of the target graphic.
[0047] In step S22, there are two asymmetric filling methods in the height direction and the width direction respectively.
[0048] The following combination Figure 2 and Figure 3 Take asymmetric filling in the height direction as an example for explanation.
[0049] like Figure 2As shown, image areas with width Win and height Hin are continuously intercepted from the continuously updated image data to obtain several first images, which are symmetrically filled in the width direction of the first image, with the left and right filling widths being Pad1 respectively. The upper edge of the height of the first image is filled, and the lower edge is not filled. The upper edge filling height is 2*Pad0, thereby obtaining a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0. Since the real data of the second image is at the lower edge, the lower edge of the second image in the height direction can be infinitely spliced, thereby achieving the target classification of continuously updated data.
[0050] like Figure 3 As shown, the lower edge is filled asymmetrically in the height direction. Figure 2 The filling shown is different in that the filling is performed on the lower edge of the height of the first image, and the upper edge is not filled. The lower edge is filled with a height of 2*Pad0, so that a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0 can be obtained. However, since the real data of the second image is at the upper edge, the edges of the second image in the height direction can be infinitely spliced, thereby achieving the target classification of continuously updated data.
[0051] The following combination Figure 4 and Figure 5 The asymmetric filling in the width direction is taken as an example for explanation.
[0052] like Figure 4 As shown, image areas with a width of Win and a height of Hin are continuously intercepted from the continuously updated image data to obtain several first images, which are symmetrically filled in the height direction of the first image, with the upper and lower filling widths of Pad0 respectively. The left edge of the width of the first image is filled, and the right edge is not filled. The filling width of the left edge is 2*Pad1, thereby obtaining a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0. Since the real data of the second image is at the right edge, the right edge of the second image in the width direction can be infinitely spliced, thereby achieving the target classification of continuously updated data.
[0053] like Figure 5 As shown, image areas with a width of Win and a height of Hin are continuously intercepted from the continuously updated image data to obtain several first images, which are symmetrically filled in the height direction of the first image, with the upper and lower filling widths of Pad0 respectively. The right edge of the width of the first image is filled, the left edge is not filled, and the right edge is filled with a width of 2*Pad1, thereby obtaining a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0. Since the real data of the second image is at the left edge, the right edge of the second image can be infinitely spliced in the width direction, thereby achieving the target classification of continuously updated data.
[0054] In actual use, since data is constantly updated and hardware resources are limited, image data is calculated continuously. During calculation, each convolution layer needs to retain a portion of the edge data of the input features. Figure 2 , 3 For the filling cases of , 4, and 5, the lower edge data, upper edge data, right edge data, and left edge data are retained respectively, and are concatenated with the new data during calculation, and then the output of the current layer is calculated.
[0055] The single channel convolution calculation process is shown in Figure 6 , see Figure 6 In step S24, the spliced image output is subjected to convolution processing to obtain target classification features. Figure 2 Taking the asymmetric filling in the height direction as an example, it specifically includes:
[0056] The input features of the spliced image follow the FIFO working mode, where input feature data 3 represents the data retained from the last input feature, input feature data 6 represents the feature data that needs to be retained after this calculation, and input feature 3 will be discarded after this calculation is completed and will no longer be saved and used. Input feature data 3, 4, 5, and 6 are the input data for this convolution calculation. Input feature data 6, 7, 8, and 9 are the input data for the next convolution calculation, and input feature data 9, 10, 11, and 12 are the input data for the next convolution calculation, and so on. The input features are continuously updated upward in the height direction, and a convolution is calculated each time it is updated, and the output features are also continuously updated, thereby realizing convolution calculation while the data is continuously updated.
[0057] When data 4, 5, and 6 are the data updated this time, data 3 is the PAD of data 4, 5, and 6 in the height direction. The filling method is the same as Figure 2 The example is consistent with that of , and data 6 is retained as the PAD for the next convolution operation. When data 7, 8, and 9 are updated data, data 6 is the PAD of data 7, 8, and 9 in the height direction, and the filling method is the same as Figure 2 The example is consistent with that of , and data 9 is retained as the PAD for the next convolution operation. When data 10, 11, and 12 are updated data, data 9 is the PAD of data 10, 11, and 12 in the height direction, and the filling method is the same as Figure 2 The example is consistent with that of , and data 12 is retained as the PAD of the next convolution operation. In this way, the reuse of the input data retained last time can be achieved, and more information can be extracted from the continuously updated input data as much as possible.
[0058] In the above examples, data 1-12 refers to a block of data, which may include several rows of image data.
[0059] If the data updated each time is not enough to complete a complete convolution operation, the convolution operation can be performed multiple times. In the above example, if the deep learning model requires 10 pieces of data to form a complete input feature map, then under the calculation method proposed by the present invention, it is calculated as follows: first perform convolution operations of 3, 4, 5, and 6 to obtain intermediate results, which are stored in a specific location; when data 7, 8, and 9 are updated, discard data 3, 4, and 5, retain data 6, and perform convolution operations on data 6, 7, 8, and 9, and the intermediate results of the convolution calculation are stored in a specific location; when data 10, 11, and 12 are updated, discard data 6, 7, and 8, retain data 9, and perform convolution operations on data 9, 10, 11, and 12, and the intermediate results of the convolution calculation are stored in a specific location. At this point, the input layer of a complete deep learning model has been processed, and the previously temporarily stored intermediate layers are spliced together to form the intermediate layer results of the convolution operation of data 3-12, and the calculation of subsequent layers of the deep learning model continues. When data 13, 14, and 15 are updated, the intermediate results of the convolution operations of data 6, 7, 8, and 9, the intermediate results of the convolution operations of data 9, 10, 11, and 12, and the intermediate results of the convolution operations of data 12, 13, 14, and 15 are spliced together to form the intermediate results of the convolution operations of data 6-15. Since the intermediate results of the convolution operations of data 6, 7, 8, and 9, and the intermediate results of the convolution operations of data 9, 10, 11, and 12 have been calculated and temporarily stored in a specific location, there is no need to repeat the calculation and they can be directly spliced. When the subsequent data is updated, the splicing of the input data and the splicing of the intermediate convolution layer calculation results are performed in a similar manner. In the above example, this method can reduce the repeated convolution calculations of data 6-9 and data 9-12 when the data is continuously updated. In this way, the amount of convolution calculations when the data is continuously updated can be greatly reduced, and the efficiency of the convolution calculations when the data is continuously updated can be improved.
[0060] The height of the convolution kernel is HK and the width is WK. The asymmetric filling method of step S22 is specifically described below. Step S22 performs asymmetric filling in the height or width direction of the first image. If asymmetric filling is performed in the height direction, symmetric filling is performed in the width direction. If asymmetric filling is performed in the width direction, symmetric filling is performed in the height direction, and a plurality of second images are obtained; specifically, the following steps are performed:
[0061] Assume that the input data is updated downward in the height direction, assume that HK is an odd number, and the step length of the convolution kernel in the height direction is 1. This scheme can also be applied to data updates in other directions, even numbers of H, and convolution kernel step lengths of other values. Let HK = 2*M+1. If a symmetric PAD is used, to keep the size of the output feature map unchanged, M rows of data should be filled above and below the input feature map as PAD.
[0062] In this example, every time N rows of input feature map data are updated, an inference calculation is performed on the deep learning model. The height of the complete deep learning model input layer is recorded as HF, and the width is recorded as WF. N here is not limited by the above HK.
[0063] When N lines of input feature map data are updated, the currently cached HF+2*MN lines of data are concatenated with the updated N lines of data, totaling HF+2*M lines of data, which are used as the input layer of the deep learning model for calculation. At this time, the height of the next layer of the convolutional neural network is still HF. The top 2*M lines of data of the cached HF+2*MN lines of data can be regarded as the asymmetric PAD of this calculation. The origin of the HF+2*MN lines of data will be described below.
[0064] After the current network model inference is completed, the bottom HF+2*MN rows of the currently cached HF+2*M data are retained, and the top N rows of data are discarded. When the next batch of N rows of data are updated, they are spliced together with the HF+2*MN rows of data retained this time to form HF+2*M rows of new data, which are sent to the deep learning model. At this time, the top 2*M rows of data in the retained HF+2*MN rows of data constitute the asymmetric PAD of this operation. This solution realizes the reuse of input data and avoids data movement; at the same time, since the top 2*M rows of data filled are valid input data each time the convolution result is calculated, more information can be extracted from the input data.
[0065] As an optimization scheme, the intermediate convolution results calculated by the above scheme are cached. The intermediate convolution results here refer to the results obtained by performing convolution operations on the input data, and are also the inputs of the subsequent layers of the deep learning model. The intermediate convolution calculation results of the bottom HF-N rows of this calculation are cached. At the same time, for the input data, only the bottom 2*M rows of data are retained, and all the data above are discarded. After the next batch of N rows of data is updated, the retained 2*M rows of input data are spliced with the updated N rows of input data, and only the convolution operation of the bottom N+2*M rows of input feature maps of the new feature map is calculated to obtain the new intermediate results of N rows of convolution operations. These N rows of new convolution operation intermediate results are spliced with the above-retained HF-N rows of convolution operation intermediate results to form a complete convolution operation intermediate result, and the calculations of subsequent layers are performed. After the complete network model composed of this batch of new data is inferred, the bottom HF-N rows of convolution calculation intermediate results and the bottom 2*M rows of input data are also retained for reuse in the subsequent calculation of the convolution results of the updated data. This method can realize the reuse of convolution operation results. When calculating the inference result of updated data once, it can reduce the recalculation of the intermediate results of HF-N row convolution calculation, thereby improving the computational efficiency of the convolutional neural network during continuous data update.
[0066] In a convolutional neural network, each layer starts calculating when there is enough data to complete a calculation. If there are multiple pool layers in the network, feature maps of multiple scales of the image are obtained. The calculation of the pool of this layer is started only when there is enough data for calculation in the pool layer input buffer. If the current pool layer input data cannot satisfy the pool calculation operation, the calculation of the current layer and subsequent layers is stopped, and the data output of the previous layer is waited for.
[0067] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A target classification method based on asymmetric PAD convolution calculation to achieve continuous data update, characterized by: The following steps are involved: Step S1: Acquire continuously updated image data; Step S2: inputting the continuously updated image data into a pre-trained convolutional neural network model to obtain a classification result of the continuously updated image data; The step S2 further comprises: S21, continuously intercepting image areas of a set size from continuously updated image data to obtain a plurality of first images; S22, performing asymmetric filling in the height or width direction of the first image, if asymmetric filling is performed in the height direction, symmetric filling is performed in the width direction, if asymmetric filling is performed in the width direction, symmetric filling is performed in the height direction, to obtain a plurality of second images; S23, continuously splicing the plurality of second images through the unfilled edges in the height or width direction; S24, the spliced image output is subjected to convolution processing to obtain target classification features; S25, inputting the target classification feature into a feature classifier to determine the classification result of the target graphic; In the step S22, there are two asymmetric filling methods in the height and width directions, namely left edge filling, right edge filling, upper edge filling and lower edge filling; Asymmetric filling in height direction uses either of the following two methods: Continuously intercept image areas with a width of Win and a height of Hin from continuously updated image data to obtain several first images, perform symmetrical filling in the width direction of the first image, with the left and right filling widths of Pad1 respectively, fill the upper edge of the height of the first image, and do not fill the lower edge, and the upper edge filling height is 2*Pad0, thereby obtaining a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0. Since the real data of the second image is at the lower edge, the lower edge of the second image in the height direction is infinitely spliced, thereby achieving the target classification of continuously updated data; Alternatively, the lower edge of the height of the first image is filled, and the upper edge is not filled. The lower edge is filled with a height of 2*Pad0, so that a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0 can be obtained. Since the real data of the second image is at the upper edge, the lower edge of the second image in the height direction is infinitely spliced, thereby achieving the target classification of continuously updated data; Asymmetric filling in the width direction uses either of the following two methods: Continuously intercept image areas with a width of Win and a height of Hin from continuously updated image data to obtain several first images, and perform symmetrical filling in the height direction of the first image. The upper and lower filling widths are Pad0 respectively. Fill the left edge of the width of the first image, and do not fill the right edge. The filling width of the left edge is 2*Pad1, thereby obtaining a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0. Since the real data of the second image is at the right edge, the right edge of the second image in the width direction is infinitely spliced, thereby achieving the target classification of continuously updated data; Alternatively, image areas with a width of Win and a height of Hin are continuously captured from continuously updated image data to obtain several first images, which are symmetrically filled in the height direction of the first image, with the upper and lower filling widths being Pad0 respectively. The right edge of the width of the first image is filled, the left edge is not filled, and the right edge is filled with a width of 2*Pad1, thereby obtaining a second image with a width of Win+2*Pad1 and a height of Win+2*Pad0. Since the real data of the second image is at the left edge, the right edge of the second image is infinitely spliced in the width direction, thereby achieving target classification of continuously updated data.
2. A target classification method for realizing continuous data update based on asymmetric PAD convolution calculation as claimed in claim 1, characterized in that: In step 24, if the input feature data updated each time is not sufficient to complete a complete convolution operation, the convolution operation is performed multiple times.
3. A target classification method for realizing continuous data update based on asymmetric PAD convolution calculation as claimed in claim 2, characterized in that: The intermediate result of each convolution calculation is stored in a specific location for cache. The intermediate result of convolution here refers to the result obtained by performing convolution operation on the input data, which is also the input of the subsequent layers of the deep learning model.
4. A target classification method for realizing continuous data update based on asymmetric PAD convolution calculation as claimed in claim 1, characterized in that: The step S22 specifically includes: Assume that the input data is updated downward in the height direction, the convolution kernel height is HK, and the width is WK. Assume that HK is an odd number, and the step length in the height direction of the convolution kernel is 1. This scheme is also applied to data updates in other directions, even HK, and convolution kernel step lengths of other values. Let HK=2*M+1. Every time N rows of input feature map data are updated, an inference calculation is performed on the deep learning model. The input layer height of the complete deep learning model is recorded as HF, and the width is recorded as WF. N here is not subject to the above-mentioned HK restrictions. When N lines of input feature map data are updated, the currently cached HF+2*MN lines of data are concatenated with the updated N lines of data, totaling HF+2*M lines of data, which are used as the input layer of the deep learning model for calculation. At this time, the height of the next layer of the convolutional neural network is still HF, and the top 2*M lines of data of the cached HF+2*MN lines of data are regarded as the asymmetric PAD of this calculation.
5. A target classification method for realizing continuous data update based on asymmetric PAD convolution calculation as claimed in claim 4, characterized in that: The process of obtaining the cached HF+2*MN rows of data is as follows: when the network model inference is completed, the bottom HF+2*MN rows of data of the currently cached HF+2*M data are retained, and the top N rows of data are discarded to obtain the cached HF+2*MN rows of data. When the next batch of N rows of data are updated, they are spliced together with the retained HF+2*MN rows of data to form HF+2*M rows of new data, which are sent to the deep learning model. At this time, the top 2*M rows of data in the retained HF+2*MN rows of data constitute the asymmetric PAD of this operation.
6. A target classification method for realizing continuous data update based on asymmetric PAD convolution calculation as claimed in claim 2 or 3, characterized in that: Assume that the input data is updated downward in the height direction, the convolution kernel height is HK, and the width is WK. Assume that HK is an odd number, and the step length in the height direction of the convolution kernel is 1. This scheme is also applied to data updates in other directions, even HK, and convolution kernel step lengths of other values. Let HK=2*M+1. Every time N rows of input feature map data are updated, an inference calculation is performed on the deep learning model. The input layer height of the complete deep learning model is recorded as HF, and the width is recorded as WF. N here is not subject to the above-mentioned HK restrictions. After each update of N rows of input data, the intermediate results of the convolution calculation of the bottom HF-N rows of this calculation are cached. At the same time, for the input data, only the bottom 2*M rows of data are retained, and all the data above are discarded.
7. A target classification method for realizing continuous data update based on asymmetric PAD convolution calculation as claimed in claim 6, characterized in that: After the next batch of N rows of data is updated, the retained 2*M rows of input data are spliced together with the updated N rows of input data, and only the convolution operation of the bottom N+2*M rows of input feature maps of the new feature map is calculated to obtain N rows of new intermediate results of convolution operation. These N rows of new convolution operation intermediate results are spliced with the above-retained HF-N rows of convolution operation intermediate results to form a complete convolution operation intermediate result, and the calculation of subsequent layers is performed. After the complete network model composed of this batch of new data is inferred, the bottom HF-N rows of convolution calculation intermediate results and the bottom 2*M rows of input data are also retained for reuse in the subsequent calculation of the convolution results of the updated data.
Citation Information
Patent Citations
Small-size traffic sign recognition method based on YOLOV3 and asymmetric convolution
CN111274980A
Traffic sign recognition method based on lightweight neural network
CN113537138A
Data processing method and processing circuit
US20210082082A1
