Video object detection algorithm based on improved CenterNet backbone network

Abstract

The present invention discloses a video target detection algorithm based on an improved CenterNet backbone network, comprising the following steps: data set acquisition; construction of a training data set; image feature extraction; image feature fusion; target detection result output. The target detection algorithm of the CenterNet backbone network improved by the present invention includes replacing the DLA34 backbone network with ADS-DLA34. In the ADS-DLA34 network, soft pooling replaces the traditional pooling method of downsampling of the DLA34 network, deformable convolution replaces the traditional 2D convolution of upsampling, and an attention mechanism is combined in the network. The improved network reduces the loss of target feature information, enhances the learning ability of deformed occluded targets, and the network pays more attention to the key information in the image, and enhances the ability of model feature extraction and fusion. Thereby, the performance of the target detection algorithm is improved from the perspective of improving the model structure, which not only ensures the target detection speed, but also improves the target detection accuracy, and enhances the detection and recognition capabilities of target objects in different fields, especially in videos.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, specifically to the field of target detection, and in particular to an ADS-DLA34 target detection algorithm for improving CenterNet backbone network DLA34. Background Art

[0002] As an important research field in computer vision, target detection has a wide range of research directions. Among them, pedestrian detection and tracking in traffic and surveillance videos is one of the important research directions. Computer vision is the study of visual perception, which processes and analyzes the collected images or video image sequences to receive information in the scene. Computer vision mainly includes target detection, target tracking, image processing, target recognition, posture analysis and other aspects. For example, in target detection, it is difficult to detect and track some obstructed pedestrians in some crowded scenes with human eyes alone, and the cost of manual detection is relatively high and the speed is slow. With the development of intelligent technology, the methods in computer vision can supplement and improve the deficiencies of human visual ability, thereby helping humans to better complete various tasks.

[0003] In recent years, deep learning technology has developed rapidly, and convolutional neural networks have been widely used in the field of target detection and tracking. The target detection and tracking algorithms based on convolutional neural networks are superior to traditional methods in terms of detection accuracy and tracking precision. However, there are still some difficult problems. For example, when detecting pedestrian targets in videos, the morphological changes caused by occlusion lead to false detection and missed detection; when extracting target features, the feature information is not extracted sufficiently, which affects the target detection accuracy.

[0004] With the rapid development of deep learning technology, mainstream deep learning target detection algorithms are divided into two categories: anchor-based and anchor-free. The anchor-based method is to preset some rectangular boxes of a certain size on the input image, but there will be some problems. For example, if there are too many preset rectangular boxes, the amount of calculation will increase, resulting in a slower speed.

[0005] CenterNet is a new anchor-free deep neural network recognition algorithm. CenterNet detects objects based on the center point of the detection box. It only requires one key point to locate the object, so there is only one branch. The network structure is simpler, reducing training and inference prediction time, thereby improving detection accuracy and speed. Summary of the invention

[0006] The purpose of the present invention is to provide a target detection algorithm with higher accuracy and faster speed to solve some problems existing in the prior art.

[0007] To achieve the above object, the present invention adopts a technical solution of improving the ADS-DLA34 target detection algorithm of CenterNet backbone network DLA34, comprising the following steps:

[0008] Step S1: Dataset acquisition. The acquired data set includes a number of images of the target to be detected. The present invention acquires open source data set images.

[0009] Step S2: Construct the ADS-DLA34 training dataset of the improved CenterNet backbone network DLA34.

[0010] Step S3: Image feature extraction, wherein the image feature extraction is performed by the ADS-DLA34 network using a small step downsampling rate to extract target features between different levels.

[0011] Step S4: Image feature fusion. The image feature fusion is that the DLA network fuses the features between different stages through iterative deep aggregation, and the layered deep aggregation fuses the basic modules between different stages.

[0012] Step S5: Output of target detection results. The detection result output is to output the attention feature map with feature weights through the attention mechanism module after feature extraction and feature fusion. Three network layers are added at the end of the improved CenterNet backbone network to output the target detection prediction results.

[0013] In the above scheme, the step S2 constructs the open source dataset images of the training dataset of the CenterNet network, including data enhancement of the dataset images. Data enhancement includes random flipping, random scaling, cropping and color jittering, and also includes adding annotation information corresponding to each image. The annotation information includes picture information, corresponding annotation location information, and category information.

[0014] In the above scheme, the image feature extraction in step S3 includes the preliminary feature extraction of the input image through the base layer and the first layer. After the preliminary feature extraction, the output size of the image is consistent with the original image. Subsequently, the output of the first layer is passed to the second layer, and the downsampling operation is performed until the final fifth layer, and the extraction of target features between different layers is achieved by using a small step downsampling rate. The downsampling operation in the ADS-DLA34 network includes replacing the traditional downsampling pooling method with soft pooling, and fusing the features between different stages through iterative deep aggregation during the sampling process. The basic modules between different stages are fused by layered deep aggregation to achieve the combination of semantic and spatial features.

[0015] In the above scheme, the image feature fusion in step S4 is performed after downsampling and feature extraction. The ADS-DLA34 of the improved CenterNet backbone network DLA34 uses the DLAUP module in the multi-level fusion network to achieve network upsampling. In the upsampling process, deformable convolution is used to replace the upsampling traditional 2D convolution to expand the target receptive field and enhance the adaptability and learning ability to deformed and occluded targets. Secondly, the IDAUP module is used to complete the multi-level fusion in upsampling.

[0016] In the above scheme, after a series of upsampling and downsampling operations, the size of the output feature map is the same as the size of the output feature map after the second layer. In the step S5, after the image feature extraction and fusion, the target detection result is output, and the network is combined with the attention mechanism. By improving the CenterNet backbone network ADS-DLA34, three branches are added to output the target detection prediction results, namely the center point heat map (Heatmap), the target center point offset (Offset) and the width and height (Weight & Height) of the target. Among them, the Heatmap branch is responsible for outputting the position of the center points of targets of different categories; the Offset branch is responsible for refining the output of the Heatmap to improve the positioning accuracy; the Weight & Height branch is responsible for predicting the width and height of the detection box centered on the key point. The losses of the three branches include heat map loss, center point offset loss and center point width and height loss. The total loss function is L det , the formula is L det =L k +λ size L size +λ off L off , where L K is the heat map prediction loss, L off is the center point bias loss, L size is the center point width and height loss, where λ size , off are the weights of different loss functions respectively.

[0017] In the above scheme, the step S5 outputs the target detection prediction result and decodes the prediction result. Decoding is to obtain intuitive detection box information according to the output of the network. Then encoding is performed to convert the detection box information into network output information to facilitate the solution of the loss function. In addition, the prediction results must be screened, including the method of non-maximum suppression processing of Heatmap prediction, and maximum pooling is performed through 3×3 convolution, retaining the largest value in the neighborhood, and setting the rest to 0.

[0018] In summary, compared with some existing target detection algorithms, the present invention has the following beneficial effects: based on the video target detection algorithm of the improved CenterNet backbone network, this method adopts the frameless method to improve the ADS-DLA34 algorithm of the CenterNet backbone network DLA34. This method replaces the traditional pooling method of the DLA34 network with soft pooling to reduce the loss of target feature information; deformable convolution replaces the upsampling traditional 2D convolution to enhance the network's learning ability for occluded and deformed targets and obtain a larger receptive field; finally, the attention mechanism is combined in the network to make the network pay more attention to the key information in the input image and reduce the attention to other irrelevant information. It further improves the accuracy and speed of target detection, enhances the detection and recognition capabilities of human targets in dataset images and surveillance videos, and improves the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a target detection flow chart of the improved CenterNet backbone network algorithm in the present invention.

[0020] Figure 2 This is a structural model diagram of the ADS-DLA34 of the improved CenterNet backbone network DLA34 in the present invention.

[0021] Figure 3 This is a schematic diagram of the ADS-DLA34 network input in the present invention after downsampling and aggregating features and then upsampling.

[0022] Figure 4 It is the target detection algorithm output parameter of the backbone network ADS-DLA34 in the present invention.

[0023] Figure 5 This is a process diagram of target detection prediction results in the present invention. Specific implementation plan

[0024] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0025] In this embodiment, combined with Figure 1 , provides a video target detection algorithm based on an improved CenterNet backbone network, the method comprising the following steps:

[0026] Step S1: Dataset acquisition. The acquired data set includes a number of images of the target to be detected. The present invention acquires open source data set images.

[0027] Step S2: Construct a training dataset of ADS-DLA34 for the improved CenterNet backbone network DLA34.

[0028] Step S3: Image feature extraction, wherein the image feature extraction is performed by the ADS-DLA34 network using a small step downsampling rate to extract target features between different levels.

[0029] Step S4: Image feature fusion. The image feature fusion is that the DLA network fuses the features between different stages through iterative deep aggregation, and the layered deep aggregation fuses the basic modules between different stages.

[0030] Step S5: Output of target detection results. The detection result output is to output the attention feature map with feature weights through the attention mechanism module after feature extraction and feature fusion. At the end of the improved CenterNet backbone network, three network layers are added to output the target detection prediction results, and the decoding-training-encoding process is performed to calculate the loss function for model updating.

[0031] Furthermore, in this embodiment, the step S2 of constructing the training data set of ADS-DLA34 of the improved CenterNet backbone network DLA34 includes data enhancement of the data set images. Data enhancement includes random flipping, random scaling, cropping and color jittering, and also includes adding annotation information corresponding to each image. The annotation information includes picture information, corresponding annotation location information, and category information. Since the collected images are open source images, the annotation information of the data set does not need to be manually annotated.

[0032] Furthermore, in this embodiment, the step S3 image feature extraction is combined with Figure 2 The ADS-DLA34 network structure first passes through base_layer and level_1 to extract the initial features of the input image F∈H×W, where H is the height of the input image and W is the width of the input image. After passing through base_layer and level_1, the output size is consistent with the original image. Secondly, the output of level_1 is passed to the level_2 layer, which is first downsampled and output Then level_2 is passed to level_3 until the final level_5 layer. The feature map size of the F3-F5 downsampling output is: The output of the first layer is passed to the second layer, and then downsampled to the final fifth layer. The downsampling rate is stepped down to extract target features between different layers. The downsampling operation includes replacing the traditional downsampling pooling method with soft pooling to reduce the loss of target feature information. In the downsampling process, the features between different stages are fused through iterative deep aggregation, and the basic modules between different stages are fused through layered deep aggregation to achieve the combination of semantic and spatial features.

[0033] Furthermore, in the present embodiment, the image feature fusion in step S4 is that the DLA network fuses the features between different stages through iterative deep aggregation, and the basic modules between different stages are fused through layered deep aggregation. After downsampling and feature extraction, the ADS-DLA34 of the improved CenterNet backbone network DLA34 adopts the DLAUP module in the multi-level fusion network to realize network upsampling. In the upsampling process, deformable convolution is used instead of the traditional 2D convolution for upsampling to expand the target receptive field and enhance the adaptability and learning ability to deformed and occluded targets. Secondly, the IDAUP module is used to complete the multi-level fusion in upsampling. Figure 3 Schematic diagram of input after downsampling, feature extraction and aggregation, and then upsampling.

[0034] Furthermore, in this embodiment, the target detection result output in step S5 is combined with the attention mechanism in the network after the image feature extraction and fusion, and the target detection prediction result is output by adding three branches by improving the CenterNet backbone network ADS-DLA34. Figure 4 , are the center point heatmap (Heatmap), target center point offset (Offset) and target width and height (Weight & Height), which are the output parameters of the network. Among them, the Heatmap branch is responsible for outputting the location of the center points of different categories of targets; the Offset branch is responsible for refining the output of the Heatmap to improve the positioning accuracy; the Weight & Height branch is responsible for predicting the width and height of the detection box centered on the key point. The losses of the three branches include heatmap loss, center point offset loss and center point width and height loss. The total loss function is L det , the formula is L det =L k +λ size L size +λ off L off , where L K is the heat map prediction loss, L off is the center point bias loss, L size is the center point width and height loss, where λ size , offare the weights of different loss functions, which are 0.1 and 1 respectively.

[0035] Where is the loss caused by the error between the center points predicted by the heat map:

[0036]

[0037] Where N is the number of objects in the image. is an estimate of the heat map, and α and β are hyperparameters of the heat map loss function.

[0038] Among them, the center point bias loss is:

[0039]

[0040] Among them, p is the coordinate of the center point of the detection box, R is the scaling factor, is the coordinate of the center point after downsampling and rounding down. The error caused by the center point is: is the deviation of the model's prediction for the center point of each target.

[0041] Among them, the center point width and height loss is:

[0042]

[0043] Where S k is the actual center point width and height, is the predicted width and height.

[0044] Furthermore, in the above scheme, the step S5 outputs the target detection prediction result and decodes the prediction result. Decoding is to obtain intuitive detection box information according to the output of the network. In addition, the prediction results are screened, including the method of non-maximum suppression processing of Heatmap prediction, and maximum pooling is performed through 3×3 convolution, retaining the largest value in the neighborhood, and setting the rest to 0. According to the feature points in the Heatmap prediction, the index is obtained to further obtain the size of the prediction box. Combined with Figure 5 , decode the feature points in the image and offset the center point; then use the center point to calculate the coordinates of the prediction box and draw the prediction box. Then perform the training-encoding process, that is, convert the detection box information into network output information, which is convenient for solving the loss function. Get the center point of the real target box, calculate the radius R of the Gaussian circle according to the size of the predicted box and the size of the real box; set the center value of the real box in the Heatmap feature map to 1, then use the center point as the center of the circle with a radius of R, and finally get the Heatmap feature map. Similarly, map the offset information and width and height information of the real box to the Offset feature map and the Height&Width feature map. Finally, train the network and calculate the loss function based on the obtained data for model update.

[0045] The present invention adopts a frameless method and is based on an improved video target detection algorithm for the CenterNet backbone network DLA34. In the DLA34 network, the method replaces the traditional pooling method of downsampling with soft pooling, replaces the traditional 2D convolution of upsampling with deformable convolution, and combines the attention mechanism in the network. After improving the CenterNet backbone network DLA34 network, the algorithm's target detection capability is enhanced, especially the feature extraction capability based on video target detection, while ensuring the detection speed, the detection accuracy is improved.

[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be understood by those skilled in the art that the above embodiments and descriptions are only intended to illustrate the principles of the present invention. The embodiments of the present invention may have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video target detection method based on an improved CenterNet backbone network, characterized in that: The method comprises the following steps: Step S1: data set acquisition, the acquired data set includes a number of images of the target to be detected, and the present invention acquires open source data set images; Step S2: construct a training data set of ADS-DLA34 for improving the CenterNet backbone network DLA34; Step S3: Image feature extraction, wherein the image feature extraction is that the ADS-DLA34 network uses a small step down sampling rate to extract target features between different levels; The image feature extraction network is improved based on the DLA34 network, including the initial feature extraction of the input image through the base layer and the first layer, passing the output of the first layer to the second layer, and then downsampling until the final fifth layer. The target feature extraction between different layers is realized by using a small step downsampling rate. The downsampling operation in the ADS-DLA34 feature extraction network replaces the traditional pooling method of the DLA34 network with soft pooling, and fuses the features between different stages through iterative deep aggregation during the sampling process. The layered deep aggregation fuses the basic modules between different stages to achieve the combination of semantic and spatial features. Step S4: image feature fusion, wherein the image feature fusion is that the DLA network fuses features between different stages through iterative deep aggregation, and the layered deep aggregation fuses basic modules between different stages; The image feature fusion is that after downsampling and feature extraction, the ADS-DLA34 of the improved CenterNet backbone network DLA34 adopts the DLAUP module in the multi-level fusion network to realize the upsampling of the network, and uses deformable convolution to replace the traditional 2D convolution in the upsampling process, and uses the IDAUP module to complete the multi-level fusion in the upsampling; Step S5: Output of target detection results. The detection result output is to output the attention feature map with feature weights through the attention mechanism module after feature extraction and feature fusion. Three network layers are added at the end of the improved CenterNet backbone network to output the target detection prediction results.

2. The video target detection method based on the improved CenterNet backbone network according to claim 1 is characterized in that: The step S2 constructs an open source dataset image of the training dataset of the CenterNet network, including data enhancement of the dataset image; the data enhancement includes random flipping, random scaling, cropping and color jittering, and also includes adding annotation information corresponding to each image; The annotation information includes picture information, corresponding annotation location information, and category information.

3. The video target detection method based on the improved CenterNet backbone network according to claim 1 is characterized in that: The target detection result output in step S5 is output after the image feature extraction and fusion, combined with the attention mechanism, by improving the CenterNet backbone network ADS-DLA34 and adding three parallel heads to output the target detection prediction result; the three output branches are the center point heat map (Heatmap), the target center point offset (Offset) and the width and height (Weight & Height) of the target.

4. The video target detection method based on the improved CenterNet backbone network according to claim 1 is characterized in that: The losses of the three branches include heat map loss, center point bias loss and center point width and height loss. The total loss function is L det , the formula is L det =L k +λ size L size +λ off L off , where L K is the heat map prediction loss, L off is the center point bias loss, L size is the center point width and height loss, where λ size , off are the weights of different loss functions respectively.

5. The video target detection method based on the improved CenterNet backbone network according to claim 1 is characterized in that: The step S5 outputs the target detection prediction result to screen the prediction result, and also includes a method for non-maximum suppression processing of Heatmap prediction, performing maximum pooling through 3×3 convolution, retaining the largest value in the neighborhood, and setting the rest to 0.

Images