A lightweight object detection system without prior boxes
By using a lightweight non-priori box system in the target detection network, combined with MobileNet3 and spatial pyramid hollow convolutional structure, the problem of insufficient utilization of scale information in the existing technology is solved, and efficient target detection effect is achieved, which significantly improves the mAP and FPS.
Patent Information
- Application Number
- CN202210205386.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-02
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2042-03-02
AI Technical Summary
The existing target detection network has insufficient utilization of scale information when detecting targets of different sizes, resulting in poor detection results. In particular, the anchor-based network is sensitive to the preset anchor size, which affects the detection results.
Using a lightweight, non-prior box object detection system, combined with MobileNet3 as the skeleton structure, the spatial pyramid cavity convolution structure is used to optimize feature extraction at the neck, and the ECA module is used to enhance feature attention, avoid repeated calculations, reduce model parameters, and realize Effective utilization of multi-scale features.
The average accuracy of object detection mAP and FPS processing frames per second are improved, which significantly improves the detection effect, especially on the 2007 VOC dataset to reach 132.8 FPS and 69.89% mAP, which is better than the traditional method.
Smart Images

Figure CN114692826B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of deep neural networks and target detection, and in particular to a lightweight target detection system without a priori frames. Background Art
[0002] With the continuous update of the YOLO series of target detection networks, such as YOLO4, YOLO5 and YOLOx, which continuously improve the mAP of target detection based on pre-selected boxes, the research focus in the field has gradually shifted from the anchor based network structure dedicated to improving mAP to the anchor free target detection network. CenterNet (Zhou, Xingyi and Wang, Dequan and Krahenbuhl, Philipp, "Objects as Points," in arXiv preprint arXiv:1904.07850, 2019.) converts the detection of the target box into the prediction of the center point and width and height. It only uses one feature layer, and uses three upsampling convolutions in the neck structure to obtain a (128, 128) feature layer, and then uses a 1*1 convolution kernel to predict the heat map, center point and width and height respectively. However, the neck structure of CenterNet only uses three upsampling, and it is unable to use multi-scale information to improve the detection effect of targets of different sizes. The YOLOF network (Chen, Qiang and Wang, Yingming and Yang, Tong and Zhang, Xiangyu and Cheng, Jian and Sun, Jian, “You Only Look One-level Feature,” in IEEE Conference on Computer Vision and Pattern Recognition, 2021.), which was made public at the IEEE Conference on Computer Vision and Pattern Recognition in 2021, achieved a higher mAP while using the same feature layer for prediction. In the neck structure of YOLOF, four dilated convolutions with different expansion coefficients are connected in series, so that the features used for head structure prediction contain multi-scale information, thereby improving the mAP of target detection. However, YOLOF is still an anchor based target detection network, and using a set anchor size for targets of special sizes will affect the detection results. Summary of the invention
[0003] In order to solve the above technical problems, the present invention proposes a lightweight object detection system without a priori frame.
[0004] The present invention is achieved by at least one of the following technical solutions.
[0005] A lightweight object detection system without prior frames, including a feature extraction network and a prediction module.
[0006] The feature extraction network is a deep convolutional network, which uses the deep convolutional network to extract features from the input image;
[0007] The prediction module includes a target detection network model, uses a spatial pyramid hole convolution structure in the neck structure of the target detection network model to optimize the target detection task, and finally outputs a prediction result.
[0008] In the neck structure of the target detection network model, a spatial pyramid hole convolution structure is used to optimize the target detection task, making the features more suitable for the target detection task and thus improving the average accuracy mAP and the number of frames per second FPS of target detection, and finally outputting the prediction results.
[0009] Furthermore, the target detection network model is a network without a priori box, and the detection result is not constrained by a pre-set priori box, which has better detection results for position objects.
[0010] Furthermore, the feature extraction network uses Mobilenet3.
[0011] Furthermore, MobileNet3 uses a*a convolution to increase feature dimension, uses depthwise separable convolution instead of standard convolution to reduce model parameters, and introduces a compression excitation module.
[0012] Furthermore, the input of the prediction module is a feature map output by a feature extraction network, and a spatial pyramid atrous convolution structure is used in the neck structure of the target detection network model.
[0013] Furthermore, the process of the spatial pyramid atrous convolution structure is as follows: first, the dimension is increased by a*a convolution, and then atrous convolutions with different expansion coefficients are connected in parallel to obtain receptive fields of different sizes. Then, the features of different expansion coefficients and the input of the prediction module are stacked and enhanced by the ECA module, and finally the prediction result is obtained through several convolution layers.
[0014] Furthermore, the parallel atrous convolutions with different expansion coefficients are optimized on the encoder of the YOLOF network.
[0015] Furthermore, the encoder structure of YOLOF is to connect multiple dilation coefficients in series, and each dilation coefficient is preceded and followed by a standard convolution of a*a to reduce the number of channels and then increase the number of channels.
[0016] Furthermore, the ECA module assigns each feature layer a weight of 0-1, and uses the ECA module to replace the fully connected layer calculation weights used by the SE module with one-dimensional convolution calculation weights.
[0017] Furthermore, the deep convolutional network uses ShuffleNet, ResNet, and EfficientNet.
[0018] Compared with the prior art, the present invention has the following beneficial effects:
[0019] The invention discloses a lightweight target detection system without prior frame, which selects MobileNet3 as a skeleton structure; proposes a neck structure capable of combining multi-scale features, and the neck structure avoids repeated calculations through a spatial pyramid structure, and uses an ECA module to enhance the attention of the model; uses the head structure of a CenterNet network to realize anchor free target detection. Thanks to the above innovations, the FPS of the system reaches 132.8 and the mAP reaches 69.89% on a 2007 VOC data set, which is much higher than 94.9FPS when the CenterNet network selects MobileNet3 as a skeleton structure and keeps the original neck structure and head structure, and the mAP is similar to 69.20% mAP when the CenterNet network selects MobileNet3 as a skeleton structure and keeps the original neck structure and head structure; and is higher than 129FPS and 58.36% mAP when the MobileNet3 is used as the skeleton structure and the serial hole convolution of YOLOF is used as the neck structure. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 A flowchart of a lightweight target detection system without a priori frame is provided in the embodiment;
[0021] Figure 2 The image to be detected is obtained by the visual sensor in the embodiment;
[0022] Figure 3 The target frame information obtained by the parsing module of the embodiment;
[0023] Figure 4 It is the test result diagram of the embodiment;
[0024] Figure 5 It is the mAP statistical result diagram of the embodiment;
[0025] Figure 6 This is the test result diagram of the original neck structure using Centernet;
[0026] Figure 7 Neck structure for instance prediction module. DETAILED DESCRIPTION
[0027] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0028] Example 1
[0029] A lightweight target detection system without prior frames includes a visual sensor module, a feature extraction network, a prediction module and a parsing module; the visual sensor module obtains a two-dimensional image of a target scene through a visual sensor and serves as input data for subsequent modules;
[0030] The feature extraction network uses the MobileNet3 deep convolutional network to extract features from the input image;
[0031] The prediction module further optimizes the extracted features for the target detection task, making the features more suitable for the target detection task and thus improving the average accuracy mAP of the target detection and the number of frames processed per second FPS.
[0032] The prediction module uses the features obtained by the feature extraction network as the input of the neck structure, uses the parallel structure of the spatial pyramid in the neck structure to avoid repeated calculations, and uses dilated convolutions with different expansion coefficients to obtain receptive fields of different sizes without increasing the parameters of the model; the ECA module is used to enhance the features after the spatial pyramid dilated convolution; at the same time, the target detection system is a network without prior frame, which reduces the steps of manually obtaining the prior frame through clustering and other methods required for training the network, thereby reducing the additional calculation amount caused by parsing the prior frame. The neck structure of the spatial pyramid dilated convolution is shown in the attached figure. Figure 7 shown.
[0033] The prediction module includes a target detection network model. The target detection network model is an anchor-free network. The detection result is not constrained by a pre-set anchor box, and has a better detection result for position objects.
[0034] MobileNet3 is selected as the skeleton structure to extract features, and 1*1 convolution is used to increase the dimension of features to enhance the representation ability of the model; depthwise separable convolution is used instead of standard convolution to reduce model parameters and speed up model inference time; the Squeeze and Excitation module (SE module) is introduced to make the more effective feature layer have a greater impact on the target detection results. Combining the above characteristics, MobileNet3 is selected as the feature extraction skeleton to reduce the number of parameters while enhancing the representation ability of the model.
[0035] The parallel structure of the spatial pyramid avoids repeated calculations, and uses dilated convolutions with different expansion coefficients to obtain receptive fields of different sizes without increasing the parameters of the model. The structure is optimized based on the encoder of the YOLOF network. The encoder structure of YOLOF is a series of 4 dilated convolutions with different expansion coefficients, and each dilated convolution uses a 1*1 standard convolution to reduce the number of channels before and after the number of channels is increased. The parallel spatial pyramid avoids repeated calculations of 1*1 standard convolutions before and after the dilated convolutions with different expansion coefficients, and speeds up the network's running speed on multi-core devices, while adding an identity mapping shortcut to ensure the effectiveness of the module. Compared with standard convolutions with different convolution kernel sizes, dilated convolutions with different expansion coefficients can also obtain receptive fields of different sizes without increasing the parameters of the target detection network model.
[0036] The ECA module is used for feature enhancement after the spatial pyramid atrous convolution. Both the ECA and SE modules give each feature layer a weight between 0 and 1 to strengthen the influence of the effective feature layer on the target detection results. In a lightweight target detection system without a priori box, ECA is used instead of SE so that the fully connected layer calculation weights used by the SE module are replaced with one-dimensional convolution calculation weights, which can further reduce the number of model parameters and implement the same attention mechanism. With similar effects, the number of parameters and calculations brought by the fully connected layer are reduced, making the target detection system more lightweight.
[0037] Different expansion coefficients represent objects of different sizes to be detected. By stacking feature layers with different expansion coefficients and passing through the ECA module process, a larger weight can be given to the feature layer corresponding to the size of the target of interest, so that the network can better detect the target.
[0038] The method for implementing the lightweight target detection system without prior frame is as follows: Figure 1 As shown, the following steps are included:
[0039] The first is the visual sensor module, which uses the visual sensor to obtain the two-dimensional image of the target scene and uses it as the input data for subsequent modules. The two-dimensional image of the target scene to be detected obtained by this module is as follows: Figure 2 shown.
[0040] The two-dimensional image to be detected is used as the input of the feature extraction network. It is first resized and then feature extracted. The extracted features are used as the input of the prediction module to obtain the information of the target frame. This system adjusts the input image size to (512, 512) and uses MobileNet3 as the skeleton to extract features from the input image. The parallel structure of the spatial pyramid dilated convolution is used in the neck structure to avoid the repeated calculation caused by the dimensional adjustment of the series structure using 1*1 convolution, and uses dilated convolutions with different expansion coefficients (such as expansion coefficients of 1, 2, 3, and 4) (instead of ordinary convolutions with different sizes of convolution kernels) to obtain feature maps of receptive fields of different sizes without increasing the parameters of the model. After the spatial pyramid dilated convolution, the ECA module (Efficient Channel Attention module) is used for feature enhancement to enhance the influence of the feature layer corresponding to the size of the target object on the result. The confidence, center point, length and width information of the target frame can be obtained through the prediction module, such as Figure 3 shown.
[0041] Finally, the parsing module performs identity mapping on the confidence, center point, length and width information of the target box obtained by the prediction module to adapt it to the size of the input image. The detection box with confidence is more vividly displayed on the two-dimensional image. The result is as follows Figure 4 The system will display the result on the result detection display device.
[0042] Compared with the detection result diagram of the original Centernet network (which also uses mobilenet3 as the skeleton and compares the performance improvement of the neck structure of the system), the detection results obtained by this system include targets of multiple sizes. This shows that the neck structure designed by this system can better combine multi-scale information and obtain more comprehensive detection results while improving FPS.
[0043] In addition, if it needs to be applied to a completely new application scenario, the model parameters of the system need to be updated and the performance of the updated model needs to be evaluated. The evaluation indicator is mAP, such as Figure 5 shown. Figure 5 In the figure, the vertical axis is the classification of the objects to be detected contained in the VOC dataset, the horizontal axis is the network's scoring accuracy for each type of object, and mAP is the average of the average accuracy of each type. If the mAP value performs well, it means that the updated model can effectively complete the target detection task.
[0044] On the 2007VOC dataset, the system's FPS reached 132.8 and mAP reached 66.79%, which is higher than the 94.9FPS when the CenterNet network uses MobileNet3 as the skeleton structure and maintains the original neck and head structures; higher than the 129FPS and 58.36% mAP when MobileNet3 is used as the skeleton structure and YOLOF's serial hole convolution is used as the neck structure.
[0045] Example 2
[0046] The first is the visual sensor module, which uses the visual sensor to obtain the two-dimensional image of the target scene and uses it as the input data for subsequent modules. The two-dimensional image of the target scene to be detected obtained by this module is as follows: Figure 2 shown.
[0047] The two-dimensional image to be detected is used as the input of the feature extraction network. It is first resized and then feature extracted. The extracted features are used as the input of the prediction module to obtain the information of the target frame. This system adjusts the input image size to (512, 512) and uses the classic skeleton to extract features. ShuffleNet, ResNet, EfficientNet, etc. can be selected. The selection of different networks has a certain impact on the mAP and FPS values of the target detection system; the parallel structure of the spatial pyramid dilated convolution in the neck structure avoids the repeated calculation caused by the dimensional adjustment of the series structure using 1*1 convolution, and uses dilated convolutions with different expansion coefficients (instead of ordinary convolutions with different sizes of convolution kernels) to obtain feature maps of different sizes of receptive fields without increasing the parameters of the model; the ECA module is used for feature enhancement after the spatial pyramid dilated convolution, which can enhance the influence of the feature layer corresponding to the size of the target object on the result. The confidence, center point, length and width information of the target frame can be obtained through the prediction module.
[0048] Finally, the parsing module performs identity mapping on the confidence, center point, length and width information of the target box obtained by the prediction module to adapt it to the size of the input image. The detection box with confidence is more vividly displayed on the two-dimensional image. The result is as follows Figure 4 The system will display the result on the result detection display device.
[0049] Example 3
[0050] The first is the visual sensor module, which uses the visual sensor to obtain the two-dimensional image of the target scene and uses it as the input data for subsequent modules. The two-dimensional image of the target scene to be detected obtained by this module is as follows: Figure 2 shown.
[0051] The two-dimensional image to be detected is used as the input of the feature extraction network. It is first resized and then feature extracted. The extracted features are used as the input of the prediction module to obtain the information of the target frame. This system adjusts the input image size to (512, 512) and uses MobileNet3 as the skeleton to extract features from the input image. The parallel structure of the spatial pyramid dilated convolution in the neck structure avoids the repeated calculation caused by the dimensional adjustment of the series structure using 1*1 convolution, and uses dilated convolutions with different expansion coefficients (instead of ordinary convolutions with different sizes of convolution kernels) to obtain feature maps of different sizes of receptive fields without increasing the parameters of the model. After the spatial pyramid dilated convolution (SPD module), the corresponding positions of the five tensor results are maximized, and the result keeps the same dimension as any tensor. Feature enhancement can enhance the influence of the feature layer corresponding to the size of the target object on the result. The confidence, center point, length and width information of the target frame can be obtained through the prediction module.
[0052] Finally, the parsing module performs identity mapping on the confidence, center point, length and width information of the target box obtained by the prediction module to adapt it to the size of the input image. The detection box with confidence is more vividly displayed on the two-dimensional image. The result is as follows Figure 4 The system will display the result on the result detection display device.
[0053] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.
Claims
1. A lightweight object detection system without prior frames, characterized in that: Including feature extraction network and prediction module, The feature extraction network is a deep convolutional network, which uses the deep convolutional network to extract features from the input image; The prediction module includes a target detection network model, uses a spatial pyramid hole convolution structure in the neck structure of the target detection network model to optimize the target detection task, and finally outputs a prediction result; The feature extraction network uses Mobilenet3, which uses a*a convolution to increase the dimension of features, uses depthwise separable convolution instead of standard convolution to reduce model parameters, and introduces a compression excitation module; The input of the prediction module is the feature map output by the feature extraction network, and a spatial pyramid dilated convolution structure is used in the neck structure of the target detection network model. The process of the spatial pyramid dilated convolution structure is as follows: first, the dimension is increased by a*a convolution, and then the dilated convolutions with different expansion coefficients are connected in parallel to obtain receptive fields of different sizes. Then, the features with different expansion coefficients and the input of the prediction module are stacked and enhanced by the ECA module, and finally the prediction result is obtained by passing through several convolution layers; The parallel-connected dilated convolutions with different expansion coefficients are optimized on the encoder of the YOLOF network. The encoder structure of YOLOF is to connect multiple dilated convolutions with different expansion coefficients in series, and each dilated convolution uses a*a standard convolution before and after to reduce the number of channels and then increase the number of channels.
2. A lightweight object detection system without a priori box according to claim 1, characterized in that: The target detection network model is a network without a priori box, and the detection result is not constrained by a pre-set priori box, and has better detection results for position objects.
3. A lightweight object detection system without a priori box according to claim 1, characterized in that: The ECA module assigns each feature layer a weight of 0-1, and uses the ECA module to replace the fully connected layer calculation weights used by the SE module with one-dimensional convolution calculation weights.
Citation Information
Patent Citations
Real-time hardware fitting detection method based on anchor-free network
CN113780358A
Remote sensing image small target detection method based on improved YOLOv3
CN113971764A
Lightweight target detection method
CN114120019A
