Target detection model based on deep learning

Abstract

The invention particularly relates to a target detection model based on deep learning. According to the target detection model based on deep learning, a YOLOx-Darknet53 network structure is adopted as a basic model, CSPDarknet is adopted as a feature extraction network, an FPN feature pyramid is adopted as an enhanced feature extraction network of YOLOx, and YOLO Head is adopted as a classifier and a regression device of feature points; an SPP module is introduced between a fifth convolution layer and a sixth convolution layer of a YOLOv3 algorithm model, and the SPP module is composed of maximum pooling of different sizes and a jump connection, and is used for fusing local features and global features, and improving the detection precision of targets with large size differences. According to the target detection model based on deep learning, on the premise that the detection precision is not lost, the model training speed is increased, the model size is compressed, the small target detection rate is increased, the robustness of the neural network is improved, and the model can better adapt to a complex interference environment.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a target detection model based on deep learning. Background Art

[0002] Since the 1950s, deep neural networks (DNNs) have gradually developed, and their increasingly mature network structures have promoted their applications in various fields.

[0003] Convolutional Neural Networks (CNN) is a deep learning model that is particularly suitable for processing image data. Compared with traditional fully connected neural networks, convolutional neural networks (CNNs) have fewer parameters and higher efficiency, mainly due to their unique convolutional layer and pooling layer structure.

[0004] A convolutional neural network (CNN) usually consists of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer.

[0005] Input layer: receives raw image data.

[0006] Convolution layer: extracts local features from the image through convolution operation. The convolution kernel (or filter) slides on the input image, performs local calculations, and obtains a feature map. The convolution operation has the characteristics of weight sharing and local connection, which can significantly reduce the number of parameters of the model.

[0007] Pooling layer: Downsamples the feature map output by the convolutional layer to reduce the size and number of parameters of the feature map, while enhancing the robustness and feature extraction capabilities of the model. Commonly used pooling operations include maximum pooling and average pooling.

[0008] Fully connected layer: maps the feature map output by the pooling layer into a high-dimensional feature space, and then performs classification or regression through the softmax function.

[0009] Output layer: Outputs the final classification results or regression values.

[0010] Convolutional neural network (CNN) has the characteristics of local connection, weight sharing and pooling, which make convolutional neural network (CNN) more efficient and have a lower number of parameters when processing image data. In addition, convolutional neural network (CNN) can also automatically learn features in images without manual feature extraction. Therefore, convolutional neural network (CNN) has been widely used in image recognition, video analysis, natural language processing and other fields. For example, convolutional neural network (CNN) has achieved remarkable results in object recognition, face recognition, medical image analysis, scene recognition and other aspects. In addition, convolutional neural network (CNN) has also been applied to tasks such as behavior recognition, event detection, video summarization, sentiment analysis, text classification, and machine translation.

[0011] Currently, environmental information recognition in autonomous driving faces huge challenges. The two-stage algorithm represented by the R-CNN series of algorithms implements the target detection model through the following steps: first, find candidate frames that may contain objects, and then perform coordinate regression and target classification on the candidate frames.

[0012] The modules used in each step of the two-stage algorithm need to be trained separately, and the step of finding candidate boxes that may contain objects takes a lot of time because it requires a lot of searches. Therefore, although the R-CNN series of algorithms have high accuracy, the detection speed is very slow.

[0013] The YOLO series of algorithms use an end-to-end approach to directly combine candidate box prediction and classification, transforming the target detection problem into a regression problem. Although the average accuracy of the detection results is slightly reduced, it can greatly improve the detection speed.

[0014] In order to improve the robustness of the neural network and better adapt to complex interference environments, the present invention proposes a target detection model based on deep learning. Summary of the invention

[0015] In order to overcome the deficiencies of the prior art, the present invention provides a simple and efficient target detection model based on deep learning.

[0016] The present invention is achieved through the following technical solutions:

[0017] A target detection model based on deep learning, characterized by: using the YOLOx-Darknet53 network structure as the basic model, using CSPDarknet as the feature extraction network, using the FPN feature pyramid as the enhanced feature extraction network of YOLOx, and using YOLO Head as the classifier and regressor of the feature point;

[0018] An SPP module is introduced between the fifth and sixth convolutional layers of the YOLOv3 algorithm model. The SPP module consists of maximum pooling of different sizes and a skip connection, which is used to fuse local features and global features and improve the detection accuracy of targets with large size differences.

[0019] The YOLOx-Darknet53 network structure includes four parts: input, primary perception, intermediate processing and high-level decision-making;

[0020] Among them, the input part is used to achieve data enhancement, the primary perception part adopts the YOLOx-Darknet53 backbone network, and the intermediate processing part adopts the FPN structure for convergence; the high-level decision-making part adopts a decoupling head design to improve accuracy and speed up network convergence. At the same time, it adopts an anchor-free design to complete label allocation and loss calculation.

[0021] The SPP module includes four parallel branches, respectively having 5×5, 9×9 and 13×13 maximum pooling kernels and a skip connection, which is used to rescatter to the next layer of the detection network after the feature maps are merged by the SPP module to enrich the expressive power of the feature maps.

[0022] The input part of the YOLOx-Darknet53 network structure adopts two data enhancement methods, Mosaic and Mixup, to improve the generalization ability of the model and the detection ability of small targets;

[0023] Before the last 15 training cycles, both the Mosaic and Mixup data enhancement methods were turned on; in the last 15 training cycles, both the Mosaic and Mixup data enhancement methods were turned off.

[0024] The decoupling head design of the high-level decision-making part first uses 1×1 convolution to reduce the dimension, then uses two 3×3 convolutions in two branches respectively, and finally customizes the adjustment.

[0025] The intermediate processing part uses the FPN neck structure to construct an FPN feature pyramid to enhance feature extraction;

[0026] The FPN feature pyramid includes three feature layers, and the three feature layers are respectively located at different positions of the middle layer, the lower middle layer and the bottom layer of the main CSDarknet. Feature fusion is performed on feature layers at different positions to better extract features.

[0027] The present invention is based on a deep learning target detection model, which is applied to the detection of vehicle-mounted videos or images, marks the vehicle target with a rectangular frame, and outputs the detection result, wherein the detection result includes the location, category and confidence of the target; the detection result can also be post-processed to improve the accuracy and robustness of the detection.

[0028] The beneficial effects of the present invention are as follows: the target detection model based on deep learning not only improves the model training speed and compresses the model size without losing detection accuracy, but also improves the detection rate of small targets, improves the robustness of the neural network, and can better adapt to complex interference environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Attached Figure 1 Schematic diagram of the basic structure of the target detection model based on deep learning in the present invention.

[0031] Attached Figure 2 This is a schematic diagram of the decoupling head input method of the present invention.

[0032] Attached Figure 3 This is a schematic diagram of the three branch information of the decoupling head of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention.

[0034] The embodiments of the present invention are part of the embodiments, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0035] This deep learning-based target detection model uses the YOLOx-Darknet53 network structure as the basic model, CSPDarknet as the feature extraction network, FPN feature pyramid as the enhanced feature extraction network of YOLOx, and YOLO Head as the classifier and regressor of feature points. The end-to-end method of the YOLO series algorithm can directly combine candidate region prediction and target classification, and transform the target detection problem into a regression problem to improve the detection speed while maintaining a certain accuracy.

[0036] An SPP module is introduced between the fifth and sixth convolutional layers of the YOLOv3 algorithm model. The SPP module consists of maximum pooling of different sizes and a skip connection, which is used to fuse local features and global features and improve the detection accuracy of targets with large size differences.

[0037] The YOLOx-Darknet53 network structure includes four parts: input, primary perception, intermediate processing and high-level decision-making;

[0038] Among them, the input part is used to achieve data enhancement, the primary perception part adopts the YOLOx-Darknet53 backbone network, and the intermediate processing part adopts the FPN structure for convergence; the high-level decision-making part adopts a decoupling head design to improve accuracy and speed up network convergence. At the same time, it adopts an anchor-free design to complete label allocation and loss calculation.

[0039] The SPP module includes four parallel branches, respectively having 5×5, 9×9 and 13×13 maximum pooling kernels and a skip connection, which is used to rescatter to the next layer of the detection network after the feature maps are merged by the SPP module to enrich the expressive power of the feature maps.

[0040] The input part of the YOLOx-Darknet53 network structure adopts two data enhancement methods, Mosaic and Mixup, to improve the generalization ability of the model and the ability to detect small targets;

[0041] Before the last 15 training cycles, both the Mosaic and Mixup data enhancement methods were turned on; in the last 15 training cycles, both the Mosaic and Mixup data enhancement methods were turned off.

[0042] Due to stronger data augmentation, ImageNet pre-training is meaningless, therefore, all models are trained from scratch.

[0043] Therefore, the current detection head used in the YOLO series lacks expressiveness and is not as expressive as the decoupled head. Using a decoupled head can not only improve accuracy but also speed up the convergence of the network. However, it is worth noting that the decoupled detection head can speed up the convergence of the model, but it also increases the complexity of the operation. Therefore, the decoupled head design of the advanced decision-making part first uses 1×1 convolution to reduce the dimension, and then uses two 3×3 convolutions in each of the two branches, and finally customizes the adjustment to slightly increase the network parameters, while improving accuracy and speeding up the network convergence.

[0044] Extract the decoupling head details from YOLOx-Darknet53. After the previous Neck layer, the length and width of the decoupling head input are 20*20, as shown in the attached figure. Figure 2 shown.

[0045] From the attached Figure 3 It can be seen that there are three branches in total before splicing:

[0046] (1) cls_output: mainly for the category of the target box and the predicted score. Since the COCO dataset has 80 categories, mainly N binary judgments, after being processed by the Sigmoid activation function, its size becomes 20*20*80.

[0047] (2) obj_output: It mainly determines whether the target frame is the foreground or the background. Therefore, after Sigmoid processing, its size becomes 20*20*1.

[0048] (3) reg_output: mainly predicts the coordinate information (x, y, w, h) of the target frame, so the size is 20*20*4.

[0049] The last three outputs are concatenated and fused together to obtain 20*20*85 feature information.

[0050] The above is only the information of decoupling head 1. The branch information of decoupling head 2 and decoupling head 3 can be found in the attached Figure 3 .

[0051] The intermediate processing part uses the FPN neck structure to construct an FPN feature pyramid to enhance feature extraction;

[0052] The FPN feature pyramid includes three feature layers, and the three feature layers are respectively located at different positions of the middle layer, the lower middle layer and the bottom layer of the main CSDarknet. Feature fusion is performed on feature layers at different positions to better extract features.

[0053] When the input is (640640,3), the shapes of the three feature layers are FEAT1=(80,80256), FEAT2=(40,40512), and FEAT3=(20,201024)

[0054] The feature layer of feat3 = (20, 201024) is obtained after 1 1X1 convolution to adjust channel P5. P5 is combined with the feature layer of feat2 = (40, 40512) after upsampling UmSampling2d, and then feature extraction is performed using CSPLayer to obtain P5_upsample. At this time, the feature layer obtained is (40, 40512).

[0055] The feature layer of P5_upsampe = (40, 40512) is combined with the feature layer of feat1 = (80, 80256) after one 1X1 convolution, and the channels are adjusted to obtain P4. P4 is combined with the feature layer of featu1 = (80, 80256) after upsampling UmSampling2d. Then CSPLayer is used to extract features from P3_out, and the feature layer obtained at this time is (8080256).

[0056] The feature layer of P3_out = (80, 80256) is downsampled by 3x3 convolution, and then stacked with P4 after downsampling, and then CSPLayer is used to extract the features of P4_out. At this time, the feature layer obtained is (40, 40512).

[0057] The feature layer of P4_out = (40, 40512) is downsampled by 3x3 convolution, and then stacked with P5 after downsampling, and then CSPLayer is used to extract the features of P5_out. At this time, the feature layer obtained is (20, 201024).

[0058] This deep learning-based target detection model was tested on real-time video, and the detection rate of vehicles within the effective field of view reached 81.28%, achieving high-precision and real-time target detection.

[0059] This deep learning-based object detection model obtains three prediction results for each feature layer.

[0060] Taking the three prediction results corresponding to (20,201024) as an example, each feature layer is equivalent to dividing the image into 20x20 feature points. If the feature point falls in the corresponding box of the object, it is used to predict the object. The three prediction results are as follows:

[0061] 1). Reg prediction result, at this time the number of convolution channels is 4, and the final result is (20,20,4). The 4 can be divided into two 2s, the first 2 is the offset between the center of the prediction box and the feature point, and the second 2 is the width and height of the prediction box compared with the logarithmic index parameter.

[0062] 2). Object prediction results. At this time, the number of convolution channels is 1, and the final result is (20, 20, 1), which indicates the probability that each feature point prediction box contains an object.

[0063] 3). Cls prediction result. At this time, the number of channels of the convolution is num_classes, and the final result is (20,20,num_classe), which indicates the probability that each feature point corresponds to a certain class of objects. The prediction value in the last dimension num_classs indicates the probability of belonging to each class.

[0064] This deep learning-based target detection model is applied to the detection of vehicle-mounted videos or images, marks the vehicle target with a rectangular frame, and outputs the detection results, where the detection results include the location, category and confidence of the target; it also includes post-processing of the detection results to improve the accuracy and robustness of the detection.

[0065] Each small target is represented by a red frame and text to indicate the vehicle category. The red frame is also marked with a number, which represents the confidence level. The higher the number, the higher the confidence level. Generally, the larger the target, the higher the confidence level.

[0066] This target detection model based on deep learning, based on an in-depth analysis of the target detection model based on convolutional neural networks, selects the YOLOx-Darknet53 network structure as the main structure of the target detection system, and uses the characteristics of feature fusion at different layers of the feature pyramid network to improve the detection rate of small targets. Using CSPDarknet as the feature extraction network, FPN can be called an enhanced feature extraction network of YOLOx. The three effective feature layers obtained in the trunk part will be feature fused in this part. The purpose of feature fusion is to combine feature information of different scales. The FPN feature pyramid can fuse features from feature layers of different shapes, which is conducive to extracting better features. YOLO Head acts as a classifier and regressor for feature points to determine whether there is an object corresponding to it at the feature point. It improves the model training speed, compresses the model size, and basically does not lose accuracy.

[0067] The embodiment described above is only one specific implementation of the present invention. Common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. A target detection model based on deep learning, characterized by: The YOLOx-Darknet53 network structure is used as the basic model, CSPDarknet is used as the feature extraction network, FPN feature pyramid is used as the enhanced feature extraction network of YOLOx, and YOLO Head is used as the classifier and regressor of feature points; An SPP module is introduced between the fifth and sixth convolutional layers of the YOLOv3 algorithm model. The SPP module consists of maximum pooling of different sizes and a skip connection, which is used to fuse local features and global features and improve the detection accuracy of targets with large size differences.

2. The deep learning-based target detection model according to claim 1, characterized in that: The SPP module includes four parallel branches, respectively having 5×5, 9×9 and 13×13 maximum pooling kernels and a skip connection, which is used to rescatter to the next layer of the detection network after the feature maps are merged by the SPP module to enrich the expressive power of the feature maps.

3. The deep learning-based target detection model according to claim 1, characterized in that: The YOLOx-Darknet53 network structure includes four parts: input, primary perception, intermediate processing and high-level decision-making; Among them, the input part is used to achieve data enhancement, the primary perception part adopts the YOLOx-Darknet53 backbone network, and the intermediate processing part adopts the FPN structure for convergence; the high-level decision-making part adopts a decoupling head design to improve accuracy and speed up network convergence. At the same time, it adopts an anchor-free design to complete label allocation and loss calculation.

4. The deep learning-based target detection model according to claim 3, characterized in that: The input part of the YOLOx-Darknet53 network structure adopts two data enhancement methods, Mosaic and Mixup, to improve the generalization ability of the model and the ability to detect small targets.

5. The deep learning-based target detection model according to claim 4, characterized in that: Before the last 15 training cycles, both the Mosaic and Mixup data enhancement methods were turned on; in the last 15 training cycles, both the Mosaic and Mixup data enhancement methods were turned off.

6. The deep learning-based target detection model according to claim 3, characterized in that: The decoupling head design of the high-level decision-making part first uses 1×1 convolution to reduce the dimension, then uses two 3×3 convolutions in two branches respectively, and finally customizes the adjustment.

7. The deep learning-based target detection model according to claim 3, characterized in that: The intermediate processing part uses the FPN neck structure to construct an FPN feature pyramid to enhance feature extraction; The FPN feature pyramid includes three feature layers, and the three feature layers are respectively located at different positions of the middle layer, the lower middle layer and the bottom layer of the main CSDarknet. Feature fusion is performed on feature layers at different positions to better extract features.

8. The deep learning-based target detection model according to any one of claims 1 to 7, characterized in that: Applied to the detection of vehicle videos or images, the vehicle target is marked with a rectangular frame and the detection result is output, where the detection result includes the location, category and confidence of the target; the detection result can also be post-processed to improve the accuracy and robustness of the detection.

Images