Object detection method based on lightweight backbone network

Abstract

The object detection method based on the lightweight backbone network disclosed in the present invention is implemented specifically according to the following steps: Step 1, prepare a pre-training data set Tiny-Imagenet and a training data set in VOC format; Step 2, build a feature extraction network LBNet; Step 3, build a feature pyramid structure composed of multi-layer output features of LBNet, and perform pre-training in the Tiny-Imagenet data set obtained in step 1 to obtain pre-training weights. Step 4, design an anchors generation container algorithm to generate the anchors size required by the regional recommendation network. Step 5, complete the training and prediction of the model. This method solves the problems of low detection efficiency and inaccurate bounding box positioning in existing detection technologies.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection and relates to a target detection method based on a lightweight backbone network. Background Art

[0002] The task of object detection is to identify all object categories in a given input image and determine the position of its bounding box. With the rapid development of deep learning, object detection technology has been widely used in fields such as autonomous driving and face recognition. However, most of the feature extraction networks currently used for object detection models have complex structures and large number of parameters, which leads to the problem of low detection efficiency. Secondly, most of the existing detection models provide manually preset rectangular bounding boxes (anchors) for the region recommendation network based on the researchers' work experience, and these anchors have a low degree of fit with the experimental dataset, which greatly affects the bounding box positioning accuracy of the subsequent detection network. Summary of the invention

[0003] The purpose of the present invention is to provide a target detection method based on a lightweight backbone network, which solves the problems of low detection efficiency and inaccurate boundary box positioning in existing detection technologies.

[0004] The technical solution adopted by the present invention is a target detection method based on a lightweight backbone network, which is specifically implemented according to the following steps:

[0005] Step 1: Prepare the pre-training dataset Tiny-Imagenet and the training dataset in VOC format;

[0006] Step 2: Build the feature extraction network LBNet (Lightweight backbone network);

[0007] Step 3: Build a feature pyramid structure consisting of multi-layer output features of LBNet, and pre-train it in the Tiny-Imagenet dataset obtained in step 1 to obtain pre-training weights.

[0008] Step 4: Design the anchors generation container algorithm CFA (Container for anchors) to generate the anchors size required by the region recommendation network.

[0009] Step 5: Complete model training and prediction.

[0010] The present invention is also characterized in that

[0011] Step 1 is implemented according to the following steps:

[0012] Step 1.1, prepare the pre-training dataset Tiny-Imagenet and generate the label file required for pre-training;

[0013] Step 1.1 is implemented according to the following steps:

[0014] Step 1.1.1, extract a part from the Imagenet dataset to make the required pre-training dataset Tiny-Imagenet, that is, the pre-training dataset TI;

[0015] Step 1.1.2, generate a label file, build a json type label file according to the actual object name corresponding to each category; divide the training set and the validation set from TI in a ratio of 8:2, and then sample each category contained in TI to obtain the label files train.csv and val.csv respectively;

[0016] Step 1.2: Prepare a training dataset in VOC format.

[0017] Step 1.2 is implemented as follows:

[0018] Step 1.2.1, merge the Pascal VOC2007 and 2012 datasets widely used for object detection to create the training dataset M in the VOC format we need;

[0019] Step 1.2.2, generate label files, annotate the categories and regions of the images in the training set and validation set, generate XML files, and put them in the Annotations folder. Generate train.txt, val.txt and trainval.txt label files required for subsequent training based on the XML files, and put them in the Main folder.

[0020] Step 2 is implemented as follows:

[0021] Step 2.1, design the internal connection sub-block;

[0022] Step 2.2: Design a residual connection block, add the input and output of each internal connection sub-block by skip connection, and then form a residual connection block;

[0023] Step 2.3: Design a local dense connection block, and concatenate the output layer features of each internal connection sub-block with the output features of all previous internal connection sub-blocks in the corresponding channel dimension to form a local dense connection block.

[0024] Step 2.4. Design the first network layer C of LBNet 1 , C 1It is a convolutional layer structure consisting of a convolution kernel with a stride of 1, a size of 3x3, and 32 channels, a Batch Normalization layer, and an activation function ReLu;

[0025] Step 2.5: Design the network layer structure C in LBNet with residual connection blocks and dense connection blocks as the main body 2 , C 3 , C 4 , C 5 , C 6 ;

[0026] Step 2.6. Design the downsampling layer d in LBNet.

[0027] Step 2.1 is implemented as follows:

[0028] Step 2.1.1, the structure of each connected sub-block (ICB) is the same, consisting of a convolution kernel (Conv) of size 1x1 and a convolution kernel of size 3x3; the specific forward propagation process is Conv(1x1)-BN-ReLu-Conv(3x3)-BN-ReLu;

[0029] Step 2.1.2: Ensure that the input and output sizes of the internally connected sub-blocks are consistent, and set the number of channels of the 1x1 convolution kernel in the internally connected sub-block to 1 / 2 of the number of channels of the 3x3 convolution kernel.

[0030] Step 2.5 is implemented as follows:

[0031] Step 2.5.1: C 2 With C 5 Set as a layer structure consisting of locally densely connected blocks; C 2 is a local densely connected block (DB) consisting of three internally connected sub-blocks (ICBs), C 5 It is a densely connected block (DB) consisting of 4 internally connected sub-blocks (ICBs);

[0032] Step 2.5.2: C 3 , C 4 , C 6 Set to a layer structure consisting of residual connection blocks; C 3 , C 4 , C 6 They are residual connection blocks consisting of 4, 6 and 6 internal connection sub-blocks respectively.

[0033] Step 2.6 is implemented as follows:

[0034] Step 2.6.1. In LBNet, C 1 ~C 6Between two adjacent network layers, a downsampling layer d with a size of 3x3 and a step size of 2 is used to complete the downsampling operation;

[0035] Step 2.6.2: For network layer C 2 ~C 6 For example, the convolution kernel with a stride of 2 and a size of 3x3 is moved to the main path outside the residual (local dense) connection block, that is, the image downsampling is completed before each independent connection block.

[0036] Step 3 is implemented as follows:

[0037] Step 3.1, design the horizontal connection structure, and connect the network layer C in LBNet through the horizontal connection structure. 2 ~C 6 The output features of are used as the input of the feature pyramid;

[0038] Step 3.1 is implemented as follows:

[0039] Step 3.1.1, set the LBNet output feature layer; select the first local dense connection block C 2 Start to output the features and set C 2 The output is denoted as F 2 In order to obtain richer multi-scale feature information, the next three residual connection blocks C 3 , C 4 , C 6 feature information and a local densely connected block C 5 Therefore, the five feature layers are recorded as follows: F = {F 2 , F 3 , F 4 , F 5 , F 6};

[0040] Step 3.1.2: Uniformly set the number of channels of the output layer and reduce the dimension of the output layer; complete the above C through a convolution kernel of size 1x1 2 ~C 6 Dimensionality reduction of the feature layer channel, due to C 2 The number of channels is 64, so the number of channels here is uniformly set to 64.

[0041] Step 3.2: Build a top-down feature pyramid structure consisting of multiple layers of LBNet output features; first, use a 1x1x64 convolution kernel to convolution the feature layer F 6 , F 5 Dimensionality reduction to obtain a new feature layer P 6 , P 5, thus obtaining the first feature output layer P 6 , and then use the bilinear interpolation method to convert P 6 Upsample to P 5 The same size; finally, P 6 and P 5 By adding the corresponding dimensions, we get the updated second feature output layer P 5 ; Based on the same strategy as above, use P 5 Update P 4 , use P 4 Update P 3 , use P 3 Update P 2 ;

[0042] Step 3.3: Send the images in the pre-training dataset TI into the feature pyramid structure composed of LBNet, set the number of categories according to the label file in step 1.1.2, perform pre-training, and obtain the pre-training weights.

[0043] Step 4 is implemented according to the following steps:

[0044] Step 4.1, input the training set image with the bounding box, i.e., the sample;

[0045] Step 4.2: Select k bounding boxes from all input images as cluster centers;

[0046] Step 4.3, calculate and compare the distance D between all bounding boxes and each cluster center in turn, and divide the class to which the bounding box belongs;

[0047] Step 4.4: Update the cluster center in each cluster, repeat steps 4.2 to 4.4 until no samples in each cluster change, and select the final anchor size.

[0048] Step 4.2 is implemented as follows:

[0049] Step 4.2.1: Randomly select a bounding box as the initial cluster center X 1 ;

[0050] Step 4.2.2: Calculate the correlation between all samples in the dataset and X 1 The IOU between them, the sample with the smallest IOU becomes the second cluster center X 2 ; And so on, until k cluster centers are obtained, denoted as G = {X 1 ,X 2 , …, X k};

[0051] Step 4.4 is implemented according to the following steps:

[0052] Step 4.4.1, sort all Bboxes in the same category in order of area from small to large;

[0053] Step 4.4.2: Take the sample represented by the median as the new cluster center;

[0054] Step 4.4.3, obtain the final anchors size; since FPN outputs multi-scale features to RPN, three anchors sizes generated by CFA are provided for the five prediction feature layers respectively, and they are transmitted to the subsequent network; the specific sizes are: first layer: [13,20], [28,31], [29,66], second layer: [60,40], [55,103], [97,67], third layer: [75,175], [121,120], [272,120], fourth layer: [126,249], [185,174], [212,292], fifth layer: [382,205], [321,368], [473,336].

[0055] Step 5 is specifically implemented according to the following steps: complete the training and prediction of the model; load the pre-trained weights obtained in step 2, send the anchor size obtained in step 4 to the RPN network to obtain the region proposal box required by the subsequent detection network, and then train on the VOC format training data set obtained in step 1; finally, complete the prediction of the model.

[0056] The beneficial effects of the present invention are:

[0057] 1. The method of the present invention innovatively proposes an improved method that can be applied to all two-stage detection models, making a significant contribution to improving detection performance.

[0058] 2. Drawing on the advantages of residual connections and dense connections, the lightweight backbone network (LBNet) designed in this invention can complete the detection task faster while ensuring detection accuracy.

[0059] 3. The Anchor Generation Container (CFA) designed in the present invention can obtain a suitable anchor size without any other auxiliary conditions, thereby improving the accuracy of subsequent network target positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is an algorithm flow chart of the target detection method based on the lightweight backbone network of the present invention;

[0061] Figure 2 It is a LBNet network structure diagram of the target detection method based on a lightweight backbone network of the present invention;

[0062] Figure 3 It is an algorithm flow chart of a container for anchor generation (CFA) of a target detection method based on a lightweight backbone network of the present invention;

[0063] Figure 4 This is a schematic diagram comparing the effects of different Anchor generation methods on image positioning. DETAILED DESCRIPTION

[0064] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] The present invention provides a target detection method based on a lightweight backbone network. Figure 1-3 As shown, the specific implementation steps are as follows:

[0066] Step 1: Prepare the pre-training dataset Tiny-Imagenet and the training dataset in VOC format;

[0067] Step 1 is implemented according to the following steps:

[0068] Step 1.1, prepare the pre-training dataset Tiny-Imagenet and generate the label file required for pre-training;

[0069] Step 1.1 is implemented according to the following steps:

[0070] Step 1.1.1. Extract 3.1G from the Imagenet dataset to create the pre-training dataset Tiny-Imagenet (TI) required for this paper; Since Imagenet is a very large open source dataset (about 100G), it will take a long time to train directly on Imagenet and the hardware requirements are also very high, so we extracted a small part of Imagenet to create TI. There are 100 categories in TI, each category has 600 pictures, so TI contains a total of 60,000 pictures of different sizes and non-fixed sizes.

[0071] Step 1.1.2, generate a label file, build a json type label file according to the actual object name corresponding to each category; divide the 60,000 images in TI into training set and validation set in a ratio of 8:2, and then sample each category contained in TI to obtain label files train.csv and val.csv respectively;

[0072] Step 1.2: Prepare a training dataset in VOC format.

[0073] Step 1.2 is implemented as follows:

[0074] Step 1.2.1, merge the Pascal VOC2007 and 2012 datasets widely used for object detection to create the VOC format training dataset M we need. M inherits all 20 categories in the original Pascal VOC dataset, with 10,000 images in the training set and validation set, and 5,000 images in the test set;

[0075] Step 1.2.2, generate label files, annotate the categories and regions of the images in the training set and validation set, generate XML files, and put them in the Annotations folder. Generate train.txt, val.txt and trainval.txt label files required for subsequent training based on the XML files, and put them in the Main folder.

[0076] Step 2: Build the feature extraction network LBNet. LBNet is a fully convolutional neural network with 52 convolutional layers and a dense connection. Figure 2 As shown;

[0077] Step 2 is implemented as follows:

[0078] Step 2.1, design the inner connection block (InnerConnectionBlock, ICB);

[0079] Step 2.1 is implemented as follows:

[0080] Step 2.1.1, the structure of each connected sub-block (ICB) is the same, consisting of a 1x1 convolution kernel (Conv) and a 3x3 convolution kernel. The 1x1 convolution kernel can greatly reduce the number of network parameters. The specific forward propagation process is Conv(1x1)-BN-ReLu-Conv(3x3)-BN-ReLu.

[0081] Step 2.1.2: Ensure that the input and output sizes of the internally connected sub-blocks are consistent, and set the number of channels of the 1x1 convolution kernel in the internally connected sub-block to 1 / 2 of the number of channels of the 3x3 convolution kernel.

[0082] Step 2.2, design the residual connection block (ResBlock, RB). The internal connection sub-block (ICB) is the basic unit of the residual connection block (RB). The input and output of each internal connection sub-block are added by skip connection to form a residual connection block. The use of skip connection can not only solve the degradation problem caused by deepening the network layer, but also enhance the feature expression of the deep network;

[0083] Step 2.3, design a local dense connection block (DesBlock, DB). The internal connection sub-block (ICB) is the basic unit of the local dense connection block (DB). The output layer features of each internal connection sub-block are spliced ​​with the output features of all previous internal connection sub-blocks in the corresponding channel dimension to form a local dense connection block. The use of local dense connections can enhance the local feature representation and efficiently transfer low-level features to deep convolution, thereby improving the information transmission efficiency and gradient transmission efficiency of the entire network.

[0084] Step 2.4. Design the first network layer C of LBNet 1 , C 1 It is a convolutional layer structure consisting of a convolution kernel with a step size of 1, a size of 3x3, and a channel number of 32, a Batch Normalization (BN) layer, and an activation function ReLu. 1 The number of channels of the input image can be changed to facilitate the input of subsequent networks and extract fine low-level features of the image;

[0085] Step 2.5: Design the network layer structure C in LBNet with residual connection blocks and dense connection blocks as the main body 2 , C 3 , C 4 , C 5 , C 6 ;

[0086] Step 2.5 is implemented as follows:

[0087] Step 2.5.1: C 2 With C 5 Set as a layer structure consisting of locally densely connected blocks; C 2 It is a local densely connected block (DB) consisting of three internally connected sub-blocks (ICBs). 2 Using dense connection blocks in the output layer can combine the output layer features with the previous C 1 The obtained features are concatenated, which can ensure that more underlying feature information is retained in subsequent network layers. 5 It is a densely connected block (DB) composed of 4 internally connected sub-blocks (ICBs). Since the dense block can efficiently transfer low-level features to high-level network structures through dense connections, it can achieve the fusion and reuse of multi-layer features and improve the information transmission efficiency and gradient transmission efficiency of the entire network.

[0088] Step 2.5.2: C 3 , C 4 , C 6 Set to a layer structure consisting of residual connection blocks; C 3 , C4 , C 6 The residual connection blocks (RB) are composed of 4, 6 and 6 internal connection sub-blocks (ICB) respectively. We use residual connection blocks to deepen the network, which can not only enhance the feature representation of the deep network, but also solve the network degradation problem caused by deepening the network.

[0089] Step 2.6, design the downsampling layer d in LBNet;

[0090] Step 2.6 is implemented as follows:

[0091] Step 2.6.1. In LBNet, C 1 ~C 6 Between two adjacent network layers, a downsampling layer d with a size of 3x3 and a step size of 2 is used to complete the downsampling operation; since the convolution layer with a step size of 2 reduces the length and width of the image by half (the area is reduced to 1 / 4 of the original image), d can achieve the same effect as using a convolution layer with a step size of 1 plus a pooling layer, but d can complete the downsampling while learning features, thereby effectively reducing the amount of parameter calculation. In addition, this ensures that LBNet is a fully convolutional neural network with no fully connected layers and pooling layers, only convolution layers, so using d to achieve image downsampling can improve the efficiency of feature extraction.

[0092] Step 2.6.2: For network layer C 2 ~C 6 For example, unlike the commonly used residual (dense) connection, the convolution kernel with a stride of 2 and a size of 3x3 is moved to the main path outside the residual (local dense) connection block, that is, the image downsampling is completed before each independent connection block, so there is no need to add additional size transformation operations in the internal connection sub-blocks of the connection block. This structural design can improve the utilization of the GPU and the efficiency of feature extraction. In addition, we do not need to design additional 1x1 convolution kernels to adjust the number of channels, reducing the amount of parameter calculations.

[0093] Step 3: Build a feature pyramid structure consisting of multi-layer output features of LBNet, and pre-train it in the Tiny-Imagenet dataset obtained in step 1 to obtain pre-training weights;

[0094] Step 3 is implemented as follows:

[0095] Step 3.1, design the horizontal connection structure, and connect the network layer C in LBNet through the horizontal connection structure. 2 ~C 6 The output features of are used as the input of the feature pyramid;

[0096] The step 3.1 is specifically implemented according to the following steps:

[0097] Step 3.1.1, set the LBNet output feature layer; since the densely connected blocks are used in the first half of LBNet to obtain rich low-level features, we choose to output the feature layer from the first local densely connected block C 2 Start to output the features and set C 2 The output is denoted as F 2 In order to obtain richer multi-scale feature information, the next three residual connection blocks C 3 , C 4 , C 6 feature information and a local densely connected block C 5 Therefore, the five feature layers are recorded as follows: F = {F 2 , F 3 , F 4 , F 5 , F 6};

[0098] Step 3.1.2, uniformly set the number of channels of the output layer and reduce the dimension of the output layer; in order to facilitate the subsequent feature pyramid structure for related operations, a convolution kernel of size 1x1 is used to complete the above C 2 ~C 6 Dimensionality reduction of the feature layer channel, due to C 2 The number of channels is 64, so the number of channels here is uniformly set to 64.

[0099] Step 3.2: Build a top-down feature pyramid structure consisting of multiple layers of LBNet output features; first, use a 1x1x64 convolution kernel to convolution the feature layer F 6 , F 5 Dimensionality reduction to obtain a new feature layer P 6 , P 5 , thus obtaining the first feature output layer P 6 , and then use the bilinear interpolation method to convert P 6 Upsample to P 5 The same size; finally, P 6 and P 5 By adding the corresponding dimensions, we get the updated second feature output layer P 5 ; Based on the same strategy as above, use P 5 Update P 4 , use P 4 Update P 3 , use P 3 Update P 2; Therefore, this constitutes a top-down feature pyramid structure starting from the top of the network to the bottom;

[0100] Step 3.3: Send the images in the pre-training dataset (TI) into the feature pyramid structure composed of LBNet, set the number of categories according to the label file in step 1.1.2, perform pre-training, and obtain the pre-training weights.

[0101] Step 4: Design the anchor generation container algorithm CFA to generate the anchor size required by the region recommendation network, such as Figure 3 As shown;

[0102] Step 4 is implemented according to the following steps:

[0103] Step 4.1, input the training set image (sample) with the bounding box;

[0104] Step 4.2: Select k bounding boxes from all input images as cluster centers;

[0105] Step 4.2 is implemented as follows:

[0106] Step 4.2.1: Randomly select a bounding box (Bbox) as the initial cluster center X 1 ;

[0107] Step 4.2.2: Calculate the correlation between all samples in the dataset and X 1 The IOU (Intersection over Union) between them, the sample with the smallest IOU becomes the second cluster center X 2 ; And so on, until k cluster centers are obtained, denoted as G = {X 1 ,X 2 , …, X k Compared with the method of directly randomly selecting k cluster centers, the initial cluster centers obtained by CFA have a higher fit with the actual cluster centers, thus avoiding the algorithm from falling into the local optimum, and the real cluster center values ​​can be obtained stably and efficiently based on the initial cluster centers;

[0108] Step 4.3, calculate and compare the distance D between all bounding boxes and each cluster center in turn, and divide the class to which the bounding box belongs;

[0109] Step 4.4: Update the cluster center in each cluster, repeat steps 4.2 to 4.4 until no samples in each cluster change, and then select the final anchor size;

[0110] Step 4.4 is implemented according to the following steps:

[0111] Step 4.4.1, sort all Bboxes in the same category in order of area from small to large;

[0112] Step 4.4.2, take the sample represented by the median as the new cluster center; suppose there are 17 sample points in the same category, and after sorting, all samples in the same category are labeled with corresponding serial numbers in the order of the size of the bounding box area: {1,2,3...,17}. Since the noise samples in the data set have little effect on the median, and the median can even remain unchanged, CFA takes the sample represented by the median 9 as the new cluster center. The method of updating clusters based on sample mean must consider outliers or noise samples when calculating the average value, so the cluster center is not determined to be the sample point represented by 9. Therefore, compared with the latter, CFA is more reliable, especially in the presence of noise or singular points.

[0113] Step 4.4.3, obtain the final anchors size; since FPN outputs multi-scale features to RPN, three anchors sizes generated by CFA are provided for the five prediction feature layers respectively, and they are transmitted to the subsequent network; the specific sizes are: first layer: [13,20], [28,31], [29,66], second layer: [60,40], [55,103], [97,67], third layer: [75,175], [121,120], [272,120], fourth layer: [126,249], [185,174], [212,292], fifth layer: [382,205], [321,368], [473,336].

[0114] Step 5: Complete model training and prediction.

[0115] Step 5 is specifically implemented according to the following steps: complete the training and prediction of the model; load the pre-trained weights obtained in step 2, send the anchor size obtained in step 4 to the RPN network to obtain the region proposal box required by the subsequent detection network, and then train on the VOC format training data set obtained in step 1; finally, complete the prediction of the model.

[0116] experiment

[0117] 1. Experimental Setup

[0118] This paper adopts the training method of transfer learning and uses the pre-trained model obtained on TI to initialize our model. The training set of the experiment uses the VOC format dataset M, and the test set uses the test set of VOC 2012. In addition, the batch size is set to 4, the total epoch of training is 20, the number of iterations is 25,000, the learning rate is set to 0.001 in the first 15,000, the learning rate is set to 0.0001 in the next 1,000, the momentum is set to 0.9, the loss function used in the experiment is the classification loss function and the bounding box regression loss, and the other settings are the same as the Faster RCNN with pyramid structure. All experiments were implemented on NVIDIA RTX2080Ti.

[0119] 2. Evaluation indicators

[0120] The experimental results are mainly evaluated from mAP (unit: %) and Inference time (unit: ms). mAP (mean Average Precision) represents the average AP of all object categories, which is usually used as the final metric for evaluating detection performance. Inference time represents the time required to detect each image. The smaller the value, the faster the detection speed.

[0121] 3. Ablation Experiment

[0122] In order to study the impact of each of our proposed components on the detection performance, we conducted the following ablation experiments with Faster RCNN with a pyramid structure as the benchmark. As shown in Table 1, compared with the baseline method without adding any components, LBNet shortens the inference time by nearly half and improves the mAP by 0.8%, which shows that LBNe improves the detection efficiency while maintaining stable detection accuracy. Since CFA is completely independent of the reasoning process of Faster RCNN, it usually does not bring any additional computational consumption. Therefore, the algorithm improves the mAP by 1.1% without affecting the detection efficiency. More importantly, the mAP of using LBNet and CFA at the same time is 2.2% higher than that of the baseline method.

[0123] Table 1 Comparison of the impact of LBNet and CFA on detection performance

[0124]

[0125] In addition, we also made some comparisons on the visual effects of bounding box positioning. Figure 4As shown in the figure, the first column of images in each figure is the detection result using the method in this paper, and the other two columns are the detection results of k-means and manual design. It can be seen that the bounding box obtained by using CFA is the most accurate. This shows that CFA plays a great role in assisting the RPN network to obtain more accurate candidate boxes, that is, CFA generates multi-scale anchors that fit the data set better, which helps to improve the detection and positioning capabilities of the model.

[0126] 4. Comparison with other methods

[0127] Table 2 Comparison of detection results of Pascal VOC 2012 test dataset

[0128]

[0129] In order to fully demonstrate the advantages of our method, we compared the results of other classic methods on the Pascal VOC dataset. Faster RCNN combined with FPN is used as the baseline method. As shown in Table 2, compared with other two-stage methods, our method has the highest mAP of 83.3%, which is 2.2% higher than the baseline method. At the same time, although our method is not as fast as the single-stage method in detection speed, its mAP is 5.3% higher than YOLOv3.

Claims

1. A target detection method based on a lightweight backbone network, characterized in that: Follow the steps below to implement it: Step 1: Prepare the pre-training dataset Tiny-Imagenet and the training dataset in VOC format; Step 2: Build the feature extraction network LBNet; The step 2 is specifically implemented according to the following steps: Step 2.1, design the internal connection sub-block; Step 2.2: Design a residual connection block, add the input and output of each internal connection sub-block by skip connection, and then form a residual connection block; Step 2.3, design a local dense connection block, and concatenate the output layer features of each internal connection sub-block with the output features of all previous internal connection sub-blocks in the corresponding channel dimension to form a local dense connection block; Step 2.

4. Design the first network layer of LBNet C 1, C 1 is a convolutional layer structure consisting of a convolution kernel with a stride of 1, a size of 3x3, and 32 channels, a Batch Normalization layer, and an activation function ReLu; Step 2.5: Design the network layer structure in LBNet with residual connection blocks and dense connection blocks as the main body C 2, C 3. C 4. C 5. C 6; Step 2.6: Design the downsampling layer in LBNet d ; Step 3: Build a feature pyramid structure consisting of multi-layer output features of LBNet, and pre-train it in the Tiny-Imagenet dataset obtained in step 1 to obtain pre-training weights; The step 3 is specifically implemented according to the following steps: Step 3.1: Design a horizontal connection structure and connect the network layer in LBNet through the horizontal connection structure. C 2~ C The output features of 6 are used as the input of the feature pyramid; Step 3.2: Build a top-down feature pyramid structure consisting of multiple layers of LBNet output features; first use a 1x1x64 convolution kernel to convolve the feature layers F 6. F 5. Reduce dimension to get new feature layer P 6. P 5. This gives us the first feature output layer P 6, and then use the bilinear interpolation method to P 6 upsampled to P 5same size; finally, P 6 and P 5 Adding on its corresponding dimension, we get the updated second feature output layer P 5. Based on the same strategy as above, use P 5 Update P 4. Use P 4 Update P 3. Use P 3 Update P 2; Step 3.3: Send the images in the pre-training dataset TI into the feature pyramid structure formed by LBNet, set the number of categories according to the label file in step 1.1.2, perform pre-training, and obtain the pre-training weights; Step 4: Design the anchor generation container algorithm CFA to generate the anchor size required by the region recommendation network; The step 4 is specifically implemented according to the following steps: Step 4.1, input the training set image with the bounding box, i.e., the sample; Step 4.2: Select from all input images k The bounding boxes are used as cluster centers; Step 4.3: Calculate and compare the distances between all bounding boxes and each cluster center in turn D , and divide the class where the bounding box belongs; Step 4.4: Update the cluster center in each cluster, repeat steps 4.2 to 4.4 until no samples in each cluster change, and then select the final anchor size; Step 5: Complete model training and prediction.

2. The target detection method based on a lightweight backbone network according to claim 1, characterized in that: The step 1 is specifically implemented according to the following steps: Step 1.1, prepare the pre-training dataset Tiny-Imagenet and generate the label file required for pre-training; The step 1.1 is specifically implemented according to the following steps: Step 1.1.1, extract a part from the Imagenet dataset to make the required pre-training dataset Tiny-Imagenet, that is, the pre-training dataset TI; Step 1.1.2, generate a label file, and build a json type label file according to the actual object name corresponding to each category; Divide TI into training set and validation set in a ratio of 8:2, then sample each category contained in TI to obtain label files train.csv and val.csv respectively; Step 1.2: Prepare a training dataset in VOC format; Step 1.2 is implemented as follows: Step 1.2.1, merge the Pascal VOC2007 and 2012 datasets widely used for object detection to create the training dataset M in the VOC format we need; Step 1.2.2, generate label files, annotate the categories and regions of the images in the training set and validation set, generate XML files, and put them in the Annotations folder. Generate train.txt, val.txt and trainval.txt label files required for subsequent training based on the XML files, and put them in the Main folder.

3. The target detection method based on a lightweight backbone network according to claim 1, characterized in that: The step 2.1 is specifically implemented according to the following steps: Step 2.1.1, the structure of each connected sub-block is the same, consisting of a convolution kernel of size 1x1 and a convolution kernel of size 3x3; the specific forward propagation process is Conv(1x1)-BN-ReLu- Conv(3x3)-BN-ReLu; Step 2.1.2: Ensure that the input and output sizes of the internally connected sub-blocks are consistent, and set the number of channels of the 1x1 convolution kernel in the internally connected sub-block to 1 / 2 of the number of channels of the 3x3 convolution kernel.

4. The target detection method based on a lightweight backbone network according to claim 3 is characterized in that: The step 2.5 is specifically implemented according to the following steps: Step 2.5.1: C 2 and C 5 is set as a layer structure consisting of local densely connected blocks; C 2 is a local dense connection block consisting of 3 internal connection sub-blocks. C 5 is a densely connected block consisting of 4 internally connected sub-blocks; Step 2.5.2: C 3. C 4. C 6 is set to a layer structure consisting of residual connection blocks; C 3. C 4. C 6 are residual connection blocks consisting of 4, 6 and 6 internal connection sub-blocks respectively.

5. The target detection method based on a lightweight backbone network according to claim 4, characterized in that: The step 2.6 is specifically implemented according to the following steps: Step 2.6.

1. In LBNet C 1~ C Between two adjacent network layers of 6, a downsampling layer d with a size of 3x3 and a step size of 2 is used to complete the downsampling operation; Step 2.6.2: For the network layer C 2~ C For example, in Figure 6, the convolution kernel with a stride of 2 and a size of 3x3 is moved to the main path outside the residual connection block, that is, the image downsampling is completed before each independent connection block.

6. The target detection method based on a lightweight backbone network according to claim 5, characterized in that: The step 3.1 is specifically implemented according to the following steps: Step 3.1.1, set the LBNet output feature layer; select the first local dense connection block C 2 starts to output features and C The output of 2 is recorded as F 2. In order to obtain richer multi-scale feature information, the next three residual connection blocks are connected C 3. C 4. C 6 feature information and 1 local dense connection block C Therefore, the five feature layers are recorded as follows: F ={ F 2, F 3. F 4. F 5. F 6}; Step 3.1.2: Uniformly set the number of channels of the output layer and reduce the dimension of the output layer; complete the above through a convolution kernel of size 1x1 C 2 ~C Dimensionality reduction of feature layer channels in 6, due to C The number of channels of 2 is 64, so the number of channels here is uniformly set to 64.

7. The target detection method based on a lightweight backbone network according to claim 1, characterized in that: The step 4.2 is specifically implemented according to the following steps: Step 4.2.1: Randomly select a bounding box as the initial cluster center X 1; Step 4.2.2: Calculate all samples in the dataset and X 1, the sample with the smallest IOU becomes the second cluster center X 2; and so on, until you get k Cluster centers, denoted as G ={ X 1, X 2, ..., X k }.

8. The target detection method based on a lightweight backbone network according to claim 7, characterized in that: The step 4.4 is specifically implemented according to the following steps: Step 4.4.1: All the Bbox Sort by area from smallest to largest; Step 4.4.2: Take the sample represented by the median as the new cluster center; Step 4.4.3, obtain the final anchors size; since FPN outputs multi-scale features to RPN, three anchors sizes generated by CFA are provided for the five prediction feature layers respectively, and transmitted to the subsequent network; the specific sizes are: first layer: [13,20], [28,31], [29,66], second layer: [60,40], [55,103], [97,67], third layer: [75,175], [121,120], [272,120], fourth layer: [126,249], [185,174], [212,292], fifth layer: [382,205], [321,368], [473,336]; Step 5 is specifically implemented according to the following steps: complete the training and prediction of the model; load the pre-trained weights obtained in step 2, send the anchor size obtained in step 4 to the RPN network to obtain the region proposal box required by the subsequent detection network, and then train on the VOC format training data set obtained in step 1; finally, complete the prediction of the model.

Images