A lightweight target detection method for drones

Abstract

The present invention discloses a lightweight target detection method for a drone end. First, the drone end aerial images are collected and annotated to obtain a data set; secondly, the CSPBiA-YOLO model is constructed based on the YOLOX‑tiny model; then, the CSPBiA‑YOLO model is trained with the collected data set; finally, the trained CSPBiA‑YOLO model is used to detect the aerial images. The present invention can be effectively mounted on an embedded device on the drone end to achieve real-time detection; it effectively solves the problem that the drone aerial photography has a large field of view, complex and diverse backgrounds, uneven target distribution, large scale changes, and a large number of small targets, which makes it more difficult to detect targets.

Description

Technical Field

[0001] The present invention belongs to the field of target detection, and in particular relates to a lightweight target detection method for a drone. Background Art

[0002] In recent years, with the continuous improvement of intelligent manufacturing technology, more and more drones are used in the fields of urban traffic supervision, power inspection, agricultural insurance, etc. in the civil field; in the military field, drones can be used for battlefield reconnaissance, surveillance and target search and positioning, providing effective information for combat while interfering with enemy electronic equipment, thereby greatly reducing the casualty rate. Drones have the advantages of low cost, high flexibility, simple operation and small size, which can make up for the shortcomings of satellite and manned aerial remote sensing technology and give rise to more diversified application scenarios. Intelligent analysis and processing of drone images can not only extract ground object information quickly and efficiently, but also expand the scene understanding ability of drones. Target detection technology can automatically identify and locate targets in images. This technology can enhance the perception function of drones under weak human-computer interaction and provide basic technical support for their autonomous detection and flight. However, due to the large aerial field of view, complex and diverse background, uneven target distribution, large scale changes and a large number of small targets, the detection of drones is more difficult. Therefore, the rapid and accurate recognition of aerial images is an important research content for realizing the integration of drones into social development. Summary of the invention

[0003] Purpose of the invention: To address the problems of low computing power and low recognition accuracy of embedded devices mounted on UAV equipment, the present invention provides a lightweight target detection method for UAVs.

[0004] Technical solution: The present invention proposes a lightweight target detection method for a UAV, comprising the following steps:

[0005] (1) Collecting aerial images from the drone and annotating them to obtain a data set;

[0006] (2) Using the YOLOX-tiny model as a benchmark, the CSPBiA-YOLO model is constructed;

[0007] (3) Train the CSPBiA-YOLO model using the collected data set;

[0008] (4) Use the trained CSPBiA-YOLO model to detect aerial images.

[0009] Furthermore, the data set in step (1) includes pedestrians, bicycles, cars, vans, trucks, tricycles, convertible electric vehicles, buses and nine other categories.

[0010] Furthermore, the CSPBiA-YOLO model described in step (2) includes:

[0011] Feature extraction network: includes three convolutional layers, a cross-stage connection layer, a convolutional layer, a cross-stage connection layer, a convolutional layer, a cross-stage connection layer, a convolutional layer, a spatial pyramid pooling layer, and a cross-stage connection layer in sequence; the convolutional layer includes depthwise separable convolution, batch normalization, and SiLU activation function; the cross-stage connection layer includes two branches, the first branch first performs a convolutional layer process, then performs two convolutional layer processes on it, and then stacks the results before the two convolutional layer processes to obtain the processing result of the first branch; the second branch only performs a convolutional layer process once, then stacks the processing result with the first branch, and then performs a convolutional layer process again to obtain the final result;

[0012] Feature fusion network: using a weighted bidirectional feature pyramid network, the feature layers after each cross-stage connection layer in the feature extraction network are sequentially processed by the attention layer as the input of the weighted bidirectional feature pyramid network, and the attention layer is processed once after each upsampling and downsampling operation in the weighted bidirectional feature pyramid network, and finally three effective feature layers are obtained; the attention layer uses an attention module based on convolution blocks, and the attention map is sequentially inferred along the two dimensions of channel and space, and then the attention map is multiplied with the input feature map for adaptive feature optimization;

[0013] A decoupling head structure is used on the three effective feature layers obtained. Each effective feature layer will first input a convolution layer for channel adjustment, and then be divided into two branches. The first branch performs two convolution layer processes and then one convolution process to obtain output result 1. The second branch performs two convolution layer processes and then is divided into two small branches. The two small branches perform convolution processes respectively to obtain output results 2 and 3. Output results 1, 2, and 3 are stacked in the channel dimension to obtain the output of the network.

[0014] Furthermore, the implementation process of step (3) is as follows:

[0015] Stochastic gradient descent is used as the optimizer, the initial learning rate is set to 1e-2, the minimum learning rate is 1e-4, the weight decay coefficient is 5e-4, the momentum of the learning rate is set to 0.937, the network is trained for 300 batches, the number of samples for each training is set to 16, and the learning rate is decreased by cosine annealing.

[0016] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: after training the data set, the present invention can effectively identify and locate the target on the drone side; it can be effectively mounted on the embedded device on the drone side to achieve real-time detection; the present invention improves the YOLOX-tiny network and proposes a CSPBiA-YOLO model, which is lighter in size than the original network and has better detection effect than the original network. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the CSPBiA-YOLO model structure. DETAILED DESCRIPTION

[0018] The present invention is further described in detail below with reference to the accompanying drawings.

[0019] In order to meet the requirements of real-time and accuracy on embedded devices, the present invention provides a lightweight target detection method for drones, targeting the characteristics of dense targets, large number of small targets, and target occlusion in drone perspective images, which specifically includes the following steps:

[0020] Step 1: Collect aerial images from the drone and annotate them to obtain a data set.

[0021] The data uses the onboard computer of the drone connected to the Hikvision camera to capture images. A total of 11,629 images were taken at two altitudes of 10m and 20m, during the day and at night, at intersections and squares, etc., including 7,629 training sets, 1,000 validation sets and 3,000 test sets. LabelImg is used to annotate the categories and locations of the targets in the images, including nine categories: pedestrians, bicycles, cars, vans, trucks, tricycles, convertible electric vehicles, buses and others.

[0022] Step 2: Build the CSPBiA-YOLO model based on the YOLOX-tiny model.

[0023] like Figure 1 As shown, the CSPBiA-YOLO model includes:

[0024] Feature extraction network: It includes three convolutional layers, one cross-stage connection layer, one convolutional layer, one cross-stage connection layer, one convolutional layer, one cross-stage connection layer, one convolutional layer, one spatial pyramid pooling layer, and one cross-stage connection layer. The convolutional layer includes depthwise separable convolution, batch normalization, and SiLU activation function. The cross-stage connection layer includes two branches. The first branch first performs a convolutional layer process, then performs two convolutional layer processes on it, and then stacks the results before the two convolutional layer processes to obtain the processing result of the first branch; the second branch only performs a convolutional layer process once, then stacks the processing result with the first branch, and then performs a convolutional layer process again to obtain the final result.

[0025] Feature fusion network: Using a weighted bidirectional feature pyramid network, the feature layers after each cross-stage connection layer in the feature extraction network are processed by the attention layer in sequence as the input of the weighted bidirectional feature pyramid network, and the attention layer is processed once after each upsampling and downsampling operation in the weighted bidirectional feature pyramid network, and finally three effective feature layers are obtained. The attention layer uses an attention module based on convolution blocks, and infers the attention map in sequence along the two dimensions of channel and space, and then multiplies the attention map with the input feature map for adaptive feature optimization.

[0026] A decoupling head structure is used on the three effective feature layers obtained. Each effective feature layer will first input a convolution layer for channel adjustment, and then be divided into two branches. The first branch performs two convolution layer processes and then one convolution process to obtain output result 1. The second branch performs two convolution layer processes and then is divided into two small branches. The two small branches perform convolution processes respectively to obtain output results 2 and 3. Output results 1, 2, and 3 are stacked in the channel dimension to obtain the output of the network.

[0027] In this implementation, the model input is set to (416, 416, 3). First, a convolution layer with a convolution kernel size of 3 and a convolution layer with a convolution kernel size of 1 are processed to obtain a feature layer with a size of (208, 208, 12), which is denoted as C. 1 ; for C 1 After a convolution layer with a convolution kernel size of 3 and a cross-stage connection layer, a feature layer of size (104, 104, 24) is obtained, denoted as C 2 ; for C 2 After a convolution layer with a convolution kernel size of 3 and a cross-stage connection layer, a feature layer of size (52, 52, 48) is obtained, denoted as C 3 ; for C 3 After a convolution layer with a convolution kernel size of 3 and a cross-stage connection layer, a feature layer of size (26, 26, 96) is obtained, denoted as C4 ; for C 4 After performing a convolution layer with a convolution kernel size of 3, a spatial pyramid pooling layer, and a cross-stage connection layer, a feature layer of size (13, 13, 192) is obtained, denoted as C. 5 .

[0028] First, C 3 , C 4 , C 5 The three feature layers are processed by the attention layer once to obtain three feature layers, which are recorded as First, The layer is processed by a convolution layer with a convolution kernel size of 3, an upsampling and an attention module to obtain a feature layer of size (26, 26, 96), denoted as Will and After weighted feature fusion, a cross-stage connection layer is processed to obtain a feature layer of size (26, 26, 96), denoted as Will A convolution layer with a convolution kernel size of 3, an upsampling and an attention module are processed to obtain a feature layer of size (52, 52, 48), denoted as Will and After weighted feature fusion, a cross-stage connection layer is processed to obtain a feature layer of size (52, 52, 48), which is recorded as Will After one downsampling and one attention layer processing, we get a feature map of size (26,26,96), denoted as Will and After weighted fusion, a cross-stage connection layer is processed to obtain a feature layer of size (26, 26, 96), denoted as Will After one downsampling and one attention layer processing, we get a feature map of size (13,13,192), denoted as Will and After weighted fusion, a cross-stage connection layer is processed to obtain a feature map of size (13, 13, 192), denoted as

[0029] right and A convolution layer with a convolution kernel size of 1 is performed to adjust the channels, and three feature maps with sizes of (52, 52, 96), (26, 26, 96) and (13, 13, 96) are obtained, which are respectively denoted as and Will Perform two convolution layers with a convolution kernel size of 3 and then perform a normal convolution with a convolution kernel size of 1 to obtain a feature layer of size (52, 52, 10), denoted as Will Perform two convolutional layers with a convolution kernel size of 3 to obtain a feature layer of size (52, 52, 96), denoted as right Two ordinary convolutions with a convolution kernel size of 1 are performed to obtain feature layers of size (52,52,1) and (52,52,4), denoted as and Will and The stacking results in a feature layer of size (52,52,15), denoted as Out 3 ;Will Perform two convolution layers with a convolution kernel size of 3 and then perform a normal convolution with a convolution kernel size of 1 to obtain a feature layer of size (26, 26, 10), denoted as Will Perform two convolutional layers with a kernel size of 3 to obtain a feature layer of size (26, 26, 96), denoted as right Perform two ordinary convolutions with a kernel size of 1 to obtain feature layers of size (26, 26, 1) and (26, 26, 4), denoted as and Will and The stacking results in a feature layer of size (26, 26, 15), denoted as Out 4 ;Will Perform two convolution layers with a convolution kernel size of 3 and then perform a normal convolution with a convolution kernel size of 1 to obtain a feature layer of size (13, 13, 10), denoted as Will Perform two convolutional layers with a kernel size of 3 to obtain a feature layer of size (13, 13, 96), denoted as right Perform two ordinary convolutions with a kernel size of 1 to obtain feature layers of size (13, 13, 1) and (13, 13, 4), denoted as and Will and The stacking results in a feature layer of size (13,13,15), denoted as Out 5 .

[0030] Step 3: Train the CSPBiA-YOLO model using the collected dataset.

[0031] During the training process, the operating system is Ubuntu 18.04 system, NVIDIA Tesla K80 is used as GPU to train the designed network, the deep learning framework adopted is Pytorch, and the stochastic gradient descent method is used as the optimizer in network training. The initial learning rate is set to 1e-2, the minimum learning rate is 1e-4, the weight decay coefficient is 5e-4, and the momentum of the learning rate is set to 0.937. The network is trained for 300 batches, the number of samples for each training is set to 16, and the learning rate is reduced by cosine annealing.

[0032] Step 4: Use the trained CSPBiA-YOLO model to detect the aerial images.

[0033] After the model training is completed, the model is deployed to the embedded device Nvidia Jetson Nano on the drone side. The operating system of Nvidia Jetson Nano is Ubuntu 18.04, the CPU is 4core ARM A57, the GPU is 128core Maxwell, the memory is 4G, and the development framework is Pytorch 1.6. TensorRT is used to accelerate the model reasoning. After acceleration, the reasoning speed of the model on Nvidia Jetson Nano reaches 42ms per frame, which meets the detection speed requirements on embedded devices.

Claims

1. A lightweight target detection method for drones, characterized in that: The following steps are involved: (1) Collecting aerial images from the drone and annotating them to obtain a data set; (2) Using the YOLOX-tiny model as a benchmark, the CSPBiA-YOLO model is constructed; (3) Train the CSPBiA-YOLO model using the collected data set; (4) Use the trained CSPBiA-YOLO model to detect aerial images; The CSPBiA-YOLO model described in step (2) includes: Feature extraction network: It includes three convolutional layers, one cross-stage connection layer, one convolutional layer, one cross-stage connection layer, one convolutional layer, one cross-stage connection layer, one convolutional layer, one spatial pyramid pooling layer, and one cross-stage connection layer in sequence; Feature fusion network: using a weighted bidirectional feature pyramid network, the feature layers after each cross-stage connection layer in the feature extraction network are sequentially processed by the attention layer as the input of the weighted bidirectional feature pyramid network, and the attention layer is processed once after each upsampling and downsampling operation in the weighted bidirectional feature pyramid network, and finally three effective feature layers are obtained; A decoupling head structure is used on the three effective feature layers obtained. Each effective feature layer will first input a convolution layer for channel adjustment, and then be divided into two branches. The first branch performs two convolution layer processes and then one convolution process to obtain output result 1. The second branch performs two convolution layer processes and then is divided into two small branches. The two small branches perform convolution processes respectively to obtain output results 2 and 3. Output results 1, 2, and 3 are stacked in the channel dimension to obtain the output of the network; The cross-stage connection layer includes two branches. The first branch first performs a convolution layer process, and then performs two convolution layer processes on it and stacks the results before the two convolution layer processes to obtain the processing result of the first branch; the second branch only performs a convolution layer process once and then stacks it with the processing result of the first branch and performs another convolution layer process to obtain the final result.

2. The lightweight target detection method for drones according to claim 1 is characterized in that: The data set in step (1) includes pedestrians, bicycles, cars, vans, trucks, tricycles, convertible electric vehicles, buses and nine other categories.

3. The lightweight target detection method for drones according to claim 1 is characterized in that: The implementation process of step (3) is as follows: Stochastic gradient descent is used as the optimizer, the initial learning rate is set to 1e-2, the minimum learning rate is 1e-4, the weight decay coefficient is 5e-4, the momentum of the learning rate is set to 0.937, the network is trained for 300 batches, the number of samples for each training is set to 16, and the learning rate is decreased by cosine annealing.

4. The lightweight target detection method for drones according to claim 1, characterized in that: The convolutional layer includes depthwise separable convolution, batch normalization, and SiLU activation function.

5. The lightweight target detection method for drones according to claim 1, characterized in that: The attention layer uses a convolutional block-based attention module to infer the attention map along the channel and space dimensions in turn, and then multiplies the attention map with the input feature map for adaptive feature optimization.

Images