A method and system for detecting small urban low-altitude drones based on improved YOLOv7

Abstract

The present invention provides a method and system for detecting small urban low-altitude drones based on an improved YOLOv7. The method comprises constructing a dataset, dividing the dataset into a training set and a test set, and building a basic YOLOv7 network structure, which includes an input terminal, a backbone network, a feature fusion network, and a detection head. The method also uses a RepVGG structure and a CBS convolutional layer to replace the first ELAN structure in the backbone network. The C3m module replaces the ELAN-H structure in the feature fusion network. A convolutional attention mechanism is added between the backbone network and the feature fusion network to obtain an improved YOLOv7 model. The improved YOLOv7 model is trained using the training set and tested using the test set to obtain a trained improved YOLOv7 model. The trained improved YOLOv7 model is then used to perform drone target recognition on actual acquired images. The present invention has the beneficial effect of significantly improving the detection accuracy and speed of small urban low-altitude drone targets in complex backgrounds.

Description

Technical Field

[0001] The present invention relates to the fields of drone countermeasure technology, computer vision and artificial intelligence technology, and in particular to a method and system for detecting small urban low-altitude drones based on an improved YOLOv7. Background Art

[0002] In recent years, the drone industry has seen rapid growth, finding widespread application in agriculture, logistics, border defense, and military applications. The increasing number of drones and the lag in control measures have led to numerous security and privacy issues, necessitating a growing demand for drone countermeasures. Currently, common drone detection technologies include acoustic, radio, radar, and computer vision. However, considering overall cost, detection range, and accuracy, computer vision-based detection technologies offer advantages such as low cost, ease of configuration, and intuitive detection results, offering promising development and application prospects.

[0003] Traditional object detection algorithms are becoming increasingly obsolete due to their complex detection processes and poor accuracy and robustness in complex backgrounds and occlusions. Deep learning-based object detection algorithms are gradually gaining prominence. The YOLO series, a typical example of a single-stage algorithm, and the new YOLO7 algorithm excel in accuracy and speed. However, for complex backgrounds like "low-altitude urban areas," detection targets like "small drones," and the high real-time detection requirements of the security sector, the original YOLOv7 model frequently missed small drones and exhibited insufficient inference speed, making it difficult to meet the requirements for real-time, high-precision detection. Summary of the Invention

[0004] To solve the above problems, the present invention provides a method and system for detecting small urban low-altitude drones based on an improved YOLOv7, so as to improve the detection accuracy of "illegal" drones in low-altitude urban areas, and to be used for the security protection of airports, military facilities, large commercial venues, private places and sensitive areas. A method for detecting small urban low-altitude drones based on an improved YOLOv7 mainly includes:

[0005] S1: Construct a dataset, divide the dataset into a training set and a test set, and build a YOLOv7 basic network structure, which includes an input terminal, a backbone network, a feature fusion network, and a detection head.

[0006] S2: Use the RepVGG structure and CBS convolutional layer to replace the first ELAN structure in the backbone network;

[0007] S3: Use the C3m module to replace the ELAN-H structure in the feature fusion network Neck;

[0008] S4: Add a convolutional attention module between the backbone network Backbone and the feature fusion network Neck to obtain an improved YOLOv7 model;

[0009] S5: Use the training set to train the improved YOLOv7 model, and use the test set to test the trained improved YOLOv7 model to obtain the trained improved YOLOv7 model;

[0010] S6: Use the trained improved YOLOv7 model to perform drone target recognition on the actual images.

[0011] Furthermore, the RepVGG structure decouples the model training phase and the inference phase through structural reparameterization:

[0012] During the training phase, the RepVGG structure mainly consists of 3×3 convolution, 1×1 convolution and identity residual branch, and the activation function is ReLU;

[0013] During the inference phase, the RepVGG structure adopts a single-path structure, which only contains 3×3 convolution and ReLU activation function.

[0014] Furthermore, the C3m module is an improvement of the C3 module. One branch of the C3 module, Conv2d+BN+SiLu, is improved into a convolutional neural network Conv2d, and the SiLu activation function in one of the Conv2d+BN+SiLu branches of another branch is replaced by the Mish activation function.

[0015] Furthermore, before the feature fusion network Neck is spliced ​​twice, the feature information is first screened, that is, a convolutional attention module CBAM is inserted before the CBS convolution layer.

[0016] Furthermore, the calculation formula of the feature map after the convolution attention module is:

[0017] M(F)=σ(M C (F)+M S (F)) (1)

[0018] Among them, F is the input feature map, M c (F) represents the channel attention module, M s (F) represents the spatial attention module, and σ represents the sigmoid function.

[0019] Furthermore, the calculation formula of the feature map after the channel attention module is:

[0020]

[0021] Among them, MLP() represents the fully connected layer, AvgPool() represents average pooling, MaxPool() represents maximum pooling, W0 and W1 are the weights of the shared fully connected layer, represents the feature map obtained by average pooling, Represents the feature map obtained by maximum pooling.

[0022] Furthermore, the calculation formula of the feature map after the spatial attention module is:

[0023]

[0024] Among them, f 7×7 Represents a 7×7 convolution.

[0025] A system for detecting small urban low-altitude UAVs based on an improved YOLOv7 algorithm comprises a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement a method for detecting small urban low-altitude UAVs based on an improved YOLOv7 algorithm.

[0026] The technical solution provided by this invention has the following beneficial effects: utilizing a lightweight network model to improve inference speed while increasing the utilization of drone feature information, thereby reducing the occurrence of missed detections. Secondly, by integrating a convolutional attention mechanism, the network's sensitivity to multi-scale drone targets is enhanced, reducing the impact of background noise in complex environments, and ultimately achieving real-time, high-precision drone detection. This invention significantly improves both the detection accuracy and speed of "small drone targets in complex low-altitude urban environments." BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0028] Figure 1 This is a flowchart of a method for detecting small urban low-altitude drones based on improved YOLOv7 in an embodiment of the present invention.

[0029] Figure 2 Schematic diagram of a data set preparation process in an embodiment of the present invention.

[0030] Figure 3 It is a structural diagram of YOLOv7 in an embodiment of the present invention.

[0031] Figure 4 2 is a diagram of the RepVGG structure in an embodiment of the present invention.

[0032] Figure 5 1 is a structural diagram of the C3 module before and after improvement in an embodiment of the present invention.

[0033] Figure 64 is a structural diagram of CBAM in an embodiment of the present invention.

[0034] Figure 7 1 is a diagram of the improved YOLOv7 network structure in an embodiment of the present invention.

[0035] Figure 8 It is a schematic diagram of the operation of the hardware device in the embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0037] The embodiments of the present invention provide a method and device for detecting small urban low-altitude drones based on an improved YOLOv7. First, a RepVGG lightweight network is introduced into the backbone network to reduce computational complexity to meet real-time requirements. Second, a C3m module is proposed to address problems such as the small proportion of small drone targets in the image and limited feature information, thereby improving detection accuracy. On this basis, a CBAM convolutional attention mechanism is inserted between the backbone network and the feature fusion network to improve the network's sensitivity to multi-scale drone targets and reduce the impact of noise.

[0038] Please refer to Figure 1 , Figure 1 This is a flow chart of a method for detecting small urban low-altitude drones based on improved YOLOv7 in an embodiment of the present invention, specifically including:

[0039] S1: Prepare UAV image datasets, such as Figure 2 As shown in the figure, the specific process is as follows: the drone video is acquired through a dual-light camera, and then the acquired video is frame-extracted to obtain the drone image, and then the data image is annotated;

[0040] Build as Figure 3 The basic YOLOv7 network structure and the required virtual environment are shown in the figure. The YOLOv7 basic network structure specifically includes the input, the backbone network, the feature fusion network, and the detection head. The image enters the backbone network through the input for feature extraction, then aggregates the features in the feature fusion network, and finally sends them to the head for detection.

[0041] Figure 3 In [1], the CBS module is composed of a convolutional layer Conv, a batch normalization layer BN and a SiLU activation function, which is mainly used to enhance the network feature expression capability.

[0042] SPPC (Spatial Pyramid Pooling, Cross-Stage Partial Channel) consists of two parts: Spatial Pyramid Pooling (SPP) and Cross-Stage Partial Channel Connection (CSPC). The SPP module divides the feature map into sub-regions of different scales for pooling, thereby fixing the feature map size. The CSPC module splits the feature map into two parts, one for convolution and the other for direct output. By combining these two sub-modules, SPPC can reduce the computational effort by half while maintaining high accuracy.

[0043] MaxPool represents maximum pooling.

[0044] Cat (Concat) represents a concatenation operation, which is used to concatenate different feature maps in the channel dimension.

[0045] Add is an addition operation, which is used to add two feature maps or tensors of the same shape element by element, that is, to add the elements in corresponding positions.

[0046] ELAN stands for Extended Efficient Layer Aggregation Network.

[0047] The full name of REP is RepConv structure. Its basic idea is to use a multi-branch residual structure during training and a single-path structure during inference.

[0048] UPSample refers to the upsampling operation.

[0049] MP module: A structure with two branches, one of which first undergoes a maximum pooling and then a CBS convolution block, and the other branch passes through two CBS convolution blocks with different convolution kernel scales and strides, and finally the two branches are connected by a concat. This structure is called an MP module.

[0050] S2: Using a layer such as Figure 4 The RepVGG architecture shown here replaces the first ELAN architecture in the backbone network with a CBS convolutional layer. The ELAN architecture adds more CBS computational blocks to the network, improving feature extraction capabilities but increasing computing resources and training time. This is inconsistent with the high-real-time requirements of drone detection. The RepVGG architecture is based on the VGG (Visual Geometry Group) architecture and introduces an identity residual branch. It also decouples the model training and inference phases through structural reparameterization.

[0051] During training, the RepVGG architecture primarily consists of 3×3 convolutions, 1×1 convolutions, and identity layers, with the ReLU activation function. During inference, the RepVGG architecture adopts a single-path architecture, consisting solely of 3×3 convolutions and the ReLU activation function. Because 3×3 convolutions are four times faster than 1×1 convolutions, inference is faster. Furthermore, the single-path architecture eliminates the need to store intermediate results, consuming less memory and offering greater flexibility. Therefore, replacing the first ELAN in the YOLOv7 backbone network with the RepVGG architecture, with its advantages of speed, memory usage, and flexibility, can improve drone detection speed without sacrificing accuracy.

[0052] S3: Use the C3m module to replace the ELAN-H structure in the feature fusion network Neck;

[0053] After the backbone network feature extraction, the image will enter the Neck part for feature fusion. The ELAN-H module in the Neck has a complex structure, a large amount of calculation, and can only splice information of fixed gradient flows. This paper proposes a new C3m module to replace the ELAN-H module. The C3m module is improved based on the C3 module. The structural diagrams of the two are compared. Figure 5 As shown in the figure, BottleNeck represents the bottleneck layer. The C3m module simplifies one branch in the C3 module, transforming the Conv2d+BN+SiLu branch into a convolutional neural network (Conv2d), reducing computational effort. Furthermore, the SiLu activation function used after fusing the data from the two branches is replaced with a smoothed Mish activation function. This improved C3m module balances detection accuracy and computational effort, reducing computational effort without sacrificing accuracy.

[0054] S4: Add a convolutional attention module between the backbone network Backbone and the feature fusion network Neck to obtain the following Figure 7 The improved YOLOv7 model shown in the figure includes a CBS structure, a RepVGG structure, several CBAM structures and multiple C3m modules.

[0055] In the YOLOv7 network, the feature fusion network will respectively splice the 24th and 37th layer feature information in the backbone network, and then pass through a convolution layer with a changed number of channels. In order to allow more effective drone feature information to flow into the feature fusion network, the present invention first screens the feature information before the two splicing of the feature fusion network, that is, insert a convolution layer before the convolution layer. Figure 6The simple and effective Convolutional Block Attention Module (CBAM) shown. Figure 6 In the figure, Input Feature represents input features, Channel AttentionModel represents channel attention module, Spatiol Attention Model represents spatial attention module, and RefinedFeature represents refined features.

[0056] In the channel attention module, the feature map first undergoes max pooling and average pooling, then passes through a shared fully connected layer, and finally passes through a sigmoid activation function to obtain the channel attention map. The channel attention map is then fed into the spatial attention module, where it undergoes max pooling and average pooling in the channel dimension, a convolutional layer to adjust the number of channels, and finally passes through a sigmoid activation function to obtain the spatial attention map. Therefore, each feature point in the input feature map is assigned a corresponding weight value, which is weighted by multiplication on the input feature layer channel by channel. This enhances the expressiveness of the input feature map and improves detection accuracy, while also avoiding the problem of vanishing or exploding gradients.

[0057] The specific calculation process is as follows:

[0058] M(F)=σ(M C (F)+M S (F)) (1)

[0059]

[0060]

[0061] Formula (1) is the calculation formula of CBAM, assuming that the input feature map is F∈R c×H×W , where R represents the feature map in the channel dimension, c represents the number of channels, H and W represent the height and width of the feature map respectively, then M c (F) represents the channel attention module, M s (F) represents the spatial attention module, and σ represents the sigmoid function.

[0062] Formula (2) is the calculation formula of the feature map after the channel attention module, where MLP() represents the fully connected layer, AvgPool() represents the average pooling, MaxPool() represents the maximum pooling, and W0∈R c / r+c , W1∈R c+c / r, W0 and W1 share the weights of the fully connected layer, W0 is mainly used for feature compression, W1 is mainly used for feature augmentation, and the number of channels of the feature map R is composed of the scaled channel number c / r and the original channel number c, where r represents the scaling factor. represents the feature map obtained by average pooling, Represents the feature map obtained by maximum pooling.

[0063] Formula (3) is the calculation formula of the feature map after the spatial attention module, f 7×7 Represents a 7×7 convolution.

[0064] S5: In step S1, the labeled drone images are obtained, and the data set is divided into a training set: test set: validation set = 6:2:2. The improved YOLOv7 model is trained using the training set, and the detection effect of the drone images is tested and verified using the test set and validation set. When the preset accuracy is reached, the trained model is obtained.

[0065] S6: Use the trained improved YOLOv7 model to perform drone target recognition on the actual images.

[0066] See Figure 8 , Figure 8 4 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically includes: an urban low-altitude small drone detection system 401 based on improved YOLOv7, a processor 402 and a storage device 403.

[0067] An urban low-altitude small drone detection system 401 based on improved YOLOv7: The urban low-altitude small drone detection system 401 based on improved YOLOv7 implements the urban low-altitude small drone detection method based on improved YOLOv7.

[0068] Processor 402: The processor 402 loads and executes the instructions and data in the storage device 403 to implement the urban low-altitude small drone detection method based on improved YOLOv7.

[0069] Storage device 403: The storage device 403 stores instructions and data; the storage device 403 is used to implement the urban low-altitude small drone detection method based on improved YOLOv7.

[0070] The beneficial effects of this invention include utilizing a lightweight network model to improve inference speed while increasing the utilization of drone feature information, thereby reducing the occurrence of missed detections. Secondly, by integrating a convolutional attention mechanism, the network's sensitivity to multi-scale drone targets is enhanced, reducing the impact of background noise in complex environments, and ultimately achieving real-time, high-precision drone detection. This invention significantly improves both the detection accuracy and speed of "small drone targets in complex low-altitude urban environments."

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting small urban low-altitude drones based on improved YOLOv7, characterized by: include: S1: Construct a dataset, divide the dataset into a training set and a test set, and build a YOLOv7 basic network structure, which includes an input terminal, a backbone network, a feature fusion network, and a detection head. S2: Use the RepVGG structure and CBS convolutional layer to replace the first ELAN structure in the backbone network; The RepVGG structure decouples the model training phase and the inference phase through structural reparameterization: During the training phase, the RepVGG structure mainly consists of 3×3 convolution, 1×1 convolution and identity residual branch, and the activation function is ReLU; During the inference phase, the RepVGG structure adopts a single-path structure, which only contains 3×3 convolution and ReLU activation function; S3: Use the C3m module to replace the ELAN-H structure in the feature fusion network Neck; The C3m module is an improvement of the C3 module. One branch of the C3 module, Conv2d+BN+SiLu, is improved to a convolutional neural network Conv2d, and the SiLu activation function in one of the Conv2d+BN+SiLu branches of the other branch is replaced with the Mish activation function. S4: Add a convolutional attention module between the backbone network Backbone and the feature fusion network Neck to obtain an improved YOLOv7 model; Before the feature fusion network Neck is spliced ​​twice, the feature information is first screened, that is, a convolutional attention module CBAM is inserted before the CBS convolution layer. The calculation formula of the feature map after the convolution attention module is: M(F)=σ(M C (F)+M S (F)) (1) Among them, F is the input feature map, M c (F) represents the channel attention module, M s (F) represents the spatial attention module, σ represents the sigmoid function; The calculation formula of the feature map after the channel attention module is: Among them, MLP() represents the fully connected layer, AvgPool() represents average pooling, MaxPool() represents maximum pooling, W0 and W1 are the weights of the shared fully connected layer, represents the feature map obtained by average pooling, Represents the feature map obtained by maximum pooling; The calculation formula of the feature map after the spatial attention module is: Among them, f 7×7 Represents a 7×7 convolution; S5: Use the training set to train the improved YOLOv7 model, and use the test set to test the trained improved YOLOv7 model to obtain the trained improved YOLOv7 model; S6: Use the trained improved YOLOv7 model to perform drone target recognition on the actual images.

2. An urban low-altitude small drone detection system based on improved YOLOv7, characterized by: include: Processor and storage device; the processor loads and executes instructions and data in the storage device to implement the urban low-altitude small drone detection method based on improved YOLOv7 as described in claim 1.

Images