A remote sensing image detection method based on an improved YOLOv7 algorithm

Abstract

The application discloses a kind of based on improved YOLOv7 algorithm remote sensing image detection method, including the following steps: first, according to YOLOv7 model builds remote sensing detection network, by sequentially carrying out parameter initialization, setting hyperparameter, loading training set sample and the mode of setting iteration number to the remote sensing detection network, realize model training to obtain optimal detection model;Second, test set sample is imported in the optimal detection model obtained and is tested, and class classification and position regression are carried out to test set sample, and final detection result is obtained.The application improves the detection precision of small object and lightens network model;It can adapt to various complex scenes;Optimize the dense target expression ability and loss function, alleviate small target dense overlapping problem, and good detection effect is obtained on remote sensing image detection.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image target detection technology, and in particular to a remote sensing image detection method based on an improved YOLOv7 algorithm. Background Technology

[0002] Remote sensing image target detection is a challenging task in the field of computer vision. It aims to accurately detect and locate targets in a given aerial remote sensing image, identifying the specific category of the target. Remote sensing image detection is indispensable in applications such as pest and disease monitoring, geological surveys, and military target identification. Therefore, remote sensing image target detection has profound research significance.

[0003] Early remote sensing image detection was achieved through traditional methods. Traditional target detection methods for visible light remote sensing images can be categorized into template matching-based, knowledge representation-based, image target analysis-based, and traditional machine learning-based algorithms. These algorithms all rely on manually defined rules or templates, lacking a unified standard and exhibiting poor robustness. When dealing with unsuitable datasets, features describing the target are typically designed manually, followed by traditional machine learning methods. However, traditional algorithms suffer from drawbacks such as poor robustness, cumbersome manual design, poor generalization ability, and inapplicability to complex backgrounds.

[0004] In recent years, deep learning-based object detection technology has seen significant advancements, with many object detection tasks incorporating deep learning, including remote sensing image detection. However, compared to natural images, remote sensing images contain numerous small targets with weaker information representation, making it easy to lose detailed information during feature extraction. This presents significant challenges for detection, particularly for densely packed targets with large scale variations. Furthermore, environmental changes such as lighting and occlusion in real-world scenes can also lead to difficulties in localization and classification during detection. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a remote sensing image detection method based on the improved YOLOv7 algorithm.

[0006] Technical solution: The present invention includes the following steps:

[0007] First, a remote sensing detection network is built based on the YOLOv7 model. The model is trained to obtain the optimal detection model by sequentially initializing parameters, setting hyperparameters, loading training set samples, and setting the number of iterations.

[0008] Secondly, the test set samples are imported into the obtained optimal detection model for testing, and the test set samples are classified and regressed to obtain the final detection results.

[0009] Furthermore, the remote sensing detection network includes a feature extraction network, a feature augmentation network, a feature fusion network, and an optimized loss function.

[0010] Furthermore, the feature enhancement network includes a context aggregation module and a multi-scale receptive field module.

[0011] Furthermore, the optimized loss function uses Adam as the model optimizer.

[0012] Furthermore, the testing method for the remote sensing detection network includes:

[0013] (1) Obtain the effective feature map P i (i=2,3,4), where P2 is the shallow feature map, P3 is the mid-to-high level feature map, and P4 is the deep feature map;

[0014] (2) Introduce a multi-receptive field layer aggregation module to replace the ELAN module in YOLOv7;

[0015] (3) A context-based dynamic feature refinement module was introduced;

[0016] (4) Optimize the localization loss by replacing the CIoU Loss applicable to localization with the Wise-IoU v3 loss function. The expression for the Wise-IoU v3 loss function is as follows:

[0017]

[0018] in, IoU is the bounding box loss IoU, where x and y are the x and y coordinates of the predicted box center point, respectively. gt y gt These are the x and y coordinates of the center point of the true bounding box, respectively, W g H g , respectively, represent the width and height of the minimum bounding rectangles of the predicted and ground truth bounding boxes; * indicates the separation operation from the computational graph; K is the non-monotonic focusing coefficient constructed by outlier β. In the formula IoU It is a moving average.

[0019] Further, step (1) includes:

[0020] (1.1) Remove the fourth ELAN and one forward MP module in YOLOv7 to remove the entire P5 detection layer;

[0021] (1.2) Connect the P4 detection layer to the feature fusion layer through a 1×1 convolution and a 2x channel compression, and then connect the SPPCSPC module in YOLOv7 to the 1×1 convolution.

[0022] (1.3) Connect the P2 detection layer to the feature fusion layer through 1×1 convolution and 2 times the number of channels. The connection method of the rest is the same as that of the P3 layer. Add a detection head to the P2 layer.

[0023] Further, step (2) includes:

[0024] (2.1) The multi-receptive field layer aggregation module replaces the two convolutional kernels of the original layer aggregation module with a large receptive field transformation module. The two kernels are 3×3 and the stride is 1.

[0025] (2.2) The first branch of the large convolution kernel transformation module uses a common 3×3 convolution kernel to extract features, and then uses activation plus normalization;

[0026] (2.3) The second branch of the large convolution kernel transformation module uses the Split function to divide the channels of the original feature map into 4 parts with a depth ratio of 1:1:1:5. Then the output feature layers after the 4 branches are vertically spliced, and then the output depth separable convolution is output.

[0027] (2.4) The output of the second branch of the large convolutional kernel transformation module is added to the first branch and output, and the large receptive field transformation module replaces the two convolutional kernels of the original layer aggregation module with ordinary convolutional modules of 3×3 and stride of 1.

[0028] (2.5) The multi-receptive field layer aggregation module mainly consists of two branches. The first branch uses a normal convolution with a 1×1 kernel to compress the original channels, reducing the channel depth to half of the original, and obtaining an output Q1;

[0029] (2.6) The second branch first uses a regular convolution with a kernel of 1×1 to compress the original number of input channels to half, resulting in an output Q2. Q2 is then input into the large convolution kernel conversion module to obtain an output Q3. Q3 is then input into a regular convolution with a kernel of 3×3 and a stride of 1 to obtain Q4. Q4 is then input into the large convolution kernel conversion module to obtain Q5. Q5 is then input into a regular convolution with a kernel of 3×3 and a stride of 1, with the channel size remaining unchanged, to output Q6.

[0030] (2.7) The first branch Q1 and the second branch Q2, Q3, Q4, Q5 and Q6 are vertically concatenated to obtain the output feature layer. The input is a 1×1 convolution kernel with a stride of 1, and the channels are compressed by 4 times before output.

[0031] Furthermore, in step (2.3), the channels of the original feature map are respectively input to depthwise separable convolution kernel 3×3, asymmetric depthwise separable convolution kernel 1×5, 5×1 and shortcut branch.

[0032] Further, step (3) includes:

[0033] (3.1) The P2 detection layer output from the feature fusion layer is downsampled using a 3×3 convolution kernel with a stride of 2 to compress its width and height back to the original value. The channel depth becomes twice as deep as before;

[0034] (3.2) The P4 detection layer output from the feature fusion layer has a 3×3 convolution kernel with a stride of 1. Then, it passes through the UpSample upsampling module to double the width and height, and the channel depth becomes the original. times;

[0035] (3.3) The feature map outputs t2, t3, and t4 are concatenated in the channel direction, and then compressed by 3 times through ordinary convolution with a kernel of 3×3 and a stride of 1 to output the feature layer;

[0036] (3.4) Split the P2 detection layer output by the feature fusion layer and then expand it through three modules with 3×3 convolutional kernels and dilation rates of 1, 2 and 3 to expand the receptive field and output the feature layer.

[0037] (3.5) The feature layer output in step (3.4) is subjected to global average pooling, and then expanded by a factor of 1 and compressed to half its original size by two convolutions with 1×1 kernels before being output.

[0038] (3.6) Multiply the feature layers from step (3.5) and step (3.4) and output the result, dynamically controlling the weights of the detail information;

[0039] (3.7) Multiply the P2 detection layer output by the feature fusion layer with step (3.6) and output it to further amplify the foreground information weight;

[0040] (3.8) Upsample the feature layer in step (3.3) and adjust its size and depth, then activate it using the Sigmod function, and multiply it with the feature layer in step (3.7);

[0041] (3.9) Add steps (3.3) and (3.8) together to obtain the detection layer M3;

[0042] (3.10) Subtract the feature layer from the feature layer in step (3.7) from the feature layer in step (3.8) to obtain the detection layer M2.

[0043] Furthermore, in step (3.1), the feature map output t2 has the same size and depth as the P3 detection layer t3 output by the feature fusion layer; in step (3.2), the feature map output t4 has the same size as the P3 detection layer t3 output by the feature fusion layer.

[0044] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0045] (1) The inefficient P5 detector head of YOLOv7 was removed and a P2 detector head was added, which reduced the information loss caused by excessive downsampling, improved the detection accuracy of small objects and reduced the weight of the network model.

[0046] (2) A multi-receptive-field layer aggregation module is proposed to replace some of the layer aggregation modules, which obtains stronger multi-scale feature representations. Furthermore, an efficient multi-scale attention mechanism (EMA) with cross-space learning is embedded in the module to adapt to complex scenarios.

[0047] (3) A context dynamic feature refinement module is proposed, which highlights the information of small targets at low level by filtering redundant features, optimizes the expression ability of dense targets and loss function, alleviates the problem of dense overlap of small targets, and achieves good detection results in remote sensing image detection. Attached Figure Description

[0048] Figure 1 This is a flowchart of the present invention;

[0049] Figure 2 This is an overall framework diagram of the present invention;

[0050] Figure 3 This is a schematic diagram of a multi-receptor field aggregation module;

[0051] Figure 4 This is a schematic diagram of the dynamic feature refinement module. Detailed Implementation

[0052] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0053] like Figure 1 As shown, the present invention includes the following steps:

[0054] 1. Data Preprocessing Stage

[0055] The VisDrone dataset was acquired and analyzed. The dataset was divided into training, validation, and test sets in a 7:1:2 ratio. Before training, the images were resized to 640×640 pixels.

[0056] This dataset contains 10,209 high-resolution still images. Of these, 6,471 were used for training, 548 for validation, and 3,190 for testing. Image resolutions range from 960 × 540 to 2000 × 1500. The dataset covers 10 different object categories: pedestrians, people, bicycles, cars, vans, trucks, tricycles, chauffeured tricycles, buses, and cars.

[0057] 2. Model Training Phase

[0058] A remote sensing detection network is built according to a YOLOv7 model, and model training is realized to obtain an optimal detection model by sequentially performing parameter initialization, setting hyperparameters, loading training set samples and setting iteration times on the remote sensing detection network,

[0059] Test set samples are imported into the obtained optimal detection model for testing, and class classification and position regression are performed on the test set samples to obtain final detection results. The remote sensing detection network includes a feature extraction network, a feature increasing network, a feature fusion network and an optimized loss function, as shown in Figure 2 The optimized loss function adopts adam as a model optimizer.

[0060] Step 2.1: A remote sensing detection network is built based on a YOLOv7 model, the network includes a feature extraction network, a feature fusion network and an optimized loss function, a modified YOLOv7 backbone with strong feature extraction capability is selected as the feature extraction network, which has five stages, and layer aggregation networks are used in the last four stages to retain context information. Each stage will perform image downsampling once, and finally 3 feature maps with different resolutions will be obtained. However, the size of the remote sensing image is too small, and the features are easy to lose. Select C2, C3 and C4 as effective feature layers to send to the feature fusion network, and use 1×1 convolution, BN and ReLU to compress the channel number of the feature map to 64, 128 and 256 respectively. The dimension-reduced feature map is denoted as (point 1).

[0061] Step 2.2: The present application proposes a multi-receptive field layer aggregation module, as shown in Figure 3 P4 is processed through SPPCSPC module, 1×1 convolution, BN and ReLU, and then is twice up-sampled and spliced with P3 as the input of layer aggregation. First, 1×1 ordinary convolution compresses the input channel number to half, the second branch also uses 1×1 ordinary convolution to compress the original input channel number to half, then 3×3 ordinary convolution is used to strengthen the features, followed by a three-branch depth separable convolution and a shortcut branch. Through 3×3, 1×5 and 5×1, the receptive field is expanded, the shortcut branch speeds up the inference and retains the basic information, and the depth separable convolution reduces the calculation consumption. The outputs of each module are spliced and compressed to obtain rich front and back information of large receptive field. Finally, the EMA attention mechanism is introduced to weaken the background information and strengthen the fusion between channels to obtain a feature layer input rich in small target information (point 2).

[0062] Step 2.3: The present application proposes a context dynamic feature refinement module, as shown in Figure 4As shown, the P2 and P4 detection layers output from the feature fusion layer are downsampled and upsampled using a 3×3 convolutional kernel for width and height compression. The output feature map is then concatenated and compressed to have the same size and depth as the P3 detection layer output from the feature fusion layer, resulting in a feature layer t with rich global information. The P2 detection layer output from the feature fusion layer is split and passed through three convolutional modules with 3×3 kernels and dilation rates of 1, 2, and 3 to obtain multi-receptor field expansion. A module that performs global average pooling and then expands, compresses, and filters the receptive field layers using two 1×1 convolutional kernels is dynamically controlled to manage the weights of each receptive field layer. The dynamically filtered small target detection layer is multiplied by the original P2 layer to amplify the foreground information. By multiplying with the feature layer t with rich global information, the information of medium and large targets is enhanced, resulting in M3 for detecting medium and large objects. M3 is then used to suppress information in P2, suppressing information other than small targets and highlighting the ability to detect small targets (point 3).

[0063] Step 2.4: S4 optimizes the localization loss by replacing the CIoU Loss with the Wise-IoU v3 loss function. CIoU causes the model to ignore smaller entities due to bias, which is detrimental to the detection of small objects in remote sensing. Wise-IoU dynamically adjusts the factor, taking into account factors such as object size, occlusion, and background complexity. It can detect small objects more accurately.

[0064] 3. Model Training Phase

[0065] The test set samples, which are divided during the data processing stage, are fed into the optimal network detection model obtained after training during the model training stage for testing. Finally, the trained weight parameters are used to classify the test data and perform position regression to obtain the final detection result.

Claims

1. A remote sensing image detection method based on an improved YOLOv7 algorithm, characterized in that, Includes the following steps: First, a remote sensing detection network is built based on the YOLOv7 model. The model is trained to obtain the optimal detection model by sequentially initializing parameters, setting hyperparameters, loading training set samples, and setting the number of iterations. Secondly, the test set samples are imported into the obtained optimal detection model for testing, and the test set samples are classified and regressed to obtain the final detection results. The testing method for the remote sensing detection network includes the following steps: (1) Obtain the effective feature map P i (i=2,3,4), where P2 is the shallow feature map, P3 is the mid-to-high level feature map, and P4 is the deep feature map; (2) Introduce a multi-receptive field layer aggregation module to replace the ELAN module in YOLOv7; (3) A context-based dynamic feature refinement module was introduced; (4) Optimize the localization loss by replacing the CIoU Loss applicable to localization with the Wise-IoU v3 loss function. The expression for the Wise-IoU v3 loss function is as follows: in, IoU The bounding box loss IoU is given by x and y, which are the x and y coordinates of the center point of the predicted bounding box, respectively. gt y gt These are the x and y coordinates of the center point of the true bounding box, respectively, W g H g , , and , respectively, represent the width and height of the minimum bounding rectangles of the predicted and ground truth boxes; * indicates the separation operation from the computational graph; and K is the non-monotonic focusing coefficient constructed by the outlier β. In the formula It is a moving average; Step (1) includes: (1.1) Remove the fourth ELAN and one forward MP module in YOLOv7 to remove the entire P5 detection layer; (1.2) Connect the P4 detection layer to the feature fusion layer through a 1×1 convolution and a 2x channel compression, and then connect the SPPCSPC module in YOLOv7 to the 1×1 convolution. (1.3) Connect the P2 detection layer to the feature fusion layer through 1×1 convolution and 2 times the number of channels. The connection method of the rest is the same as that of the P3 layer. Add a detection head to the P2 layer.

2. The remote sensing image detection method based on the improved YOLOv7 algorithm according to claim 1, characterized in that, The remote sensing detection network includes a feature extraction network, a feature augmentation network, a feature fusion network, and an optimized loss function.

3. The remote sensing image detection method based on the improved YOLOv7 algorithm according to claim 2, characterized in that, The feature enhancement network includes a context aggregation module and a multi-scale receptive field module.

4. The remote sensing image detection method based on the improved YOLOv7 algorithm according to claim 2, characterized in that, The optimization loss function uses Adam as the model optimizer.

5. The remote sensing image detection method based on the improved YOLOv7 algorithm according to claim 1, characterized in that, Step (2) includes the following steps: (2.1) The multi-receptive field layer aggregation module replaces the two convolutional kernels of the original layer aggregation module with a large receptive field transformation module. The two kernels are 3×3 and the stride is 1. (2.2) The first branch of the large convolution kernel transformation module uses a common 3×3 convolution kernel to extract features, and then uses activation plus normalization; (2.3) The second branch of the large convolution kernel transformation module uses the Split function to divide the channels of the original feature map into 4 parts with a depth ratio of 1:1:1:

5. Then the output feature layers after the 4 branches are vertically spliced ​​together, and then the output depth separable convolution is output. (2.4) The output of the second branch of the large convolutional kernel transformation module is added to the first branch and output, and the large receptive field transformation module replaces the two convolutional kernels of the original layer aggregation module with ordinary convolutional modules of 3×3 and stride of 1. (2.5) The multi-receptive field layer aggregation module mainly consists of two branches. The first branch uses a normal convolution with a 1×1 kernel to compress the original channel, and the channel depth becomes half of the original, resulting in an output Q1. (2.6) The second branch first uses a regular convolution with a kernel of 1×1 to compress the original number of input channels to half, resulting in an output Q2. Q2 is then input into the large convolution kernel conversion module to obtain an output Q3. Q3 is then input into a regular convolution with a kernel of 3×3 and a stride of 1 to obtain Q4. Q4 is then input into the large convolution kernel conversion module to obtain Q5. Q5 is then input into a regular convolution with a kernel of 3×3 and a stride of 1, with the channel size remaining unchanged, to output Q6. (2.7) The first branch Q1 and the second branch Q2, Q3, Q4, Q5 and Q6 are vertically concatenated to obtain the output feature layer. The input is a 1×1 convolution kernel with a stride of 1, and the channels are compressed by 4 times before output.

6. The remote sensing image detection method based on the improved YOLOv7 algorithm according to claim 5, characterized in that, In step (2.3), the channels of the original feature map are respectively input to depthwise separable convolution kernel 3×3, asymmetric depthwise separable convolution kernel 1×5, 5×1 and shortcut branch.

7. The remote sensing image detection method based on the improved YOLOv7 algorithm according to claim 1, characterized in that, Step (3) includes the following steps: (3.1) The P2 detection layer output from the feature fusion layer is downsampled using a 3×3 convolution kernel with a stride of 2 to compress its width and height back to the original value. The channel depth becomes twice as deep as before; (3.2) The P4 detection layer output from the feature fusion layer has a 3×3 convolution kernel with a stride of 1. Then, it passes through the UpSample upsampling module to double the width and height, and the channel depth becomes the original. times; (3.3) The feature map outputs t2, t3, and t4 are concatenated in the channel direction, and then compressed by 3 times through ordinary convolution with a kernel of 3×3 and a stride of 1 to output the feature layer; (3.4) Split the P2 detection layer output by the feature fusion layer and then expand it through three modules with 3×3 convolutional kernels and dilation rates of 1, 2 and 3 to expand the receptive field and output the feature layer. (3.5) The feature layer output in step (3.4) is subjected to global average pooling, and then expanded by a factor of 1 and compressed to half its original size by two convolutions with 1×1 kernels before being output. (3.6) Multiply the feature layers from step (3.5) and step (3.4) and output the result, dynamically controlling the weights of the detail information; (3.7) Multiply the P2 detection layer output by the feature fusion layer with step (3.6) and output it to further amplify the foreground information weight; (3.8) Upsample the feature layer in step (3.3) and adjust its size and depth, then activate it using the Sigmod function, and multiply it with the feature layer in step (3.7); (3.9) Add steps (3.3) and (3.8) together to obtain the detection layer M3; (3.10) Subtract the feature layer from the feature layer in step (3.7) from the feature layer in step (3.8) to obtain the detection layer M2.

8. The remote sensing image detection method based on the improved YOLOv7 algorithm according to claim 7, characterized in that, In step (3.1), the feature map output t2 has the same size and depth as the P3 detection layer t3 output by the feature fusion layer; in step (3.2), the feature map output t4 has the same size as the P3 detection layer t3 output by the feature fusion layer.

Images