A remote sensing image target detection method based on improved lightweight YOLOv4

By improving the lightweight YOLOv4 network and combining Haar wavelet transform and coordinate attention mechanism, the real-time and accuracy problems of UAV aerial remote sensing image detection were solved, and more efficient target detection was achieved.

CN116363529BActive Publication Date: 2026-01-30YANSHAN UNIV
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202310261559.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-17
Publication Date
2026-01-30
Estimated Expiration
2043-03-17

AI Technical Summary

Technical Problem

Existing UAV aerial remote sensing image target detection methods have a huge number of model parameters and computational load, and poor real-time performance, making them unsuitable for real-time UAV target detection.

Method used

We improve the lightweight YOLOv4 network by extracting high-frequency features through Haar wavelet transform and adding them to ordinary convolutional features. We also combine the Ghost module for feature fusion and introduce a coordinate attention mechanism before FPN to optimize training parameters and improve detection accuracy and real-time performance.

Benefits of technology

It improves the target detection accuracy and real-time performance of UAV aerial remote sensing images, and enhances the ability to locate and classify targets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116363529B_ABST
    Figure CN116363529B_ABST
Patent Text Reader

Abstract

This invention discloses a target detection method for remote sensing images based on an improved lightweight YOLOv4, belonging to the field of target detection technology for UAV aerial remote sensing images. The method includes the following steps: converting the format of the remote sensing dataset; dividing the converted dataset proportionally; improving the backbone network by performing Haar wavelet transform on the input feature maps of the low and high layers of the backbone network, fusing high-frequency features through convolution using the Ghost module, and adding the fused features to the features extracted from the feature maps by ordinary convolution; improving the Neck by introducing coordinate attention on the two branches before the FPN and after the last residual structure of the improved backbone network; setting training parameters, obtaining and saving weight data; selecting the model with the smallest loss function on the validation set, loading the training weights into the network, and testing with a test set. This invention can improve the target detection accuracy and real-time performance of UAV aerial remote sensing images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of target detection technology for drone aerial remote sensing images, and in particular to a target detection method for remote sensing images based on an improved lightweight YOLOv4. Background Technology

[0002] With the rapid development of drone technology, drones are ubiquitous in people's lives. Remote sensing images taken by drones are widely used in fields such as road defect detection, urban planning, resource exploration, crop monitoring, and military reconnaissance. Remote sensing images have also developed rapidly along with drone technology, with increasingly higher spatial and spectral resolutions. However, the detection methods for processing remote sensing images have lagged behind. Therefore, target detection technology for drone-captured remote sensing images has gradually become an important research area in computer vision. While remote sensing images have high resolution and contain a lot of information, the resolution of small targets remains low, resulting in limited information and making detection difficult. Remote sensing images often suffer from high exposure and complex backgrounds, which blurs the feature information of targets, further complicating target detection. Furthermore, UAV onboard processors have low computing power, limited GPU memory, and high real-time requirements. In 2021, the inventors proposed a Chinese invention patent, "A UAV Aerial Target Detection Method Based on an Improved SSD Approach" (CN202011580609.2), which improves the network by using seven effective feature layers to detect objects, resulting in better object detection and increased detection breadth. The improved SSD introduces an RFB module, simulating the receptive field of human vision, enhancing the network's feature extraction capabilities. Compared to existing technologies, it significantly improves accuracy and the number of detections, making it more practical. However, the method in this patent application has a huge number of model parameters and computational load, resulting in low real-time performance.

[0003] To address the aforementioned issues, the large number of parameters and computational demands of current advanced target detection methods significantly reduces their detection speed and real-time performance, making them unsuitable for real-time target detection by UAVs. In contrast, lightweight target detection methods, while exhibiting slightly lower detection accuracy, have lower parameter and computational demands and offer higher real-time performance, making them suitable for real-time target detection by UAVs. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a remote sensing image target detection method based on the improved lightweight YOLOv4, which can improve the target detection accuracy and real-time performance of UAV aerial remote sensing images.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A remote sensing image target detection method based on an improved lightweight YOLOv4 includes the following steps:

[0007] S1. Convert the format of the remote sensing dataset;

[0008] S2. Divide the converted remote sensing dataset into training set, validation set and test set according to the proportion;

[0009] S3. Improve the backbone network of YOLOv4-tiny by performing Haar wavelet transform on the input feature maps of the low and high layers of the backbone network to extract high-frequency features. Then, use the Ghost module to perform convolutional fusion on the high-frequency features and add the fused features to the features extracted from the feature map by ordinary convolution to obtain the improved backbone network.

[0010] S4. Improve the Neck of YOLOv4-tiny by introducing coordinate attention on the two branches before FPN and after the last residual structure of the improved backbone network to obtain the improved Neck. The coordinate attention is the channel attention improved based on SE attention.

[0011] S5. Set training parameters, use the training set to iteratively train the model, and obtain and save the weight data of the remote sensing image target detection model.

[0012] S6. After training, select the model with the smallest loss function on the validation set, load the training weights into the network, and test it with the test set to obtain the network's detection performance.

[0013] A further improvement to the technical solution of the present invention is that, in S1, the data format of the original remote sensing dataset is converted into VOC format.

[0014] A further improvement of the technical solution of the present invention is that: in S2, the training and validation set includes a training set and a validation set, the ratio of the training and validation set to the test set in the dataset is 9:1, and the ratio of the training set to the validation set in the training and validation set is 9:1.

[0015] A further improvement to the technical solution of this invention is as follows: In S3, the backbone of the original YOLOv4-tiny is improved by using Haar wavelet transform to downsample the feature map by a factor of 2. First, a one-dimensional Haar discretization is performed on the horizontal direction of the feature map, and then a one-dimensional Haar discretization is performed on the vertical direction of the decomposed feature map to obtain four components {LL, LH, HL, HH}. Among them, LL represents low-frequency features, which contain the contour feature information of the image, and {LH, HL, HH} all represent high-frequency features, which contain the feature information in the vertical, horizontal, and diagonal edge directions of the image, respectively. The size of the input feature map after Haar wavelet transform is 1 / 2 of the size of the input feature map. The high-frequency features after Ghost convolution fusion are added to the feature map after ordinary convolution to obtain more detailed features on the feature map, thereby enhancing the localization and classification of the target.

[0016] A further improvement to the technical solution of the present invention is that S3 specifically includes the following steps:

[0017] S3.1 The Haar wavelet transform is applied to the bottom-level feature map of the backbone network. The original input image is subjected to the Haar wavelet transform, and the resulting high-frequency features are superimposed. The superimposed features are denoted as Haar_H, where Haar_H∈R^(3C×H / 2×W / 2), where R is the set of real numbers, C is the number of channels, H is the width of the feature map, and W is the length of the feature map. The formula is as follows:

[0018] Haar_H=Concatenate(LH, HL, HH);

[0019] Concatenate means to overlay different feature maps;

[0020] The superimposed high-frequency features are fused by convolution using the Ghost module to learn the weights of the high-frequency features in each direction and adjust their number of channels.

[0021] S3.2 The Haar wavelet transform is applied to the top-level feature map of the backbone network. The last residual structure of the backbone network is subjected to the Haar wavelet transform, and the three high-frequency features obtained by the Haar transform are superimposed. Then, the superimposed high-frequency features are fused by convolution through three lightweight Ghost modules. The fused feature is denoted as x_H, and the formula is as follows:

[0022] x_H=Ghost(Ghost(Ghost(Haar_H2)));

[0023] Ghost represents Ghost convolution operation. Ghost convolution is performed in two steps: first, a feature map is obtained through regular convolution; then, an identity mapping is performed on this feature map to obtain a Ghost feature map; finally, the two feature maps are superimposed to obtain a fused feature map.

[0024] S3.3 combines the fused features with the features obtained from the first convolution operation in FPN, and uses the combined features as input for upsampling.

[0025] A further improvement to the technical solution of the present invention is that step S4 specifically includes the following steps:

[0026] S4.1 Coordinate Information Embedding;

[0027] Each channel is encoded horizontally and vertically using convolutional kernels of size (H, 1) and (1, W), respectively, where H is the height of the convolutional kernel and W is the width of the convolutional kernel. The output of the c-th dimension feature of height h is:

[0028]

[0029] The output of the c-th dimension feature of width w is:

[0030]

[0031] Where, x c (h,i) represents the horizontal direction of the c-th dimension of the input feature map, i represents the ith channel, and x c (j,w) represents the vertical direction of the c-th dimension feature of the input feature map, and j represents the j-th channel;

[0032] The above two formulas aggregate features in the horizontal and vertical directions respectively to generate a pair of orientation-aware feature maps. These two formulas can capture long-range dependencies in one spatial direction and capture accurate location information in another spatial direction, thereby helping the network to effectively locate the region of interest.

[0033] Note the generation of coordinates in S4.2;

[0034] The coordinate attention concatenates the feature maps generated by the above two equations, and then uses a 1×1 convolution transformation function F1 to generate f, as shown in the following formula:

[0035] f=δ(F1([z h ,z w ]))

[0036] Among them, [z h ,z w ] represents a connection operation along a spatial dimension, δ represents the activation function, and f∈

[0037] R C / r×(H+W) It is an intermediate feature map that encodes spatial information in the horizontal and vertical directions, where r represents the downsampling ratio; f is divided into two independent tensors along the spatial dimension. h ∈R C / r×H and f w ∈R C / r×W Using two 1×1 convolution transformation functions F h and F w Change f h and f w The number of channels is adjusted to match the number of channels in the input X, resulting in:

[0038] g h =σ(F h (f h ))

[0039] g w =σ(F w (f w ))

[0040] Where σ is the sigmoid function, which is the function of g h and g w As attention weights, the output of coordinate attention is:

[0041]

[0042] Coordinate attention takes into account both channel and spatial information, applying attention to both the horizontal and vertical directions of the input feature map. This allows coordinate attention to more accurately locate the precise position of the target of interest, helping the entire model to detect targets more effectively.

[0043] A further improvement to the technical solution of this invention is as follows: In S5, the original image is cropped to a uniform size of 608×608; during the training phase, the input image size is set to 608×608, the Adam optimizer is used to optimize the model, the learning rate is set to 0.001, the momentum is set to 0.92, the weight decay is set to 0.0005, the number of iterations is set to 150, and the batch normalization size is 16. After setting the parameters, the model is trained, and the relevant training weights are saved during training.

[0044] A further improvement to the technical solution of this invention is that, in S6, the model weights with the smallest loss function are selected and loaded into the network, and their detection accuracy and visualization results are tested.

[0045] A further improvement to the technical solution of this invention lies in:

[0046] The technological advancements achieved by this invention due to the adoption of the above technical solutions are as follows:

[0047] The standard YOLOv4-tiny network uses only ordinary convolutions to extract features from the feature map, resulting in limited and insufficient feature extraction. The improved YOLOv4-tiny network performs Haar wavelet transform on the feature map to extract high-frequency features, and adds these features to the original features to obtain more detailed information, leading to better detection performance. The original YOLOv4-tiny network did not incorporate an attention mechanism; this invention introduces various attention mechanisms, such as channel attention and spatial attention, and finds that the introduction of coordinate attention achieves better detection results. Coordinate attention considers both channel and spatial information, applying attention to both the horizontal and vertical directions of the input feature map simultaneously, which helps to more accurately locate regions of interest and improves the overall model's detection capabilities. Attached Figure Description

[0048] Figure 1 This is a flowchart of the target detection method of the present invention;

[0049] Figure 2This is a diagram of the YOLOv4-tiny structure;

[0050] Figure 3 This is an improved structure diagram of the underlying network based on Haar wavelet transform in this invention;

[0051] Figure 4 This is an improved structural diagram of the high-level network based on Haar transform in this invention;

[0052] Figure 5 This is a coordinate attention structure diagram in this invention;

[0053] Figure 6 This is a diagram of the improved YOLOv4-tiny model structure in this invention. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments:

[0055] like Figure 1 As shown, a remote sensing image target detection method based on an improved lightweight YOLOv4 specifically includes the following steps:

[0056] S1. Convert the format of the remote sensing dataset;

[0057] Convert the original remote sensing dataset to VOC format.

[0058] S2. Divide the converted remote sensing dataset into training set, validation set and test set according to the proportion;

[0059] The training and validation sets include a training set and a validation set. The ratio of the training and validation sets to the test set in the dataset is 9:1. The ratio of the training set to the validation set in the training and validation sets is 9:1.

[0060] In this embodiment, the remote sensing dataset after format conversion is divided into a total of 21,761 images. Among them, 17,625 images are selected as the training set, 1,959 images as the validation set, and 2,177 images as the test set. The training set is used to train the model to update the weights, the validation set is used to evaluate the model after each round of training, and the test set is used to evaluate the detection performance of the model after training is completed.

[0061] S3. Improve the backbone network of YOLOv4-tiny by performing Haar wavelet transform on the input feature maps of the low and high layers of the backbone network to extract high-frequency features. Then, use the Ghost module to perform convolutional fusion on the high-frequency features and add the fused features to the features extracted from the feature map by ordinary convolution to obtain the improved backbone network.

[0062] YOLOv4-tiny structure as follows Figure 2As shown, the low-frequency and high-frequency features of the feature map are obtained by performing Haar wavelet transform on the feature map. The low-frequency features include the original image, and the high-frequency features include detailed features in the horizontal, vertical and diagonal directions. The high-frequency features extracted by Haar wavelet transform are changed by using a lightweight Ghost

[10] module to change the number of channels, and then combined with the features extracted by the original convolutional layer (CBL) to obtain an improved backbone network (hereinafter referred to as Backbone) to enhance the extraction of detailed features and obtain more detailed features of the feature map.

[0063] In this embodiment, the backbone of the original YOLOv4-tiny is improved: the feature map is downsampled by 2 times using Haar wavelet transform. First, a one-dimensional Haar discretization is performed on the horizontal direction of the feature map, and then a one-dimensional Haar discretization is performed on the vertical direction of the decomposed feature map to obtain four components {LL, LH, HL, HH}. Among them, LL represents low-frequency features, which contain the contour feature information of the image, and {LH, HL, HH} all represent high-frequency features, which contain feature information in the vertical, horizontal, and diagonal edge directions of the image, respectively. The size of the input feature map after Haar wavelet transform is 1 / 2 of the size of the input feature map. The high-frequency features after Ghost convolution fusion are added to the feature map after ordinary convolution to obtain more detailed features on the feature map, which enhances the localization and classification of the target.

[0064] Specifically, the following steps are included:

[0065] S3.1 The Haar wavelet transform is applied to the bottom-level feature map of the backbone network. The original input image is subjected to the Haar wavelet transform, and the resulting high-frequency features are superimposed. The superimposed features are denoted as Haar_H, where Haar_H∈R^(3C×H / 2×W / 2), where R is the set of real numbers, C is the number of channels, H is the width of the feature map, and W is the length of the feature map. The formula is as follows:

[0066] Haar_H=Concatenate(LH, HL, HH);

[0067] Concatenate means to overlay different feature maps;

[0068] The superimposed high-frequency features are fused using a Ghost module through convolution to learn the weights of the high-frequency features in each direction and adjust their channel count, such as... Figure 3 As shown, the Ghost module is used instead of regular convolution. The Ghost module is a lightweight convolution method that can replace convolution operations in any convolutional network, effectively reducing the number of parameters and computational cost associated with adding modules. This method extracts more detailed features, which is beneficial for target localization.

[0069] S3.2 The Haar wavelet transform is applied to the top-level feature map of the backbone network. The last residual structure of the backbone network is subjected to the Haar wavelet transform, and the three high-frequency features obtained by the Haar transform are superimposed. Then, the superimposed high-frequency features are fused by convolution through three lightweight Ghost modules. The fused feature is denoted as x_H, and the formula is as follows:

[0070] x_H=Ghost(Ghost(Ghost(Haar_H2)));

[0071] Here, Ghost represents Ghost convolution operation. Ghost convolution is performed in two steps: first, a feature map is obtained through regular convolution; then, an identity mapping is performed on this feature map to obtain a Ghost feature map; finally, the two feature maps are superimposed to obtain a fused feature map.

[0072] S3.3 combines the fused features with the features obtained from the first convolution operation in FPN, and uses the combined features as input for upsampling, such as... Figure 4 As shown.

[0073] S4. Improve the Neck of YOLOv4-tiny by introducing coordinate attention on the two branches before FPN and after the last residual structure of the improved backbone network to obtain the improved Neck. The coordinate attention is the channel attention improved based on SE attention.

[0074] S4 consists of two steps: coordinate information embedding and coordinate attention generation, such as... Figure 5 As shown, the specific steps include:

[0075] S4.1 Coordinate Information Embedding;

[0076] Each channel is encoded horizontally and vertically using convolutional kernels of size (H, 1) and (1, W), respectively, where H is the height of the convolutional kernel and W is the width of the convolutional kernel. The output of the c-th dimension feature of height h is:

[0077]

[0078] The output of the c-th dimension feature of width w is:

[0079]

[0080] Where, x c (h,i) represents the horizontal direction of the c-th dimension of the input feature map, i represents the ith channel, and x c (j,w) represents the vertical direction of the c-th dimension feature of the input feature map, and j represents the j-th channel;

[0081] The above two formulas aggregate features in the horizontal and vertical directions respectively to generate a pair of orientation-aware feature maps. These two formulas can capture long-range dependencies in one spatial direction and capture accurate location information in another spatial direction, thereby helping the network to effectively locate the region of interest.

[0082] Note the generation of coordinates in S4.2;

[0083] The coordinate attention concatenates the feature maps generated by the above two equations, and then uses a 1×1 convolution transformation function F1 to generate f, as shown in the following formula:

[0084] f=δ(F1([z h ,z w ]))

[0085] Among them, [z h ,z w ] represents a connection operation along a spatial dimension, δ represents the activation function, and f∈

[0086] R C / r×(H+W) It is an intermediate feature map that encodes spatial information in the horizontal and vertical directions, where r represents the downsampling ratio; f is divided into two independent tensors along the spatial dimension. h ∈R C / r×H and f w ∈R C / r×W Using two 1×1 convolution transformation functions F h and F w Change f h and f w The number of channels is adjusted to match the number of channels in the input X, resulting in:

[0087] g h =σ(F h (f h ))

[0088] g w =σ(F w (f w ))

[0089] Where σ is the sigmoid function, which is the function of g h and g w As attention weights, the output of coordinate attention is:

[0090]

[0091] Coordinate attention takes into account both channel and spatial information, applying attention to both the horizontal and vertical directions of the input feature map. This allows coordinate attention to more accurately locate the precise position of the target of interest, helping the entire model to detect targets more effectively.

[0092] The model structure after improvements in S3 and S4 is as follows: Figure 6 As shown.

[0093] S5. Set training parameters, use the training set to iteratively train the model, and obtain and save the weight data of the remote sensing image target detection model.

[0094] The original images were cropped to a uniform size of 608×608. During the training phase, the input image size was set to 608×608, and the Adam optimizer was used to optimize the model with a learning rate of 0.001, momentum of 0.92, weight decay of 0.0005, 150 iterations, and a batch normalization size of 16. After setting the parameters, the model was trained, and the relevant training weights were saved during training.

[0095] S6. After training, select the model with the smallest loss function on the validation set, load the training weights into the network, and test it with the test set to obtain the network's detection performance.

[0096] The model weights with the smallest loss function are selected and loaded into the network to test its detection accuracy and visualization results.

[0097] In this embodiment, after training is completed, the model training weights with the smallest loss function on the validation set are selected and loaded into the network to obtain the network's detection performance.

[0098] In summary, this invention can improve the target detection accuracy and real-time performance of UAV aerial remote sensing images.

Claims

1. A remote sensing image target detection method based on improved lightweight YOLOv4, characterized in that: The method comprises the following steps: S1, format conversion is performed on a remote sensing data set; S2, the converted remote sensing data set is divided into a training set, a validation set and a test set in proportion; S3, the backbone network of YOLOv4-tiny is improved, Haar wavelet transformation is performed on the input feature maps of the low layer and the high layer of the backbone network respectively to extract the high-frequency features of the feature maps, convolution fusion is performed on the high-frequency features through a Ghost module, the fused features are added to the features extracted by ordinary convolution on the feature maps, and an improved backbone network is obtained; S3, specifically comprising the following steps: S3.1 applies Haar wavelet transformation to the bottom layer feature map of the backbone network, performs Haar wavelet transformation on the original input image, superimposes the obtained high-frequency features, and records the superimposed features as Haar_H, Haar_H∈R^(3C×H / 2×W / 2), wherein R is a real set, C is a channel number, H is a width of a feature map, and W is a length of a feature map, and the formula is as follows: Haar_H=Concatenate(LH,HL,HH); Wherein, Concatenate means superimposing different feature maps; The superimposed high-frequency features are convolved and fused through a Ghost module to learn the weights of the high-frequency features in each direction and adjust the channel number thereof; S3.2 applies Haar wavelet transformation to the top layer feature map of the backbone network, performs Haar wavelet transformation on the last residual structure of the backbone network, superimposes the three high-frequency features obtained by Haar transformation, then convolves and fuses the superimposed high-frequency features through three lightweight Ghost modules, records the fused features as x_H, and the formula is as follows: x_H=Ghost(Ghost(Ghost(Haar_H2))); Wherein, Ghost represents Ghost convolution operation, Ghost convolution is divided into two steps, first, a feature map is obtained through conventional convolution, then an identity mapping is performed on the feature map to obtain a Ghost feature map, and finally the two feature maps are superimposed to obtain the fused feature map; S3.3 combines the fused features with the features obtained by the first convolution operation in the FPN, and performs upsampling operation on the combined features as input; S4, the Neck of YOLOv4-tiny is improved, and a coordinate attention is introduced on the two branches before the FPN and after the last residual structure of the improved backbone network to obtain an improved Neck, and the coordinate attention is a channel attention improved based on SE attention; S5, setting training parameters, using the training set to iteratively train the model, obtaining and saving the remote sensing image target detection model weight data; S6, after the training is completed, the model training weight with the minimum validation set loss function is loaded into the network, and the test set is tested to obtain the detection performance of the network.

2. The remote sensing image target detection method based on improved lightweight YOLOv4 according to claim 1, characterized in that: In S1, the data format of the original remote sensing data set is converted into VOC format.

3. The remote sensing image target detection method based on improved lightweight YOLOv4 according to claim 1, characterized in that: In S2, the training verification set includes the training set and the verification set, and the division ratio of the training verification set and the test set in the data set is 9:1, and the division ratio of the training set and the verification set in the training verification set is 9:

1.

4. The remote sensing image target detection method based on improved lightweight YOLOv4 according to claim 1, characterized in that: In S3, the original YOLOv4-tiny Backbone is improved: the feature map is 2 times down-sampled by using Haar wavelet transform, one-dimensional Haar discrete decomposition is performed on the horizontal direction of the feature map, and one-dimensional Haar discrete decomposition is performed on the vertical direction of the decomposed feature map, obtaining {LL, LH, HL, HH} four components, wherein LL represents low-frequency features, containing the contour feature information of the image, {LH, HL, HH} all represent high-frequency features, containing the feature information of the vertical, horizontal and diagonal edge direction of the image; the size of the input feature map after Haar wavelet transform is 1 / 2 of the input feature map; the high-frequency features after Ghost convolution fusion are added to the feature map after ordinary convolution to obtain more detailed features on the feature map, and the positioning and classification of the target are strengthened.

5. The remote sensing image target detection method based on improved lightweight YOLOv4 according to claim 1, characterized in that: In S4, the following steps are specifically included: S4.1 coordinate information embedding; The horizontal and vertical directions of each channel are encoded using convolution kernels with sizes (H, 1) and (1, W) respectively, where H is the height of the convolution kernel and W is the width of the convolution kernel, and the output of the c-th dimension feature with height h is: The output of the c-th dimension feature with width w is: wherein x c (h,i) is the horizontal direction of the c-th feature of the input feature map, i is the i-th channel, and x c (k,w) is the vertical direction of the c-th feature of the input feature map, j is the j-th channel. The above two formulas aggregate the features in the horizontal and vertical directions respectively to generate a pair of direction-aware feature maps. These two formulas can capture long-range dependencies in one spatial direction and accurate location information in the other spatial direction, thereby helping the network effectively locate the region of interest; S4.2 coordinate attention generation; The coordinate attention concatenates the feature maps generated by the above two formulas, and then uses a 1x1 convolution transformation function F1 to generate f, as follows: f = δ(F1([z h ,z w ])) wherein [z h ,z w ] represents a connection operation along the spatial dimension, δ represents an activation function, f∈R C / r×(H+W) is an intermediate feature map for encoding spatial information in horizontal and vertical directions, and r represents a down-sampling ratio; f h ∈R C / r×H is divided into two independent tensors f w ∈R C / r×W along the spatial dimension by using two 1×1 convolution transformation functions F h and F w to change the channel number of f h and f w to be consistent with the channel number of the input X, to obtain: g h = σ (F h (f h )) g w = σ (F w (f w )) where σ is a sigmoid function, and g h and g w The output of coordinate attention as attention weights is: The coordinate attention considers both channel information and spatial information, and applies attention to the horizontal and vertical directions of the input feature map at the same time, which makes the coordinate attention more accurately locate the accurate position of the target of interest and help the entire model better detect.

6. The remote sensing image target detection method based on improved lightweight YOLOv4 according to claim 1, characterized in that: In S5, the original image is cropped to 608x608. In the training stage, the input picture size is set to 608x608, the model is optimized using the Adam optimizer, the learning rate is 0.001, the momentum is set to 0.92, the weight decay is set to 0.0005, the iteration number is set to 150, the batch normalization size is set to 16, and the model is trained after setting the parameters. Save the related training weights while training.

7. The remote sensing image target detection method based on improved lightweight YOLOv4 according to claim 1, characterized in that: In S6, the model weight with the smallest loss function is loaded into the network, and the detection accuracy and visual result graph are tested.

Citation Information

Patent Citations

  • SSD improved algorithm-based unmanned aerial vehicle aerial photography target detection method

    CN112668644A