Low-altitude remote sensing target detection method

Abstract

The embodiment of the invention relates to the technical field of low-altitude remote sensing target detection methods, in particular to a low-altitude remote sensing target detection method, and the method comprises the steps: obtaining a to-be-detected low-altitude remote sensing image, and processing the to-be-detected low-altitude remote sensing image through a pre-constructed sparse cross attention-based low-altitude remote sensing target detection model. The method comprises the following steps: obtaining a detection target of a low-altitude remote sensing image, outputting the detection target of the low-altitude remote sensing image, effectively extracting global feature information through a newly designed feature enhancement module, effectively fusing deep semantic information and shallow information through a newly designed small target feature fusion module, and reserving a relatively large feature size. A small target is better detected, a sparse mechanism is utilized, only correlation between most relevant feature information is considered, the efficiency is improved while the fusion effect is ensured, and the feature learning ability can be effectively enhanced through a newly designed global semantic information fusion module.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer vision technology, and in particular to a low-altitude remote sensing target detection method. Background Art

[0002] Optical remote sensing image target detection technology is a hot research technology in the current computer vision field. Especially with the development of drone technology, the demand for target detection in low-altitude drone images is particularly urgent. Low-altitude remote sensing images have complex environments and many types of backgrounds, including small targets such as pedestrians and motor vehicles. In addition, drone shooting is highly flexible and the target scale varies, which brings many difficulties to remote sensing target detection. Therefore, the detection of small targets and multi-scale targets in low-altitude drone remote sensing targets is of great significance for practical applications.

[0003] In recent years, deep learning technology has developed rapidly and has been widely used in the field of computer vision. With the application of deep learning technology, deep learning-based target detection frameworks such as YOLOv5 and YOLOv8 have been widely used in the field of daily target detection. However, this type of algorithm has limited detection effect on low-altitude drone images with bird's-eye view, rich scales, and small targets, and has low accuracy, making it difficult to directly apply it in actual tasks.

[0004] Hu Binghao et al. proposed a method for detecting targets using dilated convolutions with positional attention in the article "Multi-target detection method for drone images with dilated convolutions with positional attention". This method uses cascaded dilated convolutions to extract features and proposes a positional attention module to enhance the features, achieving good detection accuracy. However, this method has a slow detection speed, limited detection accuracy for small targets, and fails to integrate multi-scale global information well. Summary of the invention

[0005] In view of this, an embodiment of the present application proposes a low-altitude remote sensing target detection method, which aims to input the image to be detected into the network, extract features through the backbone network, enhance the features through the GCAM module, fuse multi-scale features using the CSAM module, and then perform global semantic information fusion on the features through the MFEM module. The feature is then input into the detection head for detection, thereby achieving accurate detection of small targets.

[0006] To achieve the above-mentioned purpose, an embodiment of the present application provides a low-altitude remote sensing target detection method, including: obtaining a low-altitude remote sensing image to be detected; using a pre-built low-altitude remote sensing target detection model based on sparse cross-attention to process the low-altitude remote sensing image to be detected, and obtaining the detection target of the low-altitude remote sensing image; the low-altitude remote sensing target detection model based on sparse cross-attention includes a backbone network, a neck network and a head network connected in sequence; wherein, the SPPF layer in the CSPDarknet is replaced by multiple feature enhancement modules to obtain the backbone network; a small target feature fusion module is added to the output end of the FPN upper loop of YOLOv8PAFPN to obtain the neck network; a small target detection head is added to YOLOv8head, and a global semantic information fusion module is added before the original large target detection heads of the YOLOv8head to obtain the head network; and the detection target of the low-altitude remote sensing image is output.

[0007] Optionally, the pre-built low-altitude remote sensing target detection model based on sparse cross attention is used to process the low-altitude remote sensing image to be detected to obtain the detection target of the low-altitude remote sensing image, including: using the feature enhancement modules of the 5th, 7th, and 10th layers of the backbone network to extract spatial global information and channel global information in the low-altitude remote sensing image to be detected to obtain multi-channel enhanced feature information; using the neck network to process multi-channel enhanced feature information to obtain multi-channel enhanced feature information of different scales, and using the small target feature fusion module to fuse the output features of the 3rd layer of the backbone network and the output features of the FPN upper loop , obtaining fused feature information after the deep semantic information and the shallow information are fused; using the multiple global semantic information fusion modules of the head network to process the enhanced feature information of each channel at different scales, correspondingly obtaining multiple receptive field expansion feature information, using the multiple large target detection heads to process the multiple receptive field expansion feature information, correspondingly obtaining multiple large target detection results of the low-altitude remote sensing image, and using the small target detection head to process the fused feature information to obtain the small target detection result of the low-altitude remote sensing image, and obtaining the detection target of the low-altitude remote sensing image based on the multiple large target detection results and the small target detection results.

[0008] Optionally, the feature enhancement module includes: a first convolutional layer, a second convolutional layer, a third convolutional layer, a global average pooling layer, a global maximum pooling layer and a Softmax layer; the feature enhancement module using the 5th, 7th and 10th layers of the backbone network extracts the spatial global information and channel global information in the low-altitude remote sensing image to be detected to obtain multi-channel enhanced feature information, including: inputting the low-altitude remote sensing image to be detected into the backbone network to obtain the first input features of each layer in the 5th, 7th and 10th layers of the backbone network; using the first convolutional layer and the second convolutional layer of each of the feature enhancement modules to process the first input features respectively, and correspondingly obtain the first convolutional features and the first convolutional features. Two convolution features, use the third convolution layer to process the first convolution feature and the second convolution feature to obtain the third convolution feature, and obtain the first output feature according to the added third convolution feature and the first input feature; use the global average pooling layer, the global maximum pooling layer and the Softmax layer to process the first output feature to obtain the second output feature; obtain the global feature based on the product of the second output feature and the first output feature; obtain the third output feature according to the product of the first output feature after dimensionality reduction and the global feature after dimensionality reduction; obtain the enhanced feature information of each channel based on the added third output feature and the first input feature.

[0009] Optionally, the small target feature fusion module includes a fourth convolutional layer, a CSAB layer, an MLP layer and a sixth convolutional layer; the small target feature fusion module is used to fuse the output features of the third layer of the backbone network and the output features of the FPN upper loop to obtain a fusion feature information after the deep semantic information and the shallow information are fused, including: wherein, the fourth convolutional layer is used to process the output features of the third layer of the backbone network and the output features of the FPN upper loop respectively to obtain the fourth convolutional feature and the fifth convolutional feature, and the fourth convolutional feature and the fifth convolutional feature are respectively combined with the The output features of the third layer and the output features of the FPN upper loop are added to obtain the fourth output feature and the fifth output feature; the fourth output feature and the fifth output feature are processed by the CSAB layer to obtain a sparse cross-attention feature; the sixth output feature is obtained based on the added fourth output feature and the sparse cross-attention feature; the sixth output feature is processed by the MLP layer after random discarding to obtain the seventh output feature; the sum of the reduced dimensional seventh output feature and the sparse cross-attention feature after normalization processing is performed to obtain the fused feature information.

[0010] Optionally, the global semantic information fusion module includes a sixth convolutional layer, a seventh convolutional layer and an eighth convolutional layer; the multiple global semantic information fusion modules of the head network are used to process the enhanced feature information of each channel at different scales, and correspondingly obtain multiple receptive field expansion feature information, and multiple large target detection heads are used to process the multiple receptive field expansion feature information, and correspondingly obtain multiple large target detection results of the low-altitude remote sensing image, including: using the single global semantic information fusion module to process one enhanced feature information to obtain one receptive field expansion feature information; using the large target detection head corresponding to the global semantic information fusion module to process the receptive field expansion feature information to obtain a large target detection result.

[0011] Optionally, the use of the single global semantic information fusion module to process the enhanced feature information to obtain a receptive field expansion feature information includes: a first dimensionality reduction process on the enhanced feature information to obtain first dimensionality reduction data; a second dimensionality reduction process on the first dimensionality reduction data to obtain second dimensionality reduction data; using the sixth convolution layer to process the first dimensionality reduction data to obtain a sixth convolution feature, wherein the sixth convolution layer is a hole convolution layer; obtaining an eighth output feature according to the dot product of the sixth convolution feature and the second dimensionality reduction data; using the seventh convolution layer to process the first dimensionality reduction data to obtain a sixth convolution feature. The seventh convolution feature is obtained by processing the seventh convolution feature and the eighth output feature after dimension reduction, and the seventh convolution feature is obtained by performing the dot product of the seventh convolution feature and the eighth output feature after dimension reduction; the ninth output feature is processed based on the eighth convolution layer to obtain the tenth output feature, and the eighth convolution layer is a hole convolution layer; the tenth output feature, the ninth output feature and the eighth output feature are concatenated to obtain the eleventh output feature; the receptive field expansion feature information is obtained according to the weighted eleventh output feature and the enhanced feature information.

[0012] Optionally, the first convolution layer and the second convolution layer of each feature enhancement module are used to process the first input feature respectively to obtain a first convolution feature and a second convolution feature respectively, the first convolution feature and the second convolution feature are processed by the third convolution layer to obtain a third convolution feature, and the process of obtaining a first output feature according to the added third convolution feature and the first input feature is expressed by a first expression, and the first expression includes:

[0013]

[0014] Among them, X1 represents the first output feature, represents the third convolutional layer, the convolution kernel size of the third convolutional layer is 1×1, concat represents the concatenation operation, represents the first convolutional layer, the convolution kernel size of the third convolutional layer is 3×3, represents the second convolutional layer, the convolution kernel size of the second convolutional layer is 5×5, and X represents the first input feature.

[0015] The process of obtaining the third output feature according to the product of the first output feature after dimension reduction and the global feature after dimension reduction; and obtaining the enhanced feature information of each channel based on the added third output feature and the first input feature is expressed by the second expression, and the second expression includes:

[0016]

[0017] Among them, X out represents enhanced feature information, represents a dimensionality reduction operation, the X s and X c They represent the product of the first output feature and the global feature, and X represents the first input feature.

[0018] Optionally, the process of using the small target feature fusion module to fuse the output features of the third layer of the backbone network and the output features of the FPN upper loop to obtain fused feature information after the deep semantic information and the shallow information are fused is expressed by a first group of expressions, and the first group of expressions includes:

[0019]

[0020] Wherein, X1 represents the output feature of the third layer of the backbone network, X2 represents the output feature of the upper loop of the FPN, X3 represents the fourth output feature, X4 represents the sixth output feature, and X out represents the fusion feature information, represents the fourth convolutional layer, Represents dimensionality reduction operation, CSAB() represents sparse cross attention algorithm, MLB() represents key value extraction operation, drop represents random discard, and BN() represents normalization processing.

[0021] Optionally, the process of using the CSAB layer to process the fourth output feature and the fifth output feature to obtain a sparse cross attention feature adopts a second group of expressions, and the second group of expressions includes:

[0022] Q1, K1, V1 = MLP (Y1)

[0023] Q2, K2, V2 = MLP (Y2)

[0024]

[0025] Among them, Y1 and Y2 represent the fourth and fifth output features of dimensions C×H×W respectively, Q1, K1 and V1 are the QKV key values ​​of the fourth output feature, and the dimensions of Q1 and K1 are both n 2 ×HW / n 2 ×CQ, K2 and V2 are the QKV key values ​​of the fifth output feature, and the dimensions of Q2 and K2 are both n 2 ×HW / n 2 ×CQ,K I 1. K I 2. V I 1. V I 2 represent the newly assigned key values, concat represents the feature concatenation operation, and softmax() represents the classification operation.

[0026] Optionally, the process of using the single global semantic information fusion module to process the enhanced feature information to obtain the receptive field expanded feature information adopts a third group of expressions, and the third group of expressions includes:

[0027]

[0028] Y=(1-β)X long +βX short

[0029] Among them, X represents the enhanced feature information, () indicates a dilated convolution with a convolution kernel of 1×3. () indicates dimensionality reduction operation, ⊙ indicates dot product operation, concat() indicates feature concatenation operation, Xlong indicates the longest path conversion result, X short represents the shortest path conversion result, and β represents the weighting coefficient.

[0030] The low-altitude remote sensing target detection method proposed in the embodiment of the present application obtains the low-altitude remote sensing image to be detected; uses the pre-built low-altitude remote sensing target detection model based on sparse cross attention to process the low-altitude remote sensing image to be detected, and obtains the detection target of the low-altitude remote sensing image; the low-altitude remote sensing target detection model based on sparse cross attention includes a backbone network, a neck network and a head network connected in sequence; wherein, the SPPF layer in CSPDarknet is replaced by multiple feature enhancement modules to obtain the backbone network; a small target feature fusion module is added to the output end of the FPN upper loop of YOLOv8PAFPN to obtain the neck network; a small target detection head is added to YOLOv8head, and a global semantic information fusion module is added before each original large target detection head of YOLOv8head to obtain the head network; and the detection target of the low-altitude remote sensing image is output. Through the newly designed feature enhancement module, global feature information can be effectively extracted. This module uses convolutions of different scales to obtain information of different receptive fields, and then uses global average pooling and global maximum pooling to obtain channel global information and spatial global information. Finally, matrix multiplication is used to fuse global information, effectively improving the effect of global information extraction. Through the newly designed small target feature fusion module, deep semantic information and shallow information can be effectively fused, and a larger feature size can be retained to better detect small targets. Using the sparse mechanism, only the association between the most relevant feature information is considered, while ensuring the fusion effect and improving efficiency. Through the newly designed global semantic information fusion module, the feature learning ability can be effectively enhanced. This module uses a multi-branch structure to extract features of different scales, uses serial convolutional layers and hollow convolutions to expand the receptive field while reducing the amount of calculation, sets learnable parameters for weighted fusion, and significantly improves the backbone network's ability to extract features. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flow chart of a low-altitude remote sensing target detection method provided by an embodiment of the present application;

[0032] Figure 2 is an algorithm flow chart of a low-altitude remote sensing target detection method provided by an embodiment of the present application;

[0033] Figure 3 is a GCAM module structure diagram of a low-altitude remote sensing target detection method provided by an embodiment of the present application;

[0034] Figure 4 It is a CSAM module structure diagram of a low-altitude remote sensing target detection method provided by an embodiment of the present application;

[0035] Figure 5 It is a structural diagram of an MFEM module of a low-altitude remote sensing target detection method provided by an embodiment of the present application;

[0036] Figure 6 This is a detection result diagram of a low-altitude remote sensing target detection method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0037] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0038] An embodiment of the present application proposes a low-altitude remote sensing target detection method, referring to Figure 1 and Figure 2 , low-altitude remote sensing target detection methods can include:

[0039] S10, obtaining a low-altitude remote sensing image to be inspected;

[0040] Specifically, a data set is obtained, divided into a training set and a test set, the input image is scaled to 3 channels and 640×640 in width and height, and the training set is enhanced using data enhancement techniques such as translation, scaling, and mosaic enhancement to improve training efficiency.

[0041] S20. Use a pre-built low-altitude remote sensing target detection model based on sparse cross attention to process the low-altitude remote sensing image to be detected to obtain the detection target of the low-altitude remote sensing image; the low-altitude remote sensing target detection model based on sparse cross attention includes a backbone network, a neck network and a head network connected in sequence; wherein, the backbone network is obtained by replacing the SPPF layer in the CSPDarknet with multiple feature enhancement modules; a small target feature fusion module is added to the output end of the FPN upper loop of YOLOv8PAFPN to obtain the neck network; a small target detection head is added to YOLOv8head, and a global semantic information fusion module is added before the original large target detection heads of the YOLOv8head to obtain the head network; and the detection target of the low-altitude remote sensing image is output.

[0042] Specifically, the backbone network adopts the basic CSPDarknet, and the SPPF module at the end of the CSPDarknet is replaced with a feature enhancement module 100 to achieve multi-scale feature extraction;

[0043] In the examples of this application, reference is made to Figure 3The feature enhancement module includes: a first convolution layer, a second convolution layer, a third convolution layer, a global average pooling layer, a global maximum pooling layer and a Softmax layer.

[0044] In an embodiment of the present invention, step S20 may specifically include the following execution process:

[0045] S21, using the feature enhancement modules of the 5th, 7th and 10th layers of the backbone network to extract the spatial global information and channel global information in the low-altitude remote sensing image to be detected, to obtain multi-channel enhanced feature information;

[0046] In an embodiment of the present application, step S21 may specifically include the following execution process:

[0047] S211. Input the low-altitude remote sensing image to be detected into the backbone network to obtain the first input features of each of the 5th, 7th, and 10th layers of the backbone network, wherein the first input feature is the output result of the 4th, 6th, and 9th layers in CSPDarknet.

[0048] S212, using the first convolution layer and the second convolution layer of each feature enhancement module to process the first input feature respectively, to obtain a first convolution feature and a second convolution feature respectively, using the third convolution layer to process the first convolution feature and the second convolution feature to obtain a third convolution feature, and obtaining a first output feature according to the added third convolution feature and the first input feature;

[0049] S213. Use the global average pooling layer, the global maximum pooling layer and the Softmax layer to process the first output feature to obtain a second output feature; obtain a global feature based on the product of the second output feature and the first output feature; obtain a third output feature based on the product of the first output feature after dimensionality reduction and the global feature after dimensionality reduction; obtain the enhanced feature information of each channel based on the added third output feature and the first input feature.

[0050] Exemplarily, when constructing the backbone network, CSPDarknet is used, and the SPPF layer of the standard CSPDarknet is replaced with the GCAM module, i.e., the feature enhancement module. The GCAM module significantly enhances the network feature extraction capability by extracting spatial global information and channel global information. The GCAM module enhances the feature characterization capability by integrating single-channel global features and multi-channel global features. The feature X with input dimensions of C×H×W is first enlarged through the first convolutional layer and the second convolutional layer to obtain the feature X1, which is represented by the first expression, which can be expressed as:

[0051]

[0052] Among them, X1 represents the first output feature, represents the third convolutional layer, the convolution kernel size of the third convolutional layer is 1×1, concat represents the concatenation operation, represents the first convolutional layer, the convolution kernel size of the third convolutional layer is 3×3, represents the second convolutional layer, the convolution kernel size of the second convolutional layer is 5×5, and X represents the first input feature.

[0053] Next, the electronic device executes step S213 to obtain a channel weight feature X having a dimension of C×2×1 by passing the first output feature through the global average pooling and global maximum pooling layers. c . Perform matrix multiplication on the channel weight, i.e. the first output feature, and X1 to obtain the global spatial weight X s That is, the global feature. For example, the dimension of the global feature is 2×H×W. The spatial weight and the channel weight are matrix multiplied and combined with the residual to obtain the final output X out .

[0054] In the embodiment of the present application, the second expression used in step S213 may be:

[0055]

[0056] Among them, X out represents enhanced feature information, represents a dimensionality reduction operation, the X s and X c They represent the product of the first output feature and the global feature, and X represents the first input feature.

[0057] S22, using the neck network to process the multiple enhanced feature information to obtain multiple enhanced feature information of different scales, and using the small target feature fusion module to fuse the output features of the third layer of the backbone network and the output features of the FPN upper loop to obtain a fusion feature information after the deep semantic information and the shallow information are fused;

[0058] Specifically, on the basis of YOLOv8FPN, a small target feature fusion layer is added. Enhanced feature information of different scales is extracted from the 3rd, 5th, 7th, and 10th layers of the backbone network. The dimensions of enhanced feature information of different scales are 64×160×160, 128×80×80, 256×40×40, and 512×20×20, respectively. Then, the electronic device uses a feature enhancement module to reduce the dimension of the 5th and 7th layer outputs of the backbone network to 128 channels, and the 10th layer output uses a 1×1 convolution to reduce the dimension of the channel number to 128. A CSAM module is added to the neck network, and the output features of the 3rd layer of the backbone network and the output features of the upper loop of the FPN can be input into the CSAM module, that is, the small target feature fusion layer for feature fusion, and the fusion feature information with an output dimension of 64×160×160 is obtained.

[0059] In other words, the input features of the backbone network are enhanced by the GCAM module and then input into the neck network, and a small target feature fusion layer is added. The CSAM module is used to perform sparse cross-attention extraction on the small target features, and the semantic information of the spatial position corresponding to other scales is fused into the small target features to enhance the small target representation capability.

[0060] In the examples of this application, reference is made to Figure 4 , the small target feature fusion module includes a fourth convolution layer, a CSAB layer, an MLP layer and a sixth convolution layer; step S22 may include the following specific execution process:

[0061] S221, using the fourth convolutional layer to process the output features of the third layer of the backbone network and the output features of the FPN upper loop respectively, to obtain fourth convolutional features and fifth convolutional features, and adding the fourth convolutional features and the fifth convolutional features to the output features of the third layer and the output features of the FPN upper loop respectively, to obtain fourth output features and fifth output features;

[0062] S222, using the CSAB layer to process the fourth output feature and the fifth output feature to obtain a sparse cross attention feature;

[0063] S223, obtaining a sixth output feature based on the added fourth output feature and the sparse cross attention feature;

[0064] S224, using an MLP layer to process the sixth output feature after the random discarding to obtain a seventh output feature;

[0065] S225. The seventh output feature after dimension reduction after normalization and addition and the sparse cross-attention feature are added to obtain the fused feature information.

[0066] Continue to refer Figure 4, the above execution process can be expressed by the first group of expressions, which include:

[0067]

[0068] Wherein, X1 represents the output feature of the third layer of the backbone network, X2 represents the output feature of the upper loop of the FPN, X3 represents the fourth output feature, X4 represents the sixth output feature, and X out represents the fusion feature information, represents the fourth convolutional layer, () represents dimensionality reduction operation, CSAB() represents sparse cross attention algorithm, MLB() represents key value extraction calculation, drop represents random discard, and BN() represents normalization processing.

[0069] The key to the CSAM module is the sparse cross attention CSAB mechanism, which enhances the feature expression capability by extracting sparse attention from the spatial information of corresponding positions at different scales. The CSAB mechanism obtains the sparse cross attention of the two by sparsely representing the corresponding spatial positions of the input features. Its input is the features Y1 and Y2 of dimensions C×H×W, and the output is the feature Y of dimensions C×H×W. out The electronic device first calculates the corresponding QKV key value of the two input features. The dimensions of Q and K are both n 2 ×HW / n 2 ×C, find the average of the second dimension of Q and K to obtain Q r 1 and K r 2. Q r 1 and K r 2 matrix multiplication to obtain the regional correlation adjacency matrix A r . Electronic equipment for A r Select the first k largest values ​​in each row and discard the rest to obtain the index matrix I r The electronic device reassigns K and V according to the index matrix, with the corresponding index position being the original value and the remaining positions being zero, to obtain a new key value K I 1. K I 2. V I 1. V I 2. The execution process of step S222 can be represented by a second set of expressions, which include:

[0070] Q1, K1, V1 = MLP (Y1)

[0071] Q2, K2, V2 = MLP (Y2)

[0072]

[0073] Among them, Y1 and Y2 represent the fourth and fifth output features of dimensions C×H×W respectively, Q1, K1 and V1 are the QKV key values ​​of the fourth output feature, and the dimensions of Q1 and K1 are both n 2 ×HW / n 2 ×CQ, K2 and V2 are the QKV key values ​​of the fifth output feature, and the dimensions of Q2 and K2 are both n 2 ×HW / n 2 ×CQ,K I 1. K I 2. V I 1. V I 2 represent the newly assigned key values, concat represents the feature concatenation operation, and softmax() represents the classification operation.

[0074] S23, using the multiple global semantic information fusion modules of the head network to process the enhanced feature information of each channel at different scales, and correspondingly obtain multiple receptive field expansion feature information, using the multiple large target detection heads to process the multiple receptive field expansion feature information, and correspondingly obtain multiple large target detection results of the low-altitude remote sensing image, and using the small target detection head to process the fused feature information to obtain the small target detection result of the low-altitude remote sensing image, and obtain the detection target of the low-altitude remote sensing image based on the multiple large target detection results and the small target detection results.

[0075] In an embodiment of the present application, step S23 may specifically include the following execution process:

[0076] S231, using the single global semantic information fusion module to process the enhanced feature information to obtain a receptive field expansion feature information;

[0077] Specifically, step S231 may specifically include the following execution process: S2311, performing a first dimensionality reduction process on the enhanced feature information to obtain first dimensionality reduction data;

[0078] S2312, performing a second dimensionality reduction process on the first dimensionality reduction data to obtain second dimensionality reduction data;

[0079] S2313. Process the first dimension reduction data using the sixth convolutional layer to obtain a sixth convolutional feature, wherein the sixth convolutional layer is a hole convolutional layer;

[0080] S2314. Obtain an eighth output feature according to a dot product of the sixth convolution feature and the second dimensionality reduction data;

[0081] S2315. Process the eighth output feature using the seventh convolutional layer to obtain a seventh convolutional feature, wherein the seventh convolutional layer is a hole convolutional layer;

[0082] S2316: Obtain a ninth output feature by performing a dot product of the seventh convolution feature and the eighth output feature after dimensionality reduction.

[0083] S2317. Process the ninth output feature based on the eighth convolutional layer to obtain a tenth output feature, wherein the eighth convolutional layer is a hole convolutional layer;

[0084] S2318, concatenating the tenth output feature, the ninth output feature and the eighth output feature to obtain an eleventh output feature;

[0085] S2319: Obtain the receptive field expansion feature information according to the weighted eleventh output feature and the enhanced feature information.

[0086] S232. Using the large target detection head corresponding to the global semantic information fusion module to process the receptive field expansion feature information, a large target detection result is obtained.

[0087] For example, when constructing the head network, YOLOv8head is mainly used as the basis, a small target detection head is added, and the multi-scale enhanced feature information is further fused through the MFEM module, i.e., the global semantic information fusion module, before being input into the large target detection head. Among them, the MFEM module increases the receptive field through a multi-branch structure and a hole convolution, utilizes the learning ability of the control shortest path enhancement module and the control longest gradient path enhancement module, and finally weights and fuses different gradient path features through learning weights, which is conducive to the detection head's accurate detection of the target. The MFEM module inputs a feature X with a dimension of Cin×H×W, which is converted into feature X through the longest path and the shortest path. long With feature X short .

[0088] For example, reference Figure 5 , the execution process of step S231 can be expressed by a third group of expressions, wherein the third group of expressions can include:

[0089]

[0090] Y=(1-β)X long +βX short

[0091] Among them, X represents the enhanced feature information, () indicates a dilated convolution with a convolution kernel of 1×3. () represents dimensionality reduction operation, ⊙ represents dot product operation, concat() represents feature concatenation operation, X long Indicates the longest path conversion result, X short represents the shortest path conversion result, and β represents the weighting coefficient.

[0092] S30, outputting the detection target of the low-altitude remote sensing image, referring to Figure 6 , Figure 6 Target detection results of low-altitude remote sensing images.

[0093] In summary, the present invention can effectively extract global feature information through the newly designed GCAM module. The module uses convolutions of different scales to obtain information of different receptive fields, then uses global average pooling and global maximum pooling to obtain channel global information and spatial global information, and finally uses matrix multiplication to fuse global information, effectively improving the effect of global information extraction. Through the newly designed CSAM module, deep semantic information can be effectively fused with shallow information, and a larger feature size is retained to better detect small targets. By using the sparse mechanism, only the association between the most relevant feature information is considered, while ensuring the fusion effect and improving efficiency. Through the newly designed MFEM module, the feature learning ability can be effectively enhanced. The module uses a multi-branch structure to extract features of different scales, uses a series of convolutional layers and a hole convolution to reduce the amount of calculation while expanding the receptive field, sets learnable parameters for weighted fusion, and significantly improves the backbone network's ability to extract features.

[0094] Before step S20, the process also includes training a pre-built low-altitude remote sensing target detection model based on sparse cross attention, wherein the basic training parameters are shown in the following table:

[0095]

[0096] Then, the training network is used to detect the test set. The experimental results are shown in the following table:

[0097]

[0098] Finally, the comparison of detection results with other algorithms is shown in the following table:

[0099]

[0100] Through comparative experiments, it can be found that the GCAM module, CSAM module, and MFEM module have greatly improved the model accuracy, significantly improved the detection accuracy and mAP50 accuracy of the model, and also improved the recall rate to a certain extent, and are all improved compared to advanced algorithms.

[0101] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A low-altitude remote sensing target detection method, characterized in that: include: Acquire low-altitude remote sensing images to be inspected; Processing the low-altitude remote sensing image to be detected by using a pre-built low-altitude remote sensing target detection model based on sparse cross attention to obtain a detection target of the low-altitude remote sensing image; Outputting the detection target of the low-altitude remote sensing image; The low-altitude remote sensing target detection model based on sparse cross attention includes a backbone network, a neck network and a head network connected in sequence; The backbone network is obtained by replacing the SPPF layer in CSPDarknet with multiple feature enhancement modules; Add a small target feature fusion module to the output end of the FPN upper loop of YOLOv8PAFPN to obtain the neck network; A small target detection head is added to the YOLOv8head, and a global semantic information fusion module is added before each original large target detection head of the YOLOv8head to obtain the head network.

2. The low-altitude remote sensing target detection method according to claim 1, characterized in that: The method of using a pre-built low-altitude remote sensing target detection model based on sparse cross attention to process the low-altitude remote sensing image to be detected to obtain a detection target of the low-altitude remote sensing image includes: The feature enhancement modules of the 5th, 7th and 10th layers of the backbone network are used to extract the spatial global information and the channel global information in the low-altitude remote sensing image to be detected, so as to obtain multi-channel enhanced feature information; The neck network is used to process the multiple enhanced feature information to obtain multiple enhanced feature information of different scales, and the small target feature fusion module is used to fuse the output features of the third layer of the backbone network and the output features of the upper loop of the FPN to obtain a fusion feature information after the deep semantic information and the shallow information are fused; The multiple global semantic information fusion modules of the head network are used to process the enhanced feature information of different scales respectively, and correspondingly obtain multiple receptive field expansion feature information; the multiple large target detection heads are used to process the multiple receptive field expansion feature information respectively, and correspondingly obtain multiple large target detection results of the low-altitude remote sensing image; the small target detection head is used to process the fused feature information to obtain the small target detection result of the low-altitude remote sensing image; and the detection target of the low-altitude remote sensing image is obtained based on the multiple large target detection results and the small target detection results.

3. The low-altitude remote sensing target detection method according to claim 2, characterized in that: The feature enhancement module includes: a first convolution layer, a second convolution layer, a third convolution layer, a global average pooling layer, a global maximum pooling layer and a Softmax layer; The method of using the feature enhancement modules of the 5th, 7th and 10th layers of the backbone network to extract the spatial global information and the channel global information in the low-altitude remote sensing image to be detected to obtain multi-channel enhanced feature information includes: Inputting the low-altitude remote sensing image to be detected into the backbone network to obtain the first input features of each of the 5th, 7th and 10th layers of the backbone network; Using the first convolution layer and the second convolution layer of each feature enhancement module to process the first input feature respectively, and correspondingly obtain a first convolution feature and a second convolution feature, using the third convolution layer to process the first convolution feature and the second convolution feature to obtain a third convolution feature, and obtaining a first output feature according to the added third convolution feature and the first input feature; Processing the first output feature using the global average pooling layer, the global maximum pooling layer and the Softmax layer to obtain a second output feature; Obtaining a global feature based on the product of the second output feature and the first output feature; Obtaining a third output feature according to the product of the first output feature after dimensionality reduction and the global feature after dimensionality reduction; Based on the added third output feature and the first input feature, each channel of enhanced feature information is obtained.

4. The low-altitude remote sensing target detection method according to claim 2, characterized in that: The small target feature fusion module includes a fourth convolutional layer, a CSAB layer, an MLP layer and a sixth convolutional layer; The method of using the small target feature fusion module to fuse the output features of the third layer of the backbone network and the output features of the upper loop of the FPN to obtain fused feature information after the deep semantic information and the shallow information are fused includes: Using the fourth convolutional layer to process the output features of the third layer of the backbone network and the output features of the FPN upper loop respectively, to obtain fourth and fifth convolutional features, and adding the fourth and fifth convolutional features to the output features of the third layer and the output features of the FPN upper loop respectively, to obtain fourth and fifth output features; Processing the fourth output feature and the fifth output feature using the CSAB layer to obtain a sparse cross-attention feature; Obtaining a sixth output feature based on the added fourth output feature and the sparse cross attention feature; Processing the sixth output feature after random discarding using the MLP layer to obtain a seventh output feature; The fusion feature information is obtained by adding the seventh output feature after dimension reduction and the sparse cross-attention feature after normalization and addition.

5. The low-altitude remote sensing target detection method according to claim 2, characterized in that: The method of using the multiple global semantic information fusion modules of the head network to process the enhanced feature information of different scales in each channel to obtain multiple receptive field expansion feature information, and using the multiple large target detection heads to process the multiple receptive field expansion feature information to obtain multiple large target detection results of the low-altitude remote sensing image, includes: Using a single global semantic information fusion module to process one channel of enhanced feature information to obtain a receptive field enlarged feature information; The large target detection head corresponding to the global semantic information fusion module is used to process the receptive field expansion feature information to obtain a large target detection result.

6. The low-altitude remote sensing target detection method according to claim 5, characterized in that: The global semantic information fusion module includes a sixth convolutional layer, a seventh convolutional layer, and an eighth convolutional layer; The method of using a single global semantic information fusion module to process one channel of enhanced feature information to obtain a receptive field enlarged feature information includes: The first dimensionality reduction process is performed on the enhanced feature information to obtain first dimensionality reduction data; Performing a second dimensionality reduction process on the first dimensionality reduction data to obtain second dimensionality reduction data; Processing the first dimension reduction data using the sixth convolutional layer to obtain a sixth convolutional feature, wherein the sixth convolutional layer is a hole convolutional layer; Obtaining an eighth output feature according to a dot product of the sixth convolution feature and the second dimensionality reduction data; Processing the eighth output feature using the seventh convolutional layer to obtain a seventh convolutional feature, wherein the seventh convolutional layer is a hole convolutional layer; The seventh convolution feature obtains a ninth output feature according to a dot product of the seventh convolution feature and the eighth output feature after dimensionality reduction; Processing the ninth output feature based on the eighth convolutional layer to obtain a tenth output feature, wherein the eighth convolutional layer is a hole convolutional layer; The tenth output feature, the ninth output feature and the eighth output feature are concatenated to obtain an eleventh output feature; The receptive field expansion feature information is obtained according to the weighted eleventh output feature and the enhanced feature information.

7. The low-altitude remote sensing target detection method according to claim 3, characterized in that: The process of using the first convolution layer and the second convolution layer of each feature enhancement module to process the first input feature respectively to obtain a first convolution feature and a second convolution feature, using the third convolution layer to process the first convolution feature and the second convolution feature to obtain a third convolution feature, and obtaining a first output feature according to the added third convolution feature and the first input feature is represented by a first expression, and the first expression includes: Among them, X1 represents the first output feature, represents the third convolutional layer. The convolution kernel size of the third convolutional layer is 1×1. concat represents the concatenation operation. represents the first convolutional layer, and the convolution kernel size of the third convolutional layer is 3×3. represents the second convolutional layer, the convolution kernel size of the second convolutional layer is 5×5, and X represents the first input feature; The process of obtaining the third output feature according to the product of the first output feature after dimension reduction and the global feature after dimension reduction; and obtaining the enhanced feature information of each channel based on the added third output feature and the first input feature is expressed by a second expression, and the second expression includes: Among them, X out represents enhanced feature information, represents a dimensionality reduction operation, the X s and X c They represent the product of the first output feature and the global feature, and X represents the first input feature.

8. The low-altitude remote sensing target detection method according to claim 4, characterized in that: The process of using the small target feature fusion module to fuse the output features of the third layer of the backbone network and the output features of the FPN upper loop to obtain fused feature information after the deep semantic information and the shallow information are fused is represented by a first group of expressions, and the first group of expressions includes: Wherein, X1 represents the output feature of the third layer of the backbone network, X2 represents the output feature of the upper loop of the FPN, X3 represents the fourth output feature, X4 represents the sixth output feature, and X out represents the fusion feature information, represents the fourth convolutional layer, Represents dimensionality reduction operation, CSAB() represents sparse cross attention algorithm, MLB() represents key value extraction operation, drop represents random discard, and BN() represents normalization processing.

9. The low-altitude remote sensing target detection method according to claim 8, characterized in that: The process of using the CSAB layer to process the fourth output feature and the fifth output feature to obtain a sparse cross attention feature adopts a second set of expressions, and the second set of expressions includes: Q1, K1, V1 = MLP (Y1) Q2, K2, V2 = MLP (Y2) Among them, Y1 and Y2 represent the fourth and fifth output features of dimensions C×H×W respectively, Q1, K1 and V1 are the QKV key values ​​of the fourth output feature, and the dimensions of Q1 and K1 are both n 2 ×HW / n 2 ×CQ, K2 and V2 are the QKV key values ​​of the fifth output feature, and the dimensions of Q2 and K2 are both n 2 ×HW / n 2 ×CQ,K I 1. K I 2. V I 1. V I 2 represent the newly assigned key values, concat represents the feature concatenation operation, and softmax() represents the classification operation.

10. The low-altitude remote sensing target detection method according to claim 5, characterized in that: The process of using the single global semantic information fusion module to process one channel of enhanced feature information to obtain a receptive field expanded feature information adopts a third group of expressions, and the third group of expressions includes: Y=(1-β)X long +βX short Among them, X represents the enhanced feature information, () indicates a dilated convolution with a convolution kernel of 1×3. () represents dimensionality reduction operation, ⊙ represents dot product operation, concat() represents feature concatenation operation, X long Indicates the longest path conversion result, X short represents the shortest path conversion result, and β represents the weighting coefficient.

Images