Unmanned aerial vehicle image target detection method based on improved RT_DETR model

Abstract

The application belongs to the technical field of target detection, and discloses a UAV image target detection method based on an improved RT_DETR model, which introduces an FAPPA module in a backbone network, effectively alleviating the information loss problem of small target features in a deep network. The method also replaces the feedforward neural network of the AIFI module in the encoder with an EDFFN module, which selectively enhances high-frequency features such as edges and textures by using a patch_wise FFT strategy and learnable frequency domain weights. In addition, the method also uses a Zoom_cat module to realize adaptive alignment of multi-scale features, and uses the convolution_attention dual-path architecture of the CAFMFusion module to fuse local details and global semantics, thereby fully utilizing complementary information at different levels. The method can significantly improve the UAV image target detection capability and also identify small target features in a complex background.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a method for UAV image target detection based on an improved RT_DETR model. Background Technology

[0002] In the process of target detection in UAV images, the flexible flight altitude and varied shooting angles of UAVs lead to significant differences in target scale within the image. Furthermore, small targets often constitute a large proportion of UAV-captured images, resulting in blurred features and lost details. In addition, in UAV images acquired against complex backgrounds, targets often appear highly similar in height to the background, increasing the difficulty of detection. Moreover, the inherent vibrations and blurring during UAV flight further reduce image sharpness, exacerbating the uncertainties in target detection.

[0003] Secondly, the challenges posed by environmental and application requirements cannot be ignored. Drones often operate outdoors, subject to interference from changes in lighting, weather conditions, and occlusion, which can easily lead to decreased target visibility and feature stability. Simultaneously, target detection often requires a balance between real-time performance and high accuracy, but drone platforms are limited by payload and power consumption, resulting in limited computing resources and making it difficult to deploy overly complex target detection models. These factors collectively constrain the performance of drone image target detection, particularly small target detection, making it difficult to achieve significant breakthroughs.

[0004] Unmanned aerial vehicle (UAV) image target detection is an important research area in computer vision. Although significant progress has been made in recent years, it still faces many challenges. In terms of algorithmic models, deep learning, especially convolutional neural networks (CNNs), is widely used for target detection. However, UAV images contain a large number of small targets with extremely low pixel counts and indistinct features, making them prone to being missed or falsely detected.

[0005] Therefore, a target detection method for UAV images is needed that can identify small target features against complex backgrounds. Specifically, this type of target detection method must not only possess the ability to fuse multi-scale features to simultaneously capture shallow texture information and deep semantic information, but also maintain robustness under interference factors such as changes in perspective, lighting differences, and target occlusion caused by high-altitude UAV shooting. Furthermore, in practical applications, it is also necessary to balance detection accuracy and real-time performance, avoiding excessive computational resource consumption due to overly large models, which would limit deployment and application on UAV platforms. Summary of the Invention

[0006] The purpose of this invention is to propose a UAV image target detection method based on an improved RT_DETR model. This method balances detection accuracy and real-time performance, and can not only recognize small target features in complex backgrounds, but also maintain robustness under interference factors such as viewpoint changes, illumination differences and target occlusion.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] The UAV image target detection method based on the improved RT_DETR model includes the following steps:

[0009] Step 1. Preprocess the images acquired by the UAV to construct a dataset for target detection in UAV images;

[0010] Step 2. Build the improved RT_DETR model, which includes a backbone network, encoder, decoder and detector head;

[0011] The improved RT_DETR model is obtained by making the following improvements to the original RT_DETR model: A C2f_MFPA module is introduced into the backbone network. The C2f_MFPA module is obtained by replacing the Bottlenect module in the C2f module with the frequency-domain enhanced parallel attention mechanism FAPPA module; the AIFI module in the encoder is replaced with the AIFI_EDFFN module. The AIFI_EDFFN module is obtained by replacing the feedforward neural network in the AIFI module with the EDFFN module; and a multi-scale alignment Zoom_cat module and a convolutional-attention fusion CAFMFusion module are introduced into the encoder.

[0012] Step 3. Use the dataset constructed in Step 1 to train the improved RT_DETR model to obtain the trained model;

[0013] Step 4. Use the trained model obtained in Step 3 to perform target detection on the images collected by the UAV.

[0014] The present invention has the following advantages:

[0015] As described above, this invention discloses a UAV image target detection method based on an improved RT_DETR model. This method introduces a C2f_MFPA module in the backbone network, which includes a FAPPA (Frequency Domain Enhancement) parallel attention mechanism. The FAPPA module, through a four-branch parallel architecture consisting of a local perception branch, a global context branch, a serial convolution branch, and a frequency domain enhancement branch, can enhance high-frequency detail information while maintaining multi-scale features, thus effectively alleviating the information loss problem of small target features in deep networks. Furthermore, this invention replaces the feedforward neural network of the AIFI module in the encoder with an EDFFN module, which introduces explicit frequency domain modeling capabilities and can selectively enhance high-frequency features such as edges and textures using a patch-wise FFT strategy and learnable frequency domain weights, thereby improving the feature representation ability of small targets while reducing computational complexity. Furthermore, the method of this invention introduces a multi-scale alignment Zoom_cat module and a convolutional-attention fusion CAFMFusion module into the encoder. The Zoom_cat module achieves adaptive alignment of multi-scale features, while the CAFMFusion module's convolutional-attention dual-path architecture fuses local details and global semantics, thereby fully utilizing complementary information from different levels. The UAV image target detection method based on the improved RT_DETR model proposed in this invention can significantly improve the target detection capability of UAV images and can also identify small target features in complex backgrounds. Attached Figure Description

[0016] Figure 1 This is a flowchart of a UAV image target detection method based on an improved RT_DETR model in an embodiment of the present invention.

[0017] Figure 2 This is a network structure diagram of the improved RT_DETR model in an embodiment of the present invention.

[0018] Figure 3 This is a network structure diagram of the C2f_MFPA module in an embodiment of the present invention.

[0019] Figure 4 This is a network structure diagram of the FAPPA module in an embodiment of the present invention.

[0020] Figure 5 The signal processing flowchart of the local sensing branch and the global context branch in the embodiment of the invention.

[0021] Figure 6 Network structure diagram of the EDFFN module in this embodiment of the invention.

[0022] Figure 7 This is a schematic diagram of the network structure of the Zoom_cat module in an embodiment of the present invention.

[0023] Figure 8 This is a schematic diagram of the network structure of the CAFMFusion module in an embodiment of the present invention.

[0024] Figure 9 This is a network structure diagram of the CAFM module in an embodiment of the present invention.

[0025] Figure 10 The specific results obtained by using the method of the present invention for UAV image detection in this embodiment of the invention are shown below. Figure 1 .

[0026] Figure 11 The specific results obtained by using the method of the present invention for UAV image detection in this embodiment of the invention are shown below. Figure 2 .

[0027] Figure 12 The specific results obtained by using the method of the present invention for UAV image detection in this embodiment of the invention are shown below. Figure 3 .

[0028] Figure 13 The specific results obtained by using the method of the present invention for UAV image detection in this embodiment of the invention are shown below. Figure 4 . Detailed Implementation

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0030] Example 1

[0031] like Figure 1 As shown, the UAV image target detection method based on the improved RT_DETR model includes the following steps:

[0032] Step 1. Preprocess the images acquired by the UAV to construct a dataset for target detection in UAV images.

[0033] Step 1 is as follows:

[0034] The images acquired by the UAV are initialized, and the images and bounding boxes are uniformly scaled to a preset size and normalized and standardized to construct a dataset for target detection in UAV images. This dataset is in COCO format.

[0035] Feature maps from datasets used for target detection in drone images As input to the improved RT_DETR model, where Indicates the number of channels. Indicates altitude, Indicates the width.

[0036] In this embodiment, the images used to construct the dataset can be images taken by a drone or images acquired by a drone's infrared thermal imaging. The images may contain targets such as people, vans, trucks, cars, and bicycles.

[0037] In this embodiment, the model is further divided into a training set, a validation set, and a test set according to a preset ratio. The training set is used to train the improved RT_DETR model subsequently to verify its effectiveness. The validation set is used to validate the improved RT_DETR model subsequently to verify its generalization and robustness.

[0038] In this embodiment, the image and annotation boxes are specifically scaled to a uniform size of 640×640. Feature map Number of channels For 3, feature map height and width It is 640.

[0039] Step 2. Build the improved RT_DETR model, which includes a backbone network, encoder, decoder and detector head.

[0040] The improved RT_DETR model is derived from the original RT_DETR model with the following improvements: A C2f_MFPA module is introduced into the backbone network. The C2f_MFPA module is obtained by replacing the Bottlenect module in the C2f module with the frequency-domain enhanced parallel attention mechanism FAPPA. The AIFI module in the encoder is replaced with the AIFI_EDFFN module. The AIFI_EDFFN module is obtained by replacing the feedforward neural network in the AIFI module with the EDFFN module. A multi-scale alignment Zoom_cat module and a convolutional-attention fusion CAFMFusion module are introduced into the encoder.

[0041] The backbone network will be described in detail below.

[0042] like Figure 2 As shown, the backbone network consists of five convolutional layers with a kernel size of 3×3 and a stride of 2, and four C2f_MFPA modules.

[0043] The four C2f_MFPA modules are defined as the first C2f_MFPA module, the second C2f_MFPA module, the third C2f_MFPA module, and the fourth C2f_MFPA module, respectively.

[0044] The signal processing flow in the backbone network is as follows:

[0045] The backbone network is fed with feature maps from a dataset used for target detection in UAV images. .

[0046] Feature map The data is processed through two convolutional layers with a kernel size of 3×3 and a stride of 2, and then fed into the first C2f_MFPA module to extract semantic features, resulting in the feature map output by the first C2f_MFPA module. .

[0047] Output of the first C2f_MFPA module After processing by a convolutional layer with a kernel size of 3×3 and a stride of 2, the data is fed into the second C2f_MFPA module to extract semantic features, resulting in the feature map output by the second C2f_MFPA module. .

[0048] Output of the second C2f_MFPA module After processing by a convolutional layer with a kernel size of 3×3 and a stride of 2, the data is fed into the third C2f_MFPA module to extract semantic features, resulting in the feature map output by the third C2f_MFPA module. .

[0049] Output of the third C2f_MFPA module After processing by a convolutional layer with a kernel size of 3×3 and a stride of 2, the data is fed into the fourth C2f_MFPA module to extract semantic features, resulting in the feature map output by the fourth C2f_MFPA module. .

[0050] The C2f_MFPA module will be described in detail below.

[0051] like Figure 3 As shown, the C2f_MFPA module includes two convolutional layers with a kernel size of 1×1, a channel segmentation module, n FAPPA modules, and a feature fusion module.

[0052] The signal processing flow in the C2f_MFPA module is as follows:

[0053] The input to the C2f_MFPA module is a feature map. Its output is a feature map. .

[0054] Specifically, for the first C2f_MFPA module, its input feature map This refers to the feature map output by the second convolutional layer in the backbone network, which has a kernel size of 3×3 and a stride of 2. That is, feature map For the second C2f_MFPA module, its input feature map This refers to the feature map output by the third convolutional layer in the backbone network, which has a kernel size of 3×3 and a stride of 2. That is, feature map For the third C2f_MFPA module, its input feature map This refers to the feature map output by the fourth convolutional layer in the backbone network, which has a kernel size of 3×3 and a stride of 2. That is, feature map For the fourth C2f_MFPA module, its input feature map This refers to the feature map output by the fifth convolutional layer in the backbone network, which has a kernel size of 3×3 and a stride of 2. That is, feature map .

[0055] Feature map Channel expansion is performed using a convolutional layer with a kernel size of 1×1, increasing the number of channels to [number missing]. The expanded feature map is obtained. .

[0056] The channel segmentation module will expand the feature map Uniformly divided along the channel dimension into and Two parts.

[0057] Feature map output by the channel segmentation module The data will be directly input into the feature fusion module without any processing.

[0058] Feature map output by the channel segmentation module The feature maps are refined sequentially through n FAPPA modules, and the output feature maps of the first to nth FAPPA modules are respectively... to .

[0059] The feature fusion module converts the feature map output by the channel segmentation module. , Feature map output by the first FAPPA module Feature map output by the nth FAPPA module The concatenation is performed along the channel dimension, and then fed into a convolutional layer with a kernel size of 1×1 for processing to obtain the feature map output by the C2f_MFPA module. .

[0060] The C2f_MFPA module proposed in this invention, based on the architecture of the C2f module, achieves adaptive fusion of multi-scale features by integrating the frequency domain enhanced parallel attention mechanism FAPPA module. This not only improves the model's ability to model multi-scale features of targets, but also enhances the model's feature representation ability and detection robustness, thereby improving the feature representation effect in small targets and complex scenes.

[0061] The FAPPA module will be introduced in detail below.

[0062] like Figure 4 As shown, the FAPPA module includes a convolutional layer with a kernel size of 1×1, a local perception branch, a global context branch, a serial convolution branch, a frequency domain enhancement branch, a channel attention mechanism ECA module, and a spatial attention mechanism SA module.

[0063] The input to the FAPPA module is a feature map. The output of the FAPPA module is a feature map. .

[0064] The signal processing flow in the FAPPA module is as follows:

[0065] Input FAPPA module feature map After processing by a convolutional layer with a kernel size of 1×1, the feature map is obtained. .

[0066] feature map It serves as the input for the local perception branch, the global context branch, the serial convolution branch, and the frequency domain enhancement branch.

[0067] The output of the local perception branch is a feature map. The output of the global context branch is a feature map. .

[0068] Feature map The feature map is obtained by processing the data through a 3×3 convolutional layer in the sequential convolution branch. Feature map The feature map is obtained by processing the data through a convolutional layer with a kernel size of 3×3. Feature map The feature map is obtained by processing the data through a convolutional layer with a kernel size of 3×3. .

[0069] The frequency domain enhancement branch includes a Fast Fourier Transform (FFT) module, an Inverse Fast Fourier Transform (IFFT) module, and a Frequency-Gated (FGM) module. The frequency domain enhancement branch utilizes Fourier transform to enhance the input feature map... Mapping to the frequency domain improves robustness to textures, edges, and periodic structures, resulting in a feature map of the frequency-enhanced branch output. for:

[0070] .

[0071] in, This indicates a frequency domain transformation, which uses the Fast Fourier Transform. This represents the inverse frequency domain transform, which employs the inverse fast Fourier transform. It is a frequency domain gating mechanism.

[0072] By using the output of the local sensing branch Output of the global context branch Output of serial convolution branch , , and the output of the frequency domain enhancement branch Add them together to obtain the feature map. :

[0073] .

[0074] In this context, addition refers to adding the corresponding positions of each feature vector in the feature map while keeping the number of channels unchanged.

[0075] The method of this invention introduces a cascaded attention mechanism for adaptive feature enhancement, which consists of an efficient channel attention mechanism. Spatial attention mechanism The composition, and the process, can be represented by the following formula:

[0076] .

[0077] Feature map After being processed sequentially by the channel attention mechanism (ECA) module and the spatial attention mechanism (SA) module, the feature map output by the FAPPA module is obtained. In this embodiment, the feature map output by the nth FAPPA module in the C2f_MFPA module is specifically used. Recorded as .

[0078] The FAPPA module enhances fine-grained feature extraction by introducing a frequency-domain enhanced parallel attention mechanism, thereby effectively mitigating the information loss problem of small target features in deep networks.

[0079] The following section provides a detailed introduction to the local perception branch and the global context branch.

[0080] like Figure 5As shown, the signal processing flow in the local sensing branch and the global context branch is as follows:

[0081] according to Non-overlapping windows will feature maps Divide into N patches, and flatten each patch into a vector:

[0082] .

[0083] in, This represents the operation of flattening the i-th patch into a vector. This represents the vector obtained after flattening the i-th patch, with length . , , In the local perception branch =2, in the global context branch =4.

[0084] For each vector Taking the mean along the channel dimension yields a vector. for:

[0085] =Mean( ) .

[0086] in, This indicates the operation of taking the average value along the channel dimension. Indicates to The vector obtained by taking the mean along the channel dimension has a length of .

[0087] Vectors are processed using a multilayer perceptron. Projecting onto a d-dimensional feature space yields a low-dimensional representation vector. for:

[0088] .

[0089] in, This represents the first multilayer perceptron. This indicates a normalization operation. This represents the second multilayer perceptron. A multilayer perceptron is specifically a fully connected network capable of projecting vectors onto a predetermined dimension.

[0090] low-dimensional representation vector Each dimension is multiplied element-wise with its own attention weight to obtain the attention-weighted feature representation. for:

[0091] .

[0092] in, Let be the feature representation of the i-th local region after attention weighting. , The original feature strengths were preserved and reweighted according to their relative importance. This represents the softmax activation function, used for normalization. This represents the operation of calculating attention weights in the channel dimension using the softmax activation function. This indicates element-wise multiplication of matrices.

[0093] Let the learned global vector Prompt be denoted as ,calculate and cosine similarity for:

[0094] .

[0095] in, , .

[0096] Using the clamp function By restricting the cosine similarity values ​​to the interval [0,1], we obtain the gating coefficients. for:

[0097] .

[0098] The above formula corresponds to the process of generating a weight mask.

[0099] By gating coefficient Adjusting local variables Perform global filtering using a matrix. Performing a linear transformation between channels maps the selected local features to a representation space controlled by the global structure, resulting in... :

[0100] .

[0101] in, The process of preserving regions with high similarity. This is a global feature representation, which includes local features and global cue information. It utilizes matrices. Features are mapped to the output space to fuse information across multiple scales or channels.

[0102] The above formula first uses a mask to selectively preserve local information relevant to the global semantics, and then transforms it into a unified global feature representation through a linear mapping. (Matrix) Specifically, it is a transformation matrix with learnable parameters.

[0103] Will Reshape it according to its original spatial position using the reshape function. Then, after bilinear upsampling, the result is obtained. .

[0104] Through a convolutional layer with a kernel size of 1×1 Will Mapping back to the original number of channels C yields... for:

[0105] .

[0106] when When =2, That is, the output of the local sensing branch. .

[0107] when When =4, This is the output of the global context branch. .

[0108] The encoder will be described in detail below.

[0109] like Figure 2 As shown, the encoder includes six convolutional layers with a kernel size of 1×1, one AIFI_EDFFN module, two Zoom_cat modules, two upsampling modules, four reparameterized C3 modules, two convolutional layers with a kernel size of 3×3, and three CAFMFusion modules.

[0110] The six convolutional layers with a kernel size of 1×1 are defined as the first 1×1 convolutional layer, the second 1×1 convolutional layer, the third 1×1 convolutional layer, the fourth 1×1 convolutional layer, the fifth 1×1 convolutional layer, and the sixth 1×1 convolutional layer, respectively.

[0111] Define the two Zoom_cat modules as the first Zoom_cat module and the second Zoom_cat module, respectively.

[0112] The two upsampling modules are defined as the first upsampling module and the second upsampling module, respectively.

[0113] The four reparameterized C3 modules are defined as the first reparameterized C3 module, the second reparameterized C3 module, the third reparameterized C3 module, and the fourth reparameterized C3 module, respectively.

[0114] Two convolutional layers with a kernel size of 3×3 are defined as the first 3×3 convolutional layer and the second 3×3 convolutional layer, respectively.

[0115] The three CAFMFusion modules are defined as the first CAFMFusion module, the second CAFMFusion module, and the third CAFMFusion module, respectively.

[0116] The signal processing flow in the encoder is as follows:

[0117] The encoder's input is the feature map output from the first C2f_MFPA module. Feature map output by the second C2f_MFPA module The feature map output by the third C2f_MFPA module The feature map output by the fourth C2f_MFPA module .

[0118] The input to the first 1×1 convolutional layer is the feature map output by the first C2f_MFPA module. .

[0119] The input to the second 1×1 convolutional layer is the feature map output from the second C2f_MFPA module. .

[0120] The input to the third 1×1 convolutional layer is the feature map output from the third C2f_MFPA module. .

[0121] The input to the fourth 1×1 convolutional layer is the feature map output by the fourth C2f_MFPA module. .

[0122] The output of the fourth 1×1 convolutional layer is processed sequentially by the AIFI_EDFFN module, the fifth 1×1 convolutional layer, and the first upsampling module before being fed into the first Zoom_cat module for further processing. The input to the first Zoom_cat module also includes feature maps output from the second 1×1 convolutional layer and the third C2f_MFPA module. .

[0123] The output of the first Zoom_cat module is processed sequentially through the first parameterization C3 module, the sixth 1×1 convolutional layer, and the second upsampling module before being fed into the second Zoom_cat module for further processing. The input to the second Zoom_cat module also includes feature maps output from the first 1×1 convolutional layer and the second C2f_MFPA module. .

[0124] The output of the second Zoom_cat module is processed sequentially through the second parameterized C3 module and the first 3×3 convolutional layer, then concatenated with the output of the sixth 1×1 convolutional layer, and finally fed into the third parameterized C3 module.

[0125] The output of the third parameterized C3 module is processed by the second 3×3 convolutional layer, concatenated with the output of the fifth 1×1 convolutional layer, and then fed into the fourth parameterized C3 module.

[0126] The input to the first CAFMFusion module is the output of the third parameterized C3 module and the third 1×1 convolutional layer.

[0127] The input to the second CAFMFusion module is the output of the second parameterized C3 module and the second 1×1 convolutional layer.

[0128] The input to the third CAFMFusion module is the output of the fourth parameterized C3 module and the fourth 1×1 convolutional layer.

[0129] The outputs of the first CAFMFusion module, the second CAFMFusion module, and the third CAFMFusion module are used as the outputs of the encoder.

[0130] The EDFFN module will be introduced in detail below.

[0131] like Figure 6 As shown, the signal processing flow in the EDFFN module is as follows:

[0132] The input to the EDFFN module is a feature map. .

[0133] Feature maps input to the EDFFFN module After channel expansion using a 1×1 convolutional layer, and then grouped convolution processing using depthwise separable convolution, intermediate features are obtained. Among them, depthwise separable convolution is used to capture local spatial information of the input features.

[0134] intermediate features Divided along the channel dimension and Two parts, activated by the GEGLU function right Perform gating operations and connect with Element-wise multiplication is performed, followed by processing through a 1×1 convolutional layer to generate the channel enhancement result. for:

[0135] .

[0136] Channel enhancement results Apply frequency domain transformation to a local or global region of a preset size. Spectral characteristics were obtained. :

[0137] .

[0138] In the frequency domain With learnable frequency domain weights Element-wise multiplication yields the modulated spectral characteristics. :

[0139] = .

[0140] Among them, learnable frequency domain weights The initial value in the value is a constant 1, and the frequency domain weights are... With frequency domain tensor, i.e., spectral characteristics The dimension can be broadcast for matching.

[0141] through inverse frequency domain transform Will Mapping back to the spatiotemporal domain yields the output of the EDFFN module. :

[0142] .

[0143] Among them, frequency domain transformation Employing Fast Fourier Transform and Inverse Frequency Domain Transform The inverse fast Fourier transform is used.

[0144] This invention introduces an integrated frequency-domain enhanced feedforward neural network into the encoder by replacing the original feedforward neural network in the AIFI module with an EDFFN module. The EDFFN module introduces deep convolution and frequency-domain enhancement, and combines spatial and frequency domain information modeling to enhance the preservation of high-frequency detail information, improve the model's ability to express local details and global patterns, thereby improving the performance of UAV image target detection.

[0145] The Zoom_cat module will be described in detail below.

[0146] like Figure 7 As shown, the Zoom_cat module adopts a multi-branch structure, which includes a first branch, a second branch, and a third branch.

[0147] The signal processing flow in the Zoom_cat module is as follows:

[0148] The Zoom_cat module takes large-scale features as input. Mesoscale features Small-scale features .

[0149] For the input of the first Zoom_cat module, large-scale features The output of the second 1×1 convolutional layer, mesoscale features Feature map output by the third C2f_MFPA module Small-scale features This is the output of the first upsampling module. The output of the first Zoom_cat module is... .

[0150] For the input of the second Zoom_cat module, large-scale features The output of the first 1×1 convolutional layer, mesoscale features Feature map output by the second C2f_MFPA module Small-scale features This is the output of the second upsampling module. The output of the second Zoom_cat module is... .

[0151] The input to the first branch is large-scale features. The first branch deals with large-scale features respectively. Perform adaptive max pooling processing and adaptive average pooling and large-scale features The outputs after adaptive max pooling and adaptive average pooling are added together to obtain the output of the first branch. :

[0152] .

[0153] The second branch performs no processing; its input and output are both mesoscale features. .

[0154] The input to the third branch is small-scale features. The third branch focuses on small-scale features. Perform nearest neighbor interpolation operation Upsampling yields the output of the third branch. :

[0155] .

[0156] By outputting the first branch The output of the second branch The output of the third branch The output of the Zoom_cat module is obtained by concatenating the data along the channel dimension.

[0157] This invention aligns and fuses multi-scale feature maps using the Zoom_cat module, which combines semantic and detailed information while ensuring consistent spatial dimensions, thereby improving the model's ability to perceive targets of different sizes.

[0158] The CAFMFusion module will be introduced in detail below.

[0159] like Figure 8 As shown, the CAFMFusion module includes a CAFM module for local convolutional feature enhancement and global self-attention feature modeling. The signal processing flow in the CAFMFusion module is as follows:

[0160] The CAFMFusion module takes shallow features as input. and deep features .

[0161] For the input of the first CAFMFusion module, shallow features The output of the third 1×1 convolutional layer represents deep features. This is the output of the third parameterized C3 module.

[0162] For the input of the second CAFMFusion module, shallow features The output of the second 1×1 convolutional layer represents the deep features. This is the output of the second parameterized C3 module.

[0163] For the input of the third CAFMFusion module, shallow features The output of the fourth 1×1 convolutional layer represents deep features. This is the output of the fourth parameterized C3 module.

[0164] The CAFM module will use shallow features and deep features The initial fusion features obtained after addition As input features:

[0165] .

[0166] CAFM module The output features are :

[0167] .

[0168] Input characteristics of CAFM module With output features After processing using pixel attention mechanism and sigmoid function, the weights are obtained. :

[0169] .

[0170] in, This represents the pixel attention mechanism. It is the sigmoid function;

[0171] Using the generated weights For initial fusion features Shallow features and deep features Perform weighted fusion to obtain the fused features. :

[0172] .

[0173] Features after fusion After passing through a convolutional layer with a kernel size of 1×1 Perform channel compression or linear transformation to obtain the output characteristics of the CAFMFusion module. :

[0174] .

[0175] like Figure 9 As shown, the signal processing flow in the CAFM module is as follows:

[0176] The input characteristics of the CAFM module are .

[0177] Input characteristics of CAFM module After processing with a 1×1 convolutional layer to adjust the channel dimensions, a channel shuffling operation is performed to enhance the information interaction between features. Then, a 3×3×3 3D convolution operation is used to further extract spatial information features, resulting in locally enhanced convolutional features. :

[0178] .

[0179] in, For channel mixed washing operation, This is a 3D convolution operation with a kernel size of 3×3×3.

[0180] Input characteristics of CAFM module The input features are processed through a 1×1 convolutional layer, followed by depthwise separable convolutions with 3×3 kernels, to obtain the final input features. The features of queries, keys, and values , , In this embodiment, the input features are specifically processed through depthwise separable 3D convolution. The query, key, and value are enhanced with local spatial features. Based on feature rearrangement, the features are divided into multiple heads, and the global self-attention matrix is ​​obtained by calculating the dot product.

[0181] Based on the self-attention mechanism for input features Global spatial feature enhancement is performed, followed by processing through a 1×1 convolutional layer, and then combined with the features from the input CAFM module. Add them together to obtain the global features. :

[0182] .

[0183] in, This represents the scaling factor.

[0184] Enhance features through local convolution With global features Add them together to obtain the output characteristics of the CAFM module. .

[0185] This invention uses the CAFMFusion module for adaptive feature fusion. This module combines local convolution and global attention information, and dynamically adjusts the contribution of each feature through pixel-level gating of the pixel attention mechanism, thereby generating more discriminative output features.

[0186] The CAFM module introduces local spatial awareness through depthwise separable convolution, uses dynamically generated convolutional kernels to achieve local convolutional modeling to extract neighborhood details, models global dependencies in the channel dimension through multi-head self-attention, and finally adds the local convolutional enhanced features to the global features to obtain a fusion feature representation that takes into account both local details and global semantics, which is then used as the output feature of the CAFM module.

[0187] The decoder and detection head are described below.

[0188] The feature map output by the first CAFMFusion module is denoted as... The feature map output by the second CAFMFusion module is denoted as... The feature map output by the third CAFMFusion module is denoted as... .

[0189] The decoder's input includes feature maps. , and .

[0190] The feature map is decoded , and It is converted into an embedded representation suitable for detection head processing, and position encoding is added for object queries to provide position information in the sequence.

[0191] The detector head performs category prediction through a linear layer and obtains the bounding box coordinates.

[0192] Step 3. Use the dataset constructed in Step 1 to train the improved RT_DETR model to obtain the trained model.

[0193] The regression method in the detection head of this invention adopts an IoU_aware strategy to improve the accuracy of the detection box.

[0194] The IoU_aware policy consists of two components:

[0195] The IoU prediction branch runs in parallel with the regression branch and is used to predict the IoU score between the current candidate box and the ground truth box. IoU score Used as a confidence level reference.

[0196] The goal of loss optimization is to introduce IoU into the regression loss so that the model training process is closer to the actual detection and evaluation standards.

[0197] Step 4. Use the trained model obtained in Step 3 to perform target detection on the images collected by the UAV.

[0198] In this embodiment, the image obtained from target detection includes type, location, and accuracy information.

[0199] Furthermore, to verify the effectiveness of the method proposed in this invention, the following specific experiments are also provided in this embodiment:

[0200] The datasets used in the experiment were the public datasets VisDrone2019 and HIT_UAV.

[0201] The VisDrone2019 dataset is divided into a training set of 6471 images, a validation set of 548 images, and a test set of 1610 images, covering 10 object categories: pedestrians, people, cars, vans, buses, trucks, bicycles, sunshade tricycles, and tricycles.

[0202] The HIT_UAV dataset is an upper-air infrared thermal dataset containing 2,898 infrared thermal images extracted from 43,470 frames of hundreds of videos captured by drones in various scenarios such as schools, parking lots, roads, and playgrounds.

[0203] The experimental environment is configured as follows:

[0204] Operating system: Linux.

[0205] Graphics cards: NVIDIA RTX A4000×2.

[0206] Processor: Intel(R) Xeon(R) W_2102.

[0207] Video memory: 16G.

[0208] Programming environment: 3.10.14.

[0209] Deep learning frameworks: PyTorch 2.3.1, CUDA 12.4.

[0210] Input: Drone image.

[0211] Output: A drone image containing location and type information after target detection by the model.

[0212] The following are the specific steps of the experiment:

[0213] The images captured by the drone were cropped to a size of 640×640 and then normalized.

[0214] The C2f_MFPA module, which incorporates the frequency-domain enhanced parallel attention mechanism FAPPA module into the backbone network of the improved RT_DETR model, enhances high-frequency detail information while maintaining multi-scale features, thereby effectively alleviating the information loss problem of small target features in deep networks.

[0215] In the encoder section, after replacing the feedforward neural network in the AIFI module with the EDFFN module, it is possible to selectively enhance high-frequency features such as edges and textures, thereby reducing computational complexity and improving the feature representation capability of small targets.

[0216] Meanwhile, the encoder also uses the Zoom_cat module to achieve adaptive alignment of multi-scale features and the CAFMFusion module's convolutional-attention dual-path architecture to fuse local details and global semantics, so as to make full use of complementary information at different levels.

[0217] Then, dynamic candidate boxes are generated based on feature maps and input into the decoder for multi-level iterative refinement. The target category and bounding box are output using the detection head. Hungarian matching and multi-task loss can also be combined to achieve a one-to-one correspondence between the predicted box and the ground truth box, thereby completing the target detection.

[0218] The image output by the target detection method of this invention contains detection results including confidence level, bounding box, and category.

[0219] The results of the experiment visualized on the VisDrone2019 dataset are as follows: Figures 10 to 13 As shown, it demonstrates the target detection results obtained by using a trained improved RT_DETR model to perform target detection on images captured by a drone. Here, pedestrian represents a pedestrian, people represents a person, car represents a car, van represents a van, truck represents a truck, bicycle represents a bicycle, awning-tricycle represents a chauffeur-driven tricycle, and tricycle represents a tricycle.

[0220] Table 1. Comparison of the method of this invention with other object detection algorithms on the VisDrone2019 dataset.

[0221]

[0222] Table 1 shows the comparative experimental results of the method of the present invention and other target detection algorithms based on the VisDrone2019 dataset. The comparative experiments with various existing target detection algorithms show that the UAV image target detection method based on the improved RT_DETR model proposed in this invention performs excellently in terms of accuracy, precision, and mean precision, which fully verifies the practicality and reliability of the method of the present invention.

[0223] Table 2. Comparative experimental results of the method of this invention with other object detection algorithms on the HIT_UAV dataset.

[0224]

[0225] Table 2 shows the comparative experimental results of the method of the present invention and other target detection algorithms based on the HIT_UAV dataset. The comparative experiments with various existing target detection algorithms show that the UAV image target detection method based on the improved RT_DETR model proposed in this invention performs well in terms of accuracy, precision, and mean precision, further verifying the generalization and robustness of the improved RT_DETR model proposed in this invention.

[0226] This invention presents a UAV image target detection method based on an improved RT_DETR model, which aims to solve the problem of small object detection in UAV images. This method has high accuracy, strong generalization ability and strong robustness, and can effectively cope with a variety of complex environments, thereby greatly enhancing the practical value and application potential of using UAV aerial images for target detection.

[0227] This invention proposes a UAV image target detection method based on an improved RT_DETR model. This method introduces a C2f_MFPA module into the backbone network, and through its FAPPA module, it introduces a frequency-domain enhanced parallel attention mechanism to strengthen fine-grained feature extraction, thereby effectively reducing computational complexity. Furthermore, in the encoder, this invention replaces the original feedforward neural network in the AIFI module with an EDFFN module. The EDFFN module retains high-frequency details of small targets through a selective frequency enhancement mechanism, while combining depthwise separable convolutions to maintain the continuity of local spatial structure. Additionally, this invention introduces a multi-scale alignment Zoom_cat module and a convolutional-attention fusion CAFMFusion module into the encoder to achieve more comprehensive cross-level feature integration. This not only expands the range of feature fusion levels but also improves fusion quality through an adaptive feature selection mechanism. This invention generates dynamic candidate boxes based on feature maps, inputs them into the decoder for multi-level iterative refinement, and then uses the detection head to output the target category and bounding box. Combined with Hungarian matching and multi-task loss, a one-to-one correspondence between the predicted and ground truth boxes is achieved, thus completing target detection.

[0228] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A UAV image target detection method based on an improved RT_DETR model, characterized in that, The method comprises the following steps: Step 1. Preprocessing the image collected by the unmanned aerial vehicle, and constructing a data set for unmanned aerial vehicle image target detection; Step 2. Building an improved RT_DETR model, which comprises a backbone network, an encoder, a decoder and a detection head; The improved RT_DETR model is obtained by improving the original RT_DETR model as follows: introducing a C2f_MFPA module in the backbone network, the C2f_MFPA module being obtained by replacing the Bottlenect module in the C2f module with a frequency domain enhanced parallel attention mechanism FAPPA module; replacing the AIFI module in the encoder with an AIFI_EDFFN module, the AIFI_EDFFN module being obtained by replacing the feedforward neural network in the AIFI module with an EDFFN module; introducing a multi-scale alignment Zoom_cat module and a convolutional attention fusion CAFMFusion module in the encoder; Step 3. Training the improved RT_DETR model using the data set constructed in step 1 to obtain a trained model; Step 4. Using the trained model obtained in step 3 to detect the target in the image collected by the unmanned aerial vehicle.

2. The improved RT-DETR model-based UAV image target detection method according to claim 1, characterized in that, The step 1 is specifically: initializing the image collected by the unmanned aerial vehicle, uniformly scaling the image and the labeled frame to a preset size, and performing normalization and standardization processing to construct a data set for unmanned aerial vehicle image target detection, the data set being a COCO format data set; Feature maps in a dataset for unmanned aerial vehicle image target detection as an input to an improved RT_DETR model, wherein denotes a number of channels, denotes a height, denotes a width.

3. The UAV image target detection method based on the improved RT_DETR model according to claim 2, characterized in that, The backbone network comprises five convolutional layers with a convolution kernel size of 3x3 and a step size of 2, and four C2f_MFPA modules; The four C2f_MFPA modules are defined as a first C2f_MFPA module, a second C2f_MFPA module, a third C2f_MFPA module and a fourth C2f_MFPA module respectively; The processing flow of the signal in the backbone network is as follows: The input of the backbone network is a feature map in a data set for unmanned aerial vehicle image target detection ; feature map After processing through two convolutional layers with a kernel size of 3x3 and a step size of 2, the image is sent to the first C2f_MFPA module to extract semantic features, and a feature map output by the first C2f_MFPA module is obtained ; The output of the first C2f_MFPA module After processing through a convolutional layer with a kernel size of 3x3 and a stride of 2, the feature map is sent to the second C2f_MFPA module to extract semantic features, and the feature map output by the second C2f_MFPA module is obtained ; The output of the second C2f_MFPA module After processing through a convolutional layer with a kernel size of 3*3 and a step size of 2, the image is sent to a third C2f_MFPA module to extract semantic features, and a feature map output by the third C2f_MFPA module is obtained ; The output of the third C2f_MFPA module After processing through a convolutional layer with a kernel size of 3x3 and a stride of 2, the feature map is sent to the fourth C2f_MFPA module to extract semantic features, and the feature map output by the fourth C2f_MFPA module is obtained .

4. The improved RT-DETR model-based UAV image target detection method according to claim 2, characterized in that, The C2f_MFPA module comprises two convolutional layers with a convolution kernel size of 1x1, a channel segmentation module, n FAPPA modules and a feature fusion module; The processing flow of the signal in the C2f_MFPA module is as follows: The input of the C2f_MFPA module is the feature map ; Feature map Channel expansion is performed through a convolution layer with a kernel size of 1x1, and the number of channels is increased to , to obtain an expanded feature map ; The channel splitting module splits the expanded feature map along the channel dimension into and two parts; The feature map output by the channel segmentation module The feature map output by the first FAPPA module is to ; The feature fusion module splices the feature maps output by the channel segmentation module , with the feature maps output by the first FAPPA module to the feature maps output by the nth FAPPA module After splicing in the channel dimension, the feature maps are sent into a convolution layer with a kernel size of 1x1 for processing, and the feature maps output by the C2f_MFPA module are obtained .

5. The improved RT-DETR model-based UAV image target detection method according to claim 2, characterized in that, The FAPPA module comprises a convolutional layer with a convolution kernel size of 1x1, a local perception branch, a global context branch, a serial convolution branch, a frequency domain enhancement branch, an ECA module and a SA module; The input of the FAPPA module is a feature map The output of the FAPPA module is a feature map ; The processing flow of the signal in the FAPPA module is as follows: feature maps input to the FAPPA module a convolutional layer with a kernel size of 1x1 ; feature maps as inputs to the local perception branch, the global context branch, the serial convolution branch, the frequency domain enhancement branch; the output of the local perception branch is a feature map the output of the global context branch is a feature map ; feature map In the serial convolution branch, the feature map is obtained by processing through a convolution layer with a convolution kernel size of 3*3 , the feature map is obtained by processing through a convolution layer with a convolution kernel size of 3*3 , the feature map is obtained by processing through a convolution layer with a convolution kernel size of 3*3 ; The frequency domain enhancement branch comprises a fast Fourier transform (FFT) module, an inverse fast Fourier transform (IFFT) module and a frequency domain gating (FGM) module; the frequency domain enhancement branch utilizes Fourier transform to map the input feature map to a frequency domain space to obtain a feature map output by the frequency domain enhancement branch is​ ； wherein denotes a frequency domain transform, the frequency domain transform employing a fast Fourier transform; denotes a frequency domain inverse transform, the frequency domain inverse transform employing an inverse fast Fourier transform, is a frequency domain gating mechanism; by adding the output of a local perception branch , the output of a global context branch , the output of a serial convolution branch , , and the output of a frequency domain enhancement branch together, a feature map is obtained. ； Feature map After being processed by the channel attention mechanism ECA module and the spatial attention mechanism SA module in sequence, the feature map output by the FAPPA module is obtained .

6. The improved RT-DETR model-based UAV image target detection method according to claim 5, characterized in that, The processing flow of the signal in the local perception branch and the global context branch is as follows: According to Non-overlapping windows will feature maps Cut into N patches and flatten each patch into a vector: ； wherein, represents the operation of flattening the i-th Patch into a vector, represents the vector obtained after flattening the i-th Patch, whose length is , , ; in the local perception branch = 2, in the global context branch = 4; For each vector Taking the mean over the channel dimension gives the vector is: = Mean( ) ; wherein, denotes an operation of taking the mean over the channel dimension; denotes an operation of taking the mean over the channel dimension; the vector obtained after taking the mean over the channel dimension, which has a length of ; The vector is projected into a d-dimensional feature space by a multilayer perceptron to obtain a low-dimensional representation vector is​ ； wherein, denotes a first multi-layer perceptron, denotes a normalization operation, denotes a second multi-layer perceptron; Each dimension of the low-dimensional representation vector is element-wise multiplied with its own attention weight to obtain the attention-weighted feature representation ​ ； wherein, ; denotes a softmax activation function; denotes element-wise multiplication of matrices; Let the Learned global vector Prompt be denoted as , the cosine similarity between and is calculated as: ​ ； wherein , ; Using the clamp function The value of the cosine similarity is limited to the interval [0, 1] to obtain the gating coefficient is: ； By gating coefficients Adjusting local variables Performing global screening, using a matrix Performing inter-channel linear transformation, resulting in : ； wherein is a global feature representation; Reshape with reshape function in original spatial position Reshape with reshape function in original spatial position Reshape with reshape function in original spatial position ; By a convolution layer with a kernel size of 1x1 The mapping is back to the original number of channels C, resulting in The mapping is back to the original number of channels C, resulting in is: ； When = 2, i.e. the output of the local perception branch ; When = 4, is the output of the global context branch .

7. The improved RT-DETR model based unmanned aerial vehicle image target detection method according to claim 3, characterized in that, The encoder comprises six convolutional layers with a convolution kernel size of 1x1, an AIFI_EDFFN module, two Zoom_cat modules, two up-sampling modules, four re-parameterized C3 modules, two convolutional layers with a convolution kernel size of 3x3, and three CAFMFusion modules; The six convolutional layers with a convolution kernel size of 1x1 are defined as a first 1x1 convolutional layer, a second 1x1 convolutional layer, a third 1x1 convolutional layer, a fourth 1x1 convolutional layer, a fifth 1x1 convolutional layer and a sixth 1x1 convolutional layer respectively. The two Zoom_cat modules are defined as a first Zoom_cat module and a second Zoom_cat module respectively; The two up-sampling modules are defined as a first up-sampling module and a second up-sampling module respectively; The four reparameterization C3 modules are defined as a first reparameterization C3 module, a second reparameterization C3 module, a third reparameterization C3 module and a fourth reparameterization C3 module respectively; The two convolution layers with a convolution kernel size of 3*3 are defined as a first 3*3 convolution layer and a second 3*3 convolution layer respectively; The three CAFMFusion modules are defined as a first CAFMFusion module, a second CAFMFusion module and a third CAFMFusion module respectively; The processing procedure of the signal in the encoder is as follows: The input of the encoder is the feature maps output by the first C2f_MFPA module , the feature maps output by the second C2f_MFPA module , the feature maps output by the third C2f_MFPA module , the feature maps output by the fourth C2f_MFPA module ; The input of the first 1x1 convolutional layer is the feature map output by the first C2f_MFPA module ; The input of the second 1x1 convolutional layer is the feature map output by the second C2f_MFPA module ; The input of the third 1x1 convolutional layer is the feature map output by the third C2f_MFPA module ; The input of the fourth 1x1 convolutional layer is the feature map output by the fourth C2f_MFPA module ; The output of the fourth 1x1 convolutional layer is sequentially processed by an AIFI EDFFN module, a fifth 1x1 convolutional layer, and a first upsampling module, and then is sent into a first Zoom_cat module for processing; the input of the first Zoom_cat module further includes feature maps output by a second 1x1 convolutional layer and a third C2f MFPA module ; The output of the first Zoom_cat module is sequentially processed by a first re-parameterization C3 module, a sixth 1x1 convolutional layer, and a second up-sampling module, and then is sent into a second Zoom_cat module for processing; the input of the second Zoom_cat module further includes feature maps output by the first 1x1 convolutional layer and the second C2f_MFPA module ; The output of the second Zoom_cat module is sequentially processed by the second reparameterization C3 module, the first 3*3 convolution layer, spliced with the output of the sixth 1*1 convolution layer, and then sent to the third reparameterization C3 module; The output of the third reparameterization C3 module is processed by the second 3*3 convolution layer, spliced with the output of the fifth 1*1 convolution layer, and then sent to the fourth reparameterization C3 module; The input of the first CAFMFusion module is the output of the third reparameterization C3 module and the third 1*1 convolution layer; The input of the second CAFMFusion module is the output of the second reparameterization C3 module and the second 1*1 convolution layer; The input of the third CAFMFusion module is the output of the fourth reparameterization C3 module and the fourth 1*1 convolution layer; The outputs of the first CAFMFusion module, the second CAFMFusion module and the third CAFMFusion module are taken as the output of the encoder.

8. The improved RT-DETR model based unmanned aerial vehicle image target detection method according to claim 2, characterized in that, The processing procedure of the signal in the EDFFN module is as follows: The input of the EDFFN module is the feature map ; The feature map inputted into the EDFFN module After channel expansion through a convolution layer with a kernel size of 1x1, and then grouped convolution processing by using deep separable convolution, the intermediate feature is obtained ; The intermediate features are divided into two parts along the channel dimension The two parts are processed by a GEGLU activation function The two parts are processed by a GEGLU activation function The two parts are processed by a GEGLU activation function The two parts are processed by a GEGLU activation function The two parts are processed by a GEGLU activation function The two parts are processed by a GEGLU activation function The two parts are processed by a GEGLU activation function ； Channel enhancement results Applying a frequency domain transform within a predetermined size , obtaining spectral features : ； In the frequency domain with learnable frequency domain weights element-wise multiplication, resulting in modulated frequency spectrum features : = ； by inverse frequency domain transform mapping back to the space-time domain, resulting in the output of the EDFFN module mapping back to the space-time domain, resulting in the output of the EDFFN module : ； wherein the frequency domain transform using an inverse fast Fourier transform using an inverse fast Fourier transform.

9. The improved RT-DETR model based unmanned aerial vehicle image target detection method according to claim 7, characterized in that, The Zoom_cat module adopts a multi-branch structure, which includes a first branch, a second branch and a third branch; The processing procedure of the signal in the Zoom_cat module is as follows: The inputs to the Zoom_cat module include large scale features , medium scale features , and small scale features ; For the input of the first Zoom_cat module, large-scale features For the output of the second 1 x 1 convolutional layer, mid-scale features For the feature map output of the third C2f_MFPA module , small-scale features For the output of the first up-sampling module; the output of the first Zoom_cat module is ; For the input of the second Zoom_cat module, large-scale features For the output of the first 1 x 1 convolutional layer, meso-scale features For the feature map output of the second C2f_MFPA module , small-scale features For the output of the second up-sampling module; the output of the second Zoom_cat module is ; The input of the first branch is the large-scale feature The maximum pooling processing and the average pooling processing are respectively performed on the large-scale feature in the first branch , and the output results of the large-scale feature after the maximum pooling processing and the average pooling processing are added to obtain the output of the first branch ； The second branch is not processed, and both its input and output are mesoscale features ; The input of the third branch is the small scale feature The nearest neighbor interpolation operation is performed on the small scale feature in the third branch to obtain the output of the third branch : ； The output of the first branch , the output of the second branch , the output of the third branch are concatenated in the channel dimension to obtain the output of the Zoom_cat module.

10. The improved RT-DETR model based unmanned aerial vehicle image target detection method according to claim 7, characterized in that, The processing procedure of the signal in the CAFMFusion module is as follows: The input to the CAFMFusion module is shallow features and deep features ; For the input of the first CAFM Fusion module, shallow features For the output of the third 1x1 convolutional layer, deep features For the output of the third reparameterization C3 module; For the input of the second CAFM Fusion module, shallow features For the output of the second 1x1 convolutional layer, deep features For the output of the second reparameterization C3 module; For the input of the third CAFM Fusion module, shallow features For the output of the fourth 1x1 convolutional layer, deep features For the output of the fourth reparameterization C3 module; The CAFM module adds the shallow features and the deep features to obtain initial fused features as the input features: ； CAFM module Output characteristics of : ； Input features of the CAFM module With output features After processing by the pixel attention mechanism and the sigmoid function, the weight : ； wherein, represents a pixel attention mechanism, is a sigmoid function; The initial fusion features , shallow features and deep features are weightedly fused to obtain fused features : ； Fused features by a convolution layer with a kernel size of 1x1 to obtain the output features of the CAFMFusion module : ； The processing procedure of the signal in the CAFM module is as follows: The input features of the CAFM module are ; Input features of a cafm module After processing through a convolution layer with a kernel size of 1x1, performing a channel shuffle operation, and then processing through a 3D convolution operation with a kernel size of 3x3x3, a local convolution enhanced feature is obtained : ； wherein, is a channel shuffle operation, is a 3D convolution operation with a kernel size of 3x3x3; Input features of the CAFM module The input features are processed through a convolutional layer with a kernel size of 1x1, and then through a depthwise separable convolution with a kernel size of 3x3, respectively, to obtain the input features The query, key, and value feature representations , , ; Based on the self-attention mechanism to input features Global spatial feature enhancement is performed, and then a convolution layer with a kernel size of 1x1 is used for processing, and then the global feature is obtained by adding the features input into the CAFM module :​ ； wherein denotes a scaling factor; adding the local convolution enhanced features with global features to obtain output features of the CAFM module .

Images