Remote sensing image real-time detection method based on transform algorithm

Abstract

The invention relates to a remote sensing image real-time detection method based on a transform algorithm, and belongs to the technical field of computer vision. The method comprises the steps of obtaining a remote sensing image and performing data enhancement preprocessing on the remote sensing image; extracting key information from the input backbone feature extraction network after data enhancement, and generating a feature map; inputting the extracted feature maps into a constructed multi-scale spatial feature fusion network for feature fusion to obtain a plurality of comprehensive feature maps; decoupling the plurality of comprehensive feature maps through a sensing query module, and selecting a plurality of features to be input into a detection head module to predict the position and category of a potential target; and training the model according to the steps, and detecting the remote sensing image in real time after the training is completed. Through the technical means of data enhancement, efficient feature extraction and coding, multi-scale feature fusion, perceptual query, efficient detection and the like, the remote sensing image detection speed is improved, and the method has high practicability and application value.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and relates to a real-time detection method for remote sensing images based on a transformer algorithm. Background Art

[0002] As a key task in the field of computer vision, object detection is of great significance for identifying and locating specific objects from images. In the field of remote sensing images, the application of object detection technology is particularly extensive, which enables image data acquired from high altitude or space to be effectively analyzed and interpreted. Object detection in remote sensing images is crucial for many fields such as environmental monitoring, urban planning, agricultural management, and disaster assessment. For example, by detecting features such as buildings, roads, and water bodies in remote sensing images, decision support can be provided for urban planning. In the agricultural field, by identifying crop diseases and pests, pest control can be carried out in a timely manner. In addition, the real-time detection capability of remote sensing images is of immeasurable value for rapid response to natural disasters and environmental changes.

[0003] Although target detection technology has great potential in remote sensing image analysis, its application still faces a series of challenges. These include: remote sensing images usually have high resolution and complex backgrounds, which requires target detection algorithms to process large amounts of data and distinguish subtle feature differences. Targets in remote sensing images often have scale diversity, ranging from small-scale individual objects to large-scale geographic structures, which places higher demands on the scale adaptability of detection algorithms. Since the acquisition conditions of remote sensing images may vary greatly, including the influence of factors such as lighting, weather, and seasons, the appearance and contrast of targets in the image vary significantly, increasing the difficulty of detection. Existing transformer-based target detection algorithms often have difficulty meeting the requirements of remote sensing image processing in terms of real-time performance, which is a problem that technicians in this field urgently need to solve. Summary of the invention

[0004] In view of this, the object of the present invention is to provide a real-time detection method for remote sensing images based on transformer algorithm.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A remote sensing image real-time detection method based on transformer algorithm comprises the following steps:

[0007] S1, acquiring remote sensing images and performing data enhancement preprocessing on the remote sensing images;

[0008] S2, extract key information from the data-enhanced input backbone feature extraction network and generate a feature map;

[0009] S3, inputting the extracted feature map into the constructed multi-scale spatial feature fusion network for feature fusion to obtain several comprehensive feature maps;

[0010] S4, decoupling several comprehensive feature maps through the perception query module, and selecting several features to input into the detection head module to predict the location and category of potential targets;

[0011] S5. Train the model according to the above steps, and use it for real-time detection of remote sensing images after training.

[0012] Furthermore, in step S1, a public remote sensing image dataset is obtained, and data enhancement preprocessing is performed on the dataset, wherein the data enhancement preprocessing adopts Mosaic image data enhancement technology, which randomly selects several images for scaling, cropping, and reorganization and splicing to expand and optimize the dataset.

[0013] Further, in the backbone extraction network of step S2, a multi-scale feature map is firstly obtained through a multi-scale feature sampling layer, and the extracted feature map includes at least a feature map Feature Map Feature Map And through the AIFI module, the feature map The encoding process is:

[0014] First, the feature map The shape required to convert to AIFI module:

[0015]

[0016] in, The dimensions are defined as above. Represent query tensor, key tensor and tensor respectively, L represents the sequence length, and d represents the sequence dimension; It means merging the second and third dimensions of the feature map, and then swapping the positions of the first and second dimensions;

[0017] Then the converted feature map Input to AIFI module, such as Figure 4 The processing diagram of the AIFI module shown in the figure is For encoding, AIFI is a classic transformer encoder operation, expressed as:

[0018]

[0019] Where: AIFI(*) operation is expressed as:

[0020] LN(Dropout(FFN(LN(V+Dropout(MHead(Q,K,V)))))+(LV(V+Dropout(MHead(Q,K,V)))))))

[0021] LN(*) is to normalize the elements on the number of channels; Dropout(*) is a regularization technique; FFN(*) is a feedforward neural network, which is defined as: FFN(x) = max(0,w1·x+b1)·w2+b2, w1 and w2 are weight parameters, b1 and b2 are bias parameters, and max(*) is the maximum value function; MHead(*) is the concatenation of multiple attention mechanisms, and each attention mechanism is defined as:

[0022]

[0023] Among them, d k is the feature dimension, (*) T represents matrix transpose; softmax(*) represents a multi-classification activation function;

[0024] Last pair Perform data shape transformation Reshape operation to transform the feature map Adjust to a four-dimensional tensor to get the feature map

[0025] Further, in step S2, the multi-scale feature sampling layer includes multiple convolution modules and at least one activation function, wherein the convolution module performs pixel-by-pixel analysis on each channel of the input feature map and extracts corresponding features. Each convolution module unit is composed of three parts: basic convolution operation, normalization processing and activation function, which can be expressed as:

[0026] x out =ReLU(BN(Conv(x input )))

[0027] Among them, x input is the input four-dimensional tensor, x input ∈R B×C×H×W , B represents the batch size, C represents the number of channels, H×W represents the scale of the feature map, x out represents the output feature map, x out ∈R B×C×H×W , Conv(*) is the basic convolution operation, BN(*) is the normalization operation, and ReLu(*) is the activation function, which are defined as follows:

[0028]

[0029] Where x is the input tensor. When the input tensor passes through the ReLU(*) function, the elements in the tensor that are less than 0 are adjusted to 0, and the remaining elements remain unchanged.

[0030] Further, in step S3, a multi-scale spatial feature fusion SFMM module is used to fusion the feature map And the recoded feature map Feature fusion is performed to obtain a comprehensive feature map, which includes:

[0031] S31, yes The two feature maps are cross-fused to obtain The first output feature map has the same shape as A second output feature map of the same shape;

[0032] S32, the first output feature map, the second output feature map and The feature maps are cross-fused to obtain Feature Map Feature Map The third output feature map, the fourth output feature map, and the fifth output feature map have the same shape.

[0033] Further, in step S31, the following steps are specifically included:

[0034] S311, yes The feature map is downsampled to obtain Feature maps with the same shape;

[0035] S312, yes The feature map is upsampled to obtain Feature maps with the same shape;

[0036] S313, will Feature maps and feature maps with the same feature map shape Perform feature fusion to obtain the first output feature map; Feature maps and feature maps with the same feature map shape Feature fusion is performed to obtain the second output feature map.

[0037] Further, in step S32, the following steps are specifically included:

[0038] S321, up-sampling and down-sampling the first output feature map respectively to obtain Feature maps with the same shape;

[0039] S322, up-sample the second output feature map to obtain features that are respectively identical to the first output feature map and the feature map Feature maps of the same shape;

[0040] S323, feature map Downsampling is performed to obtain feature maps that are respectively the same as the first output feature map and the first output feature map;

[0041] S323, down-sampling the first output feature map and up-sampling the second output feature map to obtain the feature map Feature maps and feature maps of the same shape Perform feature fusion to obtain a third output feature map;

[0042] The first output feature map and the second feature map are upsampled, and the feature map Down-sampling and feature map The feature maps with the same shape are subjected to feature fusion to obtain a fourth feature map;

[0043] Upsample the first output feature map, feature map The feature map obtained by downsampling and having the same shape as the second output feature map is feature fused with the second output feature map to obtain the fifth output feature map; the third output feature map, the fourth output feature map and the fifth output feature map are comprehensive feature maps.

[0044] Further, in step S4, it specifically includes:

[0045] S41, inputting the third output feature map, the fourth output feature map and the fifth output feature map into the perception query module, the perception query module converts the obtained feature map into a token form, and uses the IoU constraint to constrain the initial object query in the decoding process, thereby completing the decoupling;

[0046] S42, select K features with the highest classification scores and output them to the detection head module of the next layer;

[0047] S43. The detection head includes a classification head and a regression head, wherein the classification head predicts a category label for each input target query, and the regression head predicts the position of the target.

[0048] The beneficial effects of the present invention are:

[0049] The present invention extracts a multi-scale feature map through a multi-scale feature sampling layer and encodes the feature map using an AIFI module. The AIFI module uses a transformer encoder operation to efficiently capture long-distance dependencies and improve feature representation capabilities. At the same time, through data shape transformation, the feature map is adjusted to a four-dimensional tensor suitable for subsequent processing, thereby improving processing efficiency.

[0050] The present invention adopts a multi-scale spatial feature fusion SFMM module to perform feature fusion on feature maps. By cross-fusion of feature maps of different scales, the information of features of each scale is fully utilized, the model's detection ability for targets of different sizes in remote sensing images is enhanced, and the detection accuracy is improved.

[0051] The present invention decouples the comprehensive feature map through the perception query module and selects the K features with the highest classification scores to output to the detection head module. The perception query module can accurately locate potential targets, and the IoU constraint further improves the accuracy of target positioning. The detection head module includes a classification head and a regression head, which are used to predict the category label and position of the target respectively, realizing efficient target detection.

[0052] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0054] Figure 1 It is a flow chart of the real-time detection method of remote sensing images based on transformer algorithm of the present invention;

[0055] Figure 2 It is a schematic diagram of the overall framework of the real-time detection method of remote sensing images based on the transformer algorithm of the present invention;

[0056] Figure 3 It is a schematic diagram of the processing process of the multi-scale feature sampling layer of the present invention;

[0057] Figure 4 It is a schematic diagram of the processing process of the AIFI module of the present invention;

[0058] Figure 5 Schematic diagram of the processing process of the multi-scale spatial feature fusion module SFMM of the present invention;

[0059] Figure 6 It is a schematic diagram of the fusion process of the two branch input feature maps of the multi-scale spatial feature fusion module SFMM of the present invention;

[0060] Figure 7 Schematic diagram of the fusion process of the three branch input feature maps of the multi-scale spatial feature fusion module SFMM of the present invention. DETAILED DESCRIPTION

[0061] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0062] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0063] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0064] See also Figure 1 to Figure 7 , which is a real-time detection method for remote sensing images based on transformer algorithm.

[0065] Example

[0066] This embodiment discloses a real-time detection method for remote sensing images based on the transformer algorithm, which provides a stable and efficient solution for the transformer-based remote sensing image target detection algorithm and provides strong support for future remote sensing image application fields. Figure 1 The flowchart of the real-time detection method of remote sensing images based on the transformer algorithm is shown in FIG. Figure 2 The schematic diagram of the framework structure of the remote sensing image real-time detection method based on the transformer algorithm is shown, which includes the following steps:

[0067] S1, acquiring remote sensing images and performing data enhancement preprocessing on the remote sensing images;

[0068] S2, extract key information from the data-enhanced input backbone feature extraction network and generate a feature map;

[0069] S3, inputting the extracted feature map into the constructed multi-scale spatial feature fusion network for feature fusion to obtain several comprehensive feature maps;

[0070] S4. Decouple several comprehensive feature maps through the perception query module, and select several features to input into the detection head module to predict the location and category of potential targets.

[0071] In step S1 of this embodiment, a public dataset Dior is obtained, which contains 23,463 optical remote sensing images and 190,288 target instances. These images are obtained under different imaging conditions, weather, seasons and image quality. The collected remote sensing images are preprocessed by Mosaic image data enhancement technology. Mosaic image data enhancement technology is essentially to scale, crop and reassemble four images in a random manner. This technology has the following significant advantages:

[0072] Increase data diversity: By randomly selecting four images and applying random scaling, followed by random reassembly and concatenation, this method significantly increases the diversity of the dataset. In particular, the use of random scaling introduces many small-sized objects, thereby improving the network's robustness to objects of different sizes.

[0073] Optimize video memory usage: Since this technology allows four images to be processed simultaneously, efficient training can be achieved without increasing the mini-batch size, which helps reduce the demand for GPU video memory.

[0074] In step S2 of this embodiment, in the backbone extraction network, a multi-scale feature map is firstly obtained through a multi-scale feature sampling layer, and the extracted feature map at least includes a feature map Feature Map Feature Map Among them, the feature map Feature Map Feature Map They are the feature maps extracted from the third, fourth and fifth layers of the backbone network respectively.

[0075] The multi-scale feature sampling layer includes multiple convolution modules and at least one activation function, such as Figure 3 As shown, the convolution module is designed to perform pixel-by-pixel analysis on each channel of the input feature map to complete the feature extraction work, which is expressed as:

[0076] xout =ReLU(BN(Conv(x input )))

[0077] Among them, x input is the input four-dimensional tensor, x input ∈R B×C×H×W , B represents the batch size, C represents the number of channels, H×W represents the scale of the feature map, x out represents the output feature map, x out ∈R B×C×H×W , Conv(*) is the basic convolution operation, BN(*) is the normalization operation, and ReLu(*) is the activation function, which are defined as follows:

[0078]

[0079] Where x is the input tensor. When the input tensor passes through the ReLU(*) function, the elements in the tensor that are less than 0 will be adjusted to 0, and the remaining elements remain unchanged.

[0080] Each convolution module unit consists of three parts: basic convolution operation, normalization processing and activation function. The basic convolution operation is responsible for performing point-by-point calculations on each point in the input feature map in the channel dimension to achieve the purpose of feature extraction; normalization processing ensures that the calculated feature data is adjusted to the standard range of 0-1; the activation function screens the feature matrix of possible targets and increases the expressive power of the model by activating certain features and suppressing other features.

[0081] An AIFI module is set in the backbone feature extraction network, and the feature map obtained by the AIFI module is Encoding helps the model to more effectively identify potential target areas in remote sensing images, thereby improving the accuracy and efficiency of target detection. First, the feature map The shape required to convert to AIFI module:

[0082]

[0083] in: The dimensions are defined as above. Represent query tensor, key tensor and tensor respectively, L represents the sequence length, and d represents the feature dimension; It means merging the second and third dimensions of the feature map, and then swapping the positions of the first and second dimensions;

[0084] Then the converted feature map Input to AIFI module, such as Figure 4 The processing diagram of the AIFI module shown in the figure is For encoding, AIFI is a classic transformer encoder operation, expressed as:

[0085]

[0086] Where: AIFI(*) operation is expressed as:

[0087] LN(Dropout(FFN(LN(V+Dropout(MHead(Q,K,V)))))+(LN(V+Dropout(MHead(Q,K,V))))))

[0088] LN(*) is to normalize the elements on the number of channels; Dropout(*) is a regularization technique; FFN(*) is a feedforward neural network, which is defined as: FFN(x) = max(0,w1·x+b1)·w2+b2, w1 and w2 are weight parameters, b1 and b2 are bias parameters, and max(*) is the maximum value function; MHead(*) is the concatenation of multiple attention mechanisms, and each attention mechanism is defined as:

[0089]

[0090] Among them, d k is the feature dimension, (*) T represents matrix transpose; softmax(*) represents the multi-classification activation function.

[0091] Then Perform data shape transformation Reshape operation to transform the feature map Adjust to a four-dimensional tensor to get the feature map

[0092] In step S3 of this embodiment, the multi-scale feature map output in step S2 is And the recoded feature map Input the multi-scale spatial feature fusion SFMM module for feature fusion, such as Figure 5 , Figure 6 and Figure 7 As shown, the specific process is:

[0093] S31, first The two feature maps are cross-merged and obtained The first output feature map has the same shape as The second output feature map of the same shape, the specific steps are as follows:

[0094] S311, yes The feature map is downsampled to obtain a Feature maps with the same shape;

[0095] S312, yes The feature map is upsampled to obtain a Feature maps with the same shape;

[0096] S313, then perform feature fusion on the feature maps of the same shape to obtain two The feature maps with the same shape are the first output feature map and the second output feature map, that is, Feature maps and feature maps with the same feature map shape Perform feature fusion to obtain the first output feature map; Feature maps and feature maps with the same shape Perform feature fusion to obtain a second output feature map;

[0097] S32, then the first output feature map, the second output feature map and The feature maps are cross-fused and finally obtained Feature Map Feature Map The third output feature map, the fourth output feature map, and the fifth output feature map of the same shape, the specific steps are as follows:

[0098] S321, up-sample and down-sample the first output feature map respectively to obtain two features that are identical to the second output feature map and Feature maps with the same shape;

[0099] S322, up-sample the second output feature map to obtain two feature maps that are identical to the first output feature map and the feature map Feature maps of the same shape;

[0100] S323, yes The feature maps are downsampled respectively to obtain two feature maps that are the same as the first output feature map and the first output feature map;

[0101] S323, perform multi-scale fusion on the feature maps of the same shape to obtain three The third output feature map, the fourth output feature map and the fifth output feature map of the same shape as the first output feature map and the second output feature map, that is, the first output feature map is downsampled and the second output feature map is upsampled to obtain the same shape as the feature map Feature maps and feature maps of the same shape Perform feature fusion to obtain a third output feature map; upsample the first output feature map and the second feature map, and then Down-sampling and feature map The feature maps with the same shape are feature fused to obtain the fourth feature map; the first output feature map is upsampled and the feature map is The feature map obtained by downsampling and having the same shape as the second output feature map is feature fused with the second output feature map to obtain the fifth output feature map; the third output feature map, the fourth output feature map and the fifth output feature map are the aforementioned comprehensive feature maps.

[0102] In step S4 of this embodiment,

[0103] S41, firstly inputting the third output feature map, the fourth output feature map and the fifth output feature map into the perception query module for decoupling, specifically, converting the obtained feature map into a token form, and then providing a higher quality initial object query to the decoder through the IoU constraint;

[0104] S42 then selects K features with higher classification scores and outputs them to the detection head module of the next layer.

[0105] S43, the detection head is divided into a classification head and a regression head. The classification head predicts the category label for each input target query. The regression head is responsible for predicting the location of the target. The output of the decoder passes through the detection head and outputs its possible target category and area.

[0106] S5. The entire model is trained according to the above process, and after the training is completed, it is used for real-time detection of remote sensing images.

[0107] In this embodiment, the performance evaluation indicators precision (Precision P), recall rate (Recall R) and mean average precision (mean Average Precision mAP) are also established, where:

[0108] Precision is defined as the ratio of correctly identified targets to the total number of detected targets, expressed as:

[0109]

[0110] The recall rate is defined as the proportion of samples that are correctly identified among all samples that are actually positive, expressed as:

[0111]

[0112] The average precision is defined as the average value of the accuracy at a series of different recall levels, which is used to measure the prediction performance of the model for a single category, expressed as:

[0113] AP = ∫0 1 P(R)dR

[0114] Among them, TP is the number of samples that are actually positive and correctly predicted as positive. FP is the number of samples that are actually negative but wrongly predicted as positive. FN is the number of samples that are actually positive but wrongly predicted as negative. TN is the number of samples that are actually negative and correctly predicted as negative.

[0115] The mean of the average precision is defined as the average of the AP values ​​of all categories, which can fully reflect the overall performance of the model and is expressed as:

[0116]

[0117] Among them, AP i represents the AP value of the i-th class, N cls Indicates how many categories there are.

[0118] In this embodiment, the performance of the model of the present invention and the existing RT-DETR are compared in combination with the above performance evaluation indicators. The specific performance indicator comparison results are shown in Table 1:

[0119] Table 1

[0120]

[0121] According to Table 1, the improved model of the present invention reduces the computational complexity by about 30G while ensuring the original accuracy (only decreased by 0.43% compared with the original model), and increases the detection rate by nearly 8 frames, which is more conducive to practical applications.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A remote sensing image real-time detection method based on transformer algorithm, characterized in that: It includes the following steps: S1, acquiring remote sensing images and performing data enhancement preprocessing on the remote sensing images; S2, extract key information from the data-enhanced input backbone feature extraction network and generate a feature map; S3, inputting the extracted feature map into the constructed multi-scale spatial feature fusion network for feature fusion to obtain several comprehensive feature maps; S4, decoupling several comprehensive feature maps through the perception query module, and selecting several features to input into the detection head module to predict the location and category of potential targets; S5. Train the model according to the above steps, and use it for real-time detection of remote sensing images after training.

2. The remote sensing image real-time detection method based on transformer algorithm according to claim 1, characterized in that: In step S1, a public remote sensing image dataset is obtained and data enhancement preprocessing is performed on the dataset, wherein the data enhancement preprocessing adopts Mosaic image data enhancement technology, which randomly selects several images for scaling, cropping, and reorganization and splicing to expand and optimize the dataset.

3. The remote sensing image real-time detection method based on transformer algorithm according to claim 1, characterized in that: In the backbone extraction network of step S2, a multi-scale feature map is firstly obtained through a multi-scale feature sampling layer, and the extracted feature map includes at least a feature map Feature Map Feature Map And through the AIFI module, the feature map The encoding process is: First, the feature map The shape required to convert to AIFI module: in, The dimensions are defined as above. Represent query tensor, key tensor and tensor respectively, L represents the sequence length, and d represents the feature dimension; It means merging the second and third dimensions of the feature map, and then swapping the positions of the first and second dimensions; Then the converted feature map Input to the AIFI module, as shown in Figure 4, the processing process of the AIFI module v is encoded, AIFI is the classic transformer encoder operation, expressed as: Where: AIFI(*) operation is expressed as: LN(Dropout(FFN(LN(V+Dropout(MHead(Q,K,V)))))+(LN(V+Dropout(MHead(Q,K,V)))))) LN(*) is the normalization of the elements on the number of channels; Dropout(*) indicates the random death of neurons; FFN(*) indicates a feedforward neural network, which is defined as: FFN(x) = max(0, w1 x+b1) w2+b2, w1 and w2 are weight parameters, b1 and b2 are bias parameters, and max(*) is the maximum value function; MHead(*) is the concatenation of multiple attention mechanisms, and each attention mechanism is defined as: Among them, d k is the feature dimension, (*) T represents matrix transpose; softmax(*) represents multi-classification activation function; Last pair Perform data shape transformation Reshape operation to transform the feature map Adjust to a four-dimensional tensor to get the feature map 4. The real-time detection method of remote sensing images based on transformer algorithm according to claim 3 is characterized in that: In step S2, the multi-scale feature sampling layer includes multiple convolution modules and at least one activation function, wherein the convolution module performs pixel-by-pixel analysis on each channel of the input feature map and extracts corresponding features. Each convolution module unit is composed of three parts: basic convolution operation, normalization processing and activation function, which can be expressed as: x out =ReLU(BN(Conv(x input ))) Among them, x input is the input four-dimensional tensor, x input ∈R B×C×H×W , B represents the batch size, C represents the number of channels, H×W represents the scale of the feature map, x out represents the output feature map, x out ∈R B×C×H×W , Conv(*) is the basic convolution operation, BN(*) is the normalization operation, and ReLu(*) is the activation function, which are defined as follows: Where x is the input tensor. When the input tensor passes through the ReLU(*) function, the elements in the tensor that are less than 0 are adjusted to 0, and the remaining elements remain unchanged.

5. The remote sensing image real-time detection method based on transformer algorithm according to claim 3, characterized in that: In step S3, the multi-scale spatial feature fusion SFMM module is used to fusion the feature map And the recoded feature map Feature fusion is performed to obtain a comprehensive feature map, which includes: S31, yes The two feature maps are cross-fused to obtain The first output feature map has the same shape as A second output feature map of the same shape; S32, the first output feature map, the second output feature map and The feature maps are cross-fused to obtain Feature Map Feature Map The third output feature map, the fourth output feature map, and the fifth output feature map have the same shape.

6. The remote sensing image real-time detection method based on transformer algorithm according to claim 5, characterized in that: In step S31, the following steps are specifically included: S311, yes The feature map is downsampled to obtain Feature maps with the same shape; S312, yes The feature map is upsampled to obtain Feature maps with the same shape; S313, will Feature maps and feature maps with the same shape Perform feature fusion to obtain the first output feature map; Feature maps and feature maps with the same shape Feature fusion is performed to obtain the second output feature map.

7. The remote sensing image real-time detection method based on transformer algorithm according to claim 5, characterized in that: In step S32, the following steps are specifically included: S321, up-sampling and down-sampling the first output feature map respectively to obtain Feature maps with the same shape; S322, up-sample the second output feature map to obtain features that are respectively identical to the first output feature map and the feature map Feature maps of the same shape; S323, feature map Downsampling is performed to obtain feature maps that are respectively the same as the first output feature map and the first output feature map; S323, down-sampling the first output feature map and up-sampling the second output feature map to obtain the feature map Feature maps and feature maps of the same shape Perform feature fusion to obtain a third output feature map; The first output feature map and the second feature map are upsampled, and the feature map Down-sampling and feature map The feature maps with the same shape are subjected to feature fusion to obtain a fourth feature map; Upsample the first output feature map, feature map The feature map obtained by downsampling and having the same shape as the second output feature map is feature fused with the second output feature map to obtain the fifth output feature map; the third output feature map, the fourth output feature map and the fifth output feature map are comprehensive feature maps.

8. The remote sensing image real-time detection method based on transformer algorithm according to claim 5, characterized in that: In step S4, it specifically includes: S41, inputting the third output feature map, the fourth output feature map and the fifth output feature map into the perception query module, the perception query module converts the obtained feature map into a token form, and uses the IoU constraint to constrain the initial object query in the decoding process, thereby completing the decoupling; S42, select K features with the highest classification scores and output them to the detection head module of the next layer; S43. The detection head includes a classification head and a regression head, wherein the classification head predicts a category label for each input target query, and the regression head predicts the position of the target.

Images