Remote sensing image target detection method based on improved FCOS

Abstract

The invention provides a remote sensing image target detection method based on an improved FCOS, and relates to the technical field of target detection, and the method comprises the following steps: carrying out the convolution and pooling operation of a remote sensing target image based on a pre-trained convolutional neural network, generating a preliminary feature, and carrying out the enhancement processing of the preliminary feature through a twin multilayer perceptron; carrying out fusion processing on different levels of feature information in the multi-level feature map through a feature map splicing mode and a feature map enhancement mode; and determining the position of the remote sensing image target according to the analysis result to obtain a target detection result. According to the method, the feature pyramid network is combined with an innovative coordinate attention mechanism, fusion of high-level semantic information and low-level detail information is realized, an enhanced multi-scale feature map is generated to cope with diversity of target scales in a remote sensing image, multi-scale fusion and an anchor-frame-free design are combined, false detection is effectively reduced, and the detection accuracy is improved. Therefore, the method shows efficient, accurate and wide application potential in the field of remote sensing image target detection.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a remote sensing image target detection method based on an improved FCOS. Background Art

[0002] Traditional target detection methods, such as Faster R-CNN, SSD, and YOLO series, are mainly based on anchor-based mechanisms, relying on predefined anchor boxes to generate candidate regions and make adjustments to generate prediction results. However, the introduction of anchor boxes brings multiple problems, including sensitivity, difficulty in handling targets with large scale differences, difficulty in detecting small targets, and imbalance between positive and negative samples. In the field of remote sensing images, ground objects have the characteristics of variable sizes, dense distribution, large aspect ratios, and uncertain orientations, which makes traditional anchor-based methods face severe challenges in dealing with these complex scenes.

[0003] In recent years, anchor-free object detection methods have gradually become a research hotspot. They avoid the complex calculations and hyperparameter adjustments brought by anchor boxes, simplify model design, and improve generalization capabilities. FCOS (Fully Convolutional One-Stage Object Detection), as a typical anchor-free object detection method, achieves efficient object detection by directly predicting each position on the feature map. Considering the special properties of remote sensing images, such as high resolution, large scenes, and complex backgrounds, traditional object detection methods often find it difficult to achieve ideal detection results.

[0004] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0005] In view of this, the present invention provides a remote sensing image target detection method with improved FCOS to solve the above-mentioned problems.

[0006] In order to solve the above problems, the specific technical solutions adopted by the present invention are as follows:

[0007] A remote sensing image target detection method based on improved FCOS, the method comprising the following steps:

[0008] S1. Obtain a remote sensing target image, perform convolution and pooling operations on the remote sensing target image based on a pre-trained convolutional neural network to generate preliminary features, and use a twin multi-layer perceptron to enhance the preliminary features to obtain a multi-level feature map;

[0009] S2, based on the feature pyramid network of the fusion coordinate attention mechanism, the feature information of different levels in the multi-level feature map is fused and processed by the feature map splicing method and the feature map enhancement method to obtain the feature output map;

[0010] S3. The detection head of the FCOS algorithm performs target classification, center prediction and bounding box regression analysis on the feature output image, and determines the position of the remote sensing image target based on the analysis results to obtain the target detection result.

[0011] Preferably, the step of acquiring a remote sensing target image, performing convolution and pooling operations on the remote sensing target image based on a pre-trained convolutional neural network to generate preliminary features, and enhancing the preliminary features using a twin multi-layer perceptron to obtain a multi-level feature map comprises the following steps:

[0012] S11, preprocessing the acquired remote sensing target image to obtain a preprocessed remote sensing target image;

[0013] S12, using a pre-trained convolutional neural network to perform convolution and pooling operations on the pre-processed remote sensing target image, and extracting a preliminary feature map from the pre-processed remote sensing target image;

[0014] S13, using the dual-branch structure of the twin network to perform feature matching and similarity discrimination on the extracted preliminary feature map, and obtain a feature vector and a similarity discrimination result;

[0015] S14. According to the feature vector and the similarity judgment result, the feature vector that meets the preset similarity threshold is selected, and the selected feature vector is enhanced through the multi-layer perceptron structure to obtain a multi-level feature map.

[0016] Preferably, the method of using a pre-trained convolutional neural network to perform convolution and pooling operations on the pre-processed remote sensing target image, and extracting a preliminary feature map from the pre-processed remote sensing target image comprises the following steps:

[0017] S121, performing a dot product operation on the preprocessed remote sensing target image through a convolution kernel in a convolutional neural network to obtain a convolved feature map;

[0018] S122, using a normalization layer to normalize the convolutional features, and activating the normalized features through a nonlinear activation function to obtain an activated feature map;

[0019] S123, performing a maximum pooling operation on the activated features through a maximum pooling method, extracting and storing a preliminary feature map after pooling.

[0020] Preferably, the step of performing feature matching and similarity discrimination on the extracted preliminary feature graph using the dual-branch structure of the twin network to obtain the feature vector and similarity discrimination result comprises the following steps:

[0021] S131, flattening the extracted preliminary features, and inputting the flattened feature graph into each branch in the twin network;

[0022] S132. In each branch, a pixel-by-pixel correlation operation is performed on the flattened feature map, and an initial correlation map is generated by feature decomposition;

[0023] S133, based on the attention mechanism, the initial correlation graph is optimized using channel attention and spatial attention to obtain the final correlation graph;

[0024] S134. According to the cosine similarity principle, the similarity between the feature vectors in the final correlation graphs of the two branches is calculated to obtain the feature vector and the similarity judgment result.

[0025] Preferably, the principle expression of the pixel-by-pixel correlation operation is:

[0026]

[0027] Where M represents the initial correlation map output by the pixel-by-pixel correlation operation;

[0028] M j Represents the result of the cross-correlation operation between the jth 1×1 convolution kernel and the search area feature map;

[0029] K j Represents the jth 1×1 convolution kernel of the flattened feature map;

[0030] S represents the feature map of the search area;

[0031] * indicates convolution operation;

[0032] H0×W0 represents the size of the feature map after flattening.

[0033] Preferably, the method of optimizing the initial correlation graph using channel attention and spatial attention based on the attention mechanism to obtain the final correlation graph includes the following steps:

[0034] S1331. Calculate the variance of each channel of the initial correlation graph according to the channel attention mechanism;

[0035] S1332, normalizing the variance of each channel of the initial correlation graph so that each channel has zero mean and unit variance, and scaling and translating the normalized feature graph through affine transformation parameters;

[0036] S1333, based on the pre-learned weight parameters and the normalized feature map, a channel attention weight map is generated using a Sigmoid activation function, and the weight map is multiplied by the initial correlation map channel by channel to obtain a channel-optimized correlation map;

[0037] S1334. Use the spatial attention mechanism to perform spatial weighted processing on the channel-optimized correlation graph to obtain the final correlation graph.

[0038] Preferably, the feature pyramid network based on the fusion coordinate attention mechanism fuses the feature information of different levels in the multi-level feature map by means of feature map splicing and feature map enhancement to obtain the feature output map, including the following steps:

[0039] S21, based on the feature pyramid network, a top-down path is constructed according to the multi-level feature map, and the feature maps of different levels are spliced ​​by horizontal connection;

[0040] S22, using the coordinate attention mechanism to enhance the attention of each layer of the spliced ​​feature map, generate an attention map, and perform element-by-element multiplication operation on the generated attention map and the corresponding feature map to obtain an enhanced feature map;

[0041] S23. The feature maps of all levels after enhancement are fused through a pre-configured fusion strategy to obtain a final feature output map.

[0042] Preferably, the expression for constructing a top-down path based on a feature pyramid network and concatenating feature maps of different levels by horizontal connection is:

[0043] FPN(F i )=UpSample(Conv(Concat(F i+1 ,LateralConn(F i ))));

[0044] In the formula, FPN(F i ) represents the concatenated feature map;

[0045] UpSample indicates upsampling operation;

[0046] Conv represents the product operation;

[0047] Concat represents feature concatenation operation;

[0048] LateralConn represents a lateral connection operation;

[0049] F i Represents the feature map of the i-th layer.

[0050] Preferably, the coordinate attention mechanism is used to enhance the attention of each layer of the spliced ​​feature map to generate an attention map, and the generated attention map is element-by-element multiplied with the corresponding feature map to obtain the expression of the enhanced feature map:

[0051]

[0052] In the formula, CA(F) represents the enhanced feature map;

[0053] F represents the concatenated feature map;

[0054] σ represents the activation function;

[0055] AvgPool h Indicates that the average pooling operation is performed along the horizontal direction;

[0056] AvgPool w Indicates that the average pooling operation is performed along the vertical direction;

[0057] Concat means concatenating the average pooling results in the horizontal and vertical directions;

[0058] Represents element-wise multiplication.

[0059] Preferably, the detection head includes a target classification branch, a bounding box regression branch and a center prediction branch;

[0060] A generalized focal loss function is used in the target classification branch;

[0061] The bounding box regression branch is implemented by predicting the distance from each point on the feature output graph to the four sides of the true bounding box;

[0062] The center prediction branch predicts points on the feature output graph and reduces the weight of the target center prediction box by multiplying the predicted value with the target classification score.

[0063] The beneficial effects of the present invention are:

[0064] 1. The present invention uses an efficient convolutional neural network as the backbone of the FCOS target detection framework for feature extraction. The network deeply mines the multi-level and multi-scale features of remote sensing images through multi-layer convolution, normalization and attention mechanism, and constructs a feature pyramid rich in contextual information. Subsequently, the feature pyramid network is combined with an innovative coordinate attention mechanism to achieve the fusion of high-level semantic information and low-level detail information, and generate an enhanced multi-scale feature map to cope with the diversity of target scales in remote sensing images. The GFL loss function is used to solve the imbalance problem of small target samples, unify classification and positioning tasks, and combine multi-scale fusion with anchor-free frame design to effectively reduce false detections, thereby showing efficient, accurate and wide application potential in the field of remote sensing image target detection.

[0065] 2. The present invention uses twin multi-layer perceptrons to enhance preliminary features, and can expand feature space through nonlinear transformations, so that the model can have a deeper understanding of complex image patterns. Multi-level feature map generation can capture a wider range of details and hierarchical information, and provide multi-level support for subsequent classification or detection tasks, thereby improving the recognition and discrimination capabilities of the model. The dual-branch structure of the twin network realizes feature matching and similarity discrimination, and can accurately compare the differences between different feature maps, which is conducive to identifying similar objects in remote sensing images. It further enhances feature differences through pixel-by-pixel correlation operations and attention mechanisms, and uses channel attention and spatial attention to optimize the initial correlation map, so that the model can automatically focus on important channels and key spatial areas, thereby suppressing redundant information. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0067] Figure 1 is a flow chart of a remote sensing image target detection method for improving FCOS according to an embodiment of the present invention;

[0068] Figure 2 It is a flowchart of feature extraction, feature fusion, classification and regression in a remote sensing image target detection method of improving FCOS according to an embodiment of the present invention;

[0069] Figure 3 is a schematic diagram of a feature extraction process in a remote sensing image target detection method with an improved FCOS according to an embodiment of the present invention;

[0070] Figure 4is a schematic diagram of a GA attention mechanism in a feature fusion process of a remote sensing image target detection method with an improved FCOS according to an embodiment of the present invention;

[0071] Figure 5 It is a schematic diagram of a detection head in a remote sensing image target detection method of improving FCOS according to an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0073] According to an embodiment of the present invention, a remote sensing image target detection method with improved FCOS is provided.

[0074] The present invention is further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to the improved FCOS remote sensing image target detection method of the embodiment of the present invention, the method comprises the following steps:

[0075] S1. Obtain a remote sensing target image, perform convolution and pooling operations on the remote sensing target image based on a pre-trained convolutional neural network to generate preliminary features, and use a twin multi-layer perceptron to enhance the preliminary features to obtain a multi-level feature map;

[0076] It should be noted that if Figure 2 As shown in the figure, by selecting the pre-trained ConvNeXt network (convolutional neural network) as the backbone network, the network stands out for its efficient and powerful feature extraction capabilities. Through its innovative architecture design, combined with stacked convolutional layers, normalization layers and possible attention mechanisms, ConvNeXt performs multi-level and multi-scale feature extraction and representation learning on the input remote sensing images. This process not only retains the shallow detail information of the image, but also deeply mines the high-level semantic features and constructs a feature pyramid containing rich contextual information. These characteristics of the ConvNeXt network provide a solid feature foundation for subsequent classification, regression and other tasks of FCOS, and significantly improve the accuracy and efficiency of remote sensing image target detection.

[0077] The ConvNeXt network is an advanced convolutional neural network architecture that combines concepts such as large kernel convolution, layer normalization, MLP (multi-layer perceptron) structure and deep layered design. It achieves efficient processing of input remote sensing images through multi-level and multi-scale feature extraction and representation learning. Its core lies in using large kernel convolution to capture a wider range of contextual information. Although the specific formula of the ConvNeXt network is relatively complex, its core idea can be summarized as gradually extracting and transforming image features through a series of carefully designed convolutional layers and normalization layers, as well as MLP structures, and finally outputting high-level feature representations for classification or subsequent processing. The core formula of the process is as follows:

[0078] Y=ConvLargeKernel(X)→LayerNorm(Y)→MLP(Y);

[0079] In the formula, Y represents the feature representation after processing in each step;

[0080] X represents the input remote sensing target image;

[0081] ConvLargeKernel represents a large kernel convolution operation, which is used to extract image features;

[0082] LayerNorm represents the layer normalization operation, which is used to stabilize the training process;

[0083] MLP stands for Multi-layer Perceptron structure, which is used to further transform and enhance features;

[0084] This process not only retains the shallow detail information of the image, but also deeply explores the high-level semantic features and constructs a feature pyramid containing rich contextual information. These characteristics of the ConvNeXt network provide a solid feature foundation for subsequent classification, regression and other tasks of FCOS, and significantly improve the accuracy and efficiency of remote sensing image target detection. Figure 3 shown, and Figure 3 Here, Down sample means down sampling, GlobalAvg Pooling means global average pooling, Linear means linear transformation, and LN: Layer Normalization means layer normalization.

[0085] The ConvNeXt module effectively expands the receptive field by introducing a large 7×7 convolution kernel in its deep convolution layer, thereby capturing more global information. At the same time, the module uses a 2×2 convolution layer combined with a convolution kernel with a step size of 2 to implement downsampling operations in the spatial dimension. In order to stabilize the training process when the spatial resolution changes, ConvNeXt adds a LayerNorm (LN) normalization layer at the corresponding position. This aims to reduce gradient fluctuations and improve training stability. In addition, ConvNeXt also draws on the essence of the Swin Transformer's architectural design, such as hierarchical structure and reverse bottleneck features, while reducing the use of activation functions and BatchNorm. This effectively prevents the loss of small-scale target feature information while reducing the computational burden, thereby improving the overall performance of the network. Figure 3 The GELU activation function shown in the figure incorporates the idea of ​​random regularization in its design. It processes each input x to make it conform to the characteristics of the standard normal distribution N(0,1), and through the multiplication operation with the Bernoulli distribution, it not only maintains the probabilistic characteristics of the data, but also ensures the continuity of the dependency relationship between inputs. For the input x, the specific form of the GELU activation function can be expressed by the given formula. This mechanism further enhances the network's expressive power and generalization performance. The expression of the GELU activation function is:

[0086] GELU(x)=x*σ(1.702x);

[0087] Where x represents input; σ represents Sigmoid function.

[0088] As a preferred implementation, the step of acquiring a remote sensing target image, performing convolution and pooling operations on the remote sensing target image based on a pre-trained convolutional neural network, generating preliminary features, and enhancing the preliminary features using a twin multi-layer perceptron to obtain a multi-level feature map includes the following steps:

[0089] S11, preprocessing the acquired remote sensing target image to obtain a preprocessed remote sensing target image;

[0090] It should be noted that the preprocessing of the acquired remote sensing target image includes adjusting the image size, normalizing pixel values, data enhancement, etc.

[0091] S12, using a pre-trained convolutional neural network to perform convolution and pooling operations on the pre-processed remote sensing target image, and extracting a preliminary feature map from the pre-processed remote sensing target image;

[0092] As a preferred embodiment, the method of using a pre-trained convolutional neural network to perform convolution and pooling operations on the pre-processed remote sensing target image, and extracting a preliminary feature map from the pre-processed remote sensing target image includes the following steps:

[0093] S121, performing a dot product operation on the preprocessed remote sensing target image through a convolution kernel in a convolutional neural network to obtain a convolved feature map;

[0094] It should be noted that in convolutional neural networks, convolution kernels (also called filters or weights) are the key components used to extract features from input images. The convolution operation is performed by sliding the convolution kernel over the input image and calculating the dot product between the convolution kernel and the local area of ​​the image (usually called the receptive field). This process can extract local features in the image, such as edges, textures, and shapes.

[0095] S122, using a normalization layer to normalize the convolutional features, and activating the normalized features through a nonlinear activation function to obtain an activated feature map;

[0096] It should be noted that in convolutional neural networks, normalization layers and nonlinear activation functions are two indispensable components for building deep networks. They each play an important role in feature processing and enhancing the network's expressiveness. The main purpose of the normalization layer is to adjust the numerical range of the convolutional features to make them more stable and easier to handle. Normalization can accelerate the convergence of the model, improve training efficiency, and help prevent problems such as gradient disappearance or explosion. In deep learning, commonly used normalization methods include batch normalization and layer normalization.

[0097] The nonlinear activation function is the key to introducing network nonlinearity. Without the nonlinear activation function, the multi-layer neural network will degenerate into a linear model and cannot handle complex nonlinear problems. Common nonlinear activation functions include ReLU (rectifier linear unit), GELU (Gaussian error linear unit), Sigmoid and Tanh. The normalized features are activated by a nonlinear activation function. This converts the linear combination of features into a nonlinear representation, thereby enhancing the network's expressive power.

[0098] S123, performing a maximum pooling operation on the activated features through a maximum pooling method, extracting and storing a preliminary feature map after pooling.

[0099] It should be noted that max pooling is a downsampling technique commonly used in convolutional neural networks. Its main purpose is to reduce the size of feature maps while retaining the most important feature information. Through the max pooling operation, the dimension of the data can be reduced, the amount of calculation can be reduced, and overfitting can be prevented to a certain extent, thereby improving the generalization ability of the model.

[0100] S13, using the dual-branch structure of the twin network to perform feature matching and similarity discrimination on the extracted preliminary feature map, and obtain a feature vector and a similarity discrimination result;

[0101] It should be noted that the twin network consists of two (or more) branches with the same structure and weights. Each branch processes the input feature map independently.

[0102] As a preferred implementation, the dual-branch structure of the twin network is used to perform feature matching and similarity discrimination on the extracted preliminary feature map to obtain the feature vector and similarity discrimination result, which includes the following steps:

[0103] S131, flattening the extracted preliminary features, and inputting the flattened feature graph into each branch in the twin network;

[0104] Specifically, the preliminary feature map is converted from multi-dimensional (e.g., 3D: height × width × number of channels) to a one-dimensional vector. This usually involves arranging all pixel values ​​of the feature map into a long vector in a certain order (such as row-first or column-first). The flattened feature map is input into each branch of the twin network separately. If the feature map maintains a two-dimensional structure, it is directly input as two-dimensional data; if it is converted to a one-dimensional vector, it is input as one-dimensional data.

[0105] S132. In each branch, a pixel-by-pixel correlation operation is performed on the flattened feature map, and an initial correlation map is generated by feature decomposition;

[0106] It should be noted that in each branch, the flattened feature map is subjected to pixel-by-pixel correlation operations, which can be achieved by using a convolution kernel. The role of the convolution kernel is to adjust the depth (i.e., the number of channels) of the feature map without changing the spatial size, and to capture the linear and nonlinear relationships between features, thereby generating an initial correlation map. Specifically:

[0107] Given a flattened feature map and a search region feature map, a pixel-by-pixel correlation operation can be performed by computing the cross-correlation between the two at each position. The cross-correlation operation is essentially a measure of the similarity between two signals, and when applied to a feature map, it can help determine the degree of match between different features.

[0108] In this process, feature decomposition refers to transforming the feature map through the convolution kernel to extract more meaningful feature representations. Each convolution kernel corresponds to a specific aspect or pattern of the original feature map. By learning these patterns, the important information in the feature map can be better captured. Ultimately, the initial correlation map obtained by the above operation can be regarded as a quantification of the similarity between the original feature map and the feature map of the search area. The value at each position reflects the degree of feature matching at that position.

[0109] As a preferred implementation, the principle expression of the pixel-by-pixel correlation operation is:

[0110]

[0111] Where M represents the initial correlation map output by the pixel-by-pixel correlation operation;

[0112] M j Represents the result of the cross-correlation operation between the jth 1×1 convolution kernel and the search area feature map;

[0113] K j Represents the jth 1×1 convolution kernel of the flattened feature map;

[0114] S represents the feature map of the search area;

[0115] * indicates convolution operation;

[0116] H0×W0 represents the size of the feature map after flattening.

[0117] S133, based on the attention mechanism, the initial correlation graph is optimized using channel attention and spatial attention to obtain the final correlation graph;

[0118] As a preferred implementation, the initial correlation graph is optimized based on the attention mechanism using channel attention and spatial attention to obtain the final correlation graph, including the following steps:

[0119] S1331. Calculate the variance of each channel of the initial correlation graph according to the channel attention mechanism;

[0120] It should be noted that according to the channel attention mechanism, the specific purpose of calculating the variance of each channel of the initial correlation map is to identify the differences in feature information on different channels, so as to better highlight important features. Variance indicates the degree of dispersion of data. For each channel in the feature map, the variance calculation can reflect the degree of change of the feature value in the channel. Channels with larger variances usually contain more useful feature information and can reflect the significant features or patterns of the target object. Channels with smaller variances may contain more redundant information or noise, less feature information, and relatively less contribution to target recognition.

[0121] S1332, normalizing the variance of each channel of the initial correlation graph so that each channel has zero mean and unit variance, and scaling and translating the normalized feature graph through affine transformation parameters;

[0122] It should be noted that the purpose of normalizing the variance of each channel of the initial correlation map is to standardize the feature distribution of the feature map, ensure that each channel has zero mean and unit variance, thereby reducing the feature differences between channels. At the same time, the normalized feature map is scaled and translated through the affine transformation parameters to provide flexibility for the model so that different channels can be weighted differently according to their importance.

[0123] While normalization adjusts each channel to the same standard distribution, further adjustments are sometimes made to the response strengths of different channels. Affine transformations allow for adaptive scaling and offsetting of features of each channel before attention weighting.

[0124] Operation: Enhance the expressiveness of feature maps by introducing scaling and shifting parameters.

[0125] The affine transformation formula is:

[0126] G = e × P + r;

[0127] In the formula, G represents the feature map after affine transformation, e represents the scaling factor, which is used to control the amplitude of the feature map, P represents the normalized channel, and r represents the translation factor, which is used to adjust the offset of the feature map.

[0128] S1333, based on the pre-learned weight parameters and the normalized feature map, a channel attention weight map is generated using a Sigmoid activation function, and the weight map is multiplied by the initial correlation map channel by channel to obtain a channel-optimized correlation map;

[0129] It should be noted that the weight parameters are usually obtained through deep learning models (such as convolutional neural networks). These parameters can represent the importance or pattern of features and are used to generate attention weight maps. The initial correlation map (usually a map that represents some kind of similarity or correlation) is multiplied by the channel attention weight map channel by channel to emphasize important features and suppress unimportant features.

[0130] S1334. Use the spatial attention mechanism to perform spatial weighted processing on the channel-optimized correlation graph to obtain the final correlation graph.

[0131] It should be noted that the main function of the spatial attention mechanism is to identify important spatial positions in the feature map, thereby highlighting the feature values ​​of these positions and suppressing the influence of background or unimportant areas.

[0132] In the feature map, pixel information at different spatial locations has different contributions. Through spatial weighting processing, the model can focus on key areas in the image (such as the target area in the remote sensing image).

[0133] The spatial attention weight map is a weighted representation of the channel-optimized correlation map in the spatial dimension, which is usually generated by average pooling or maximum pooling on the channel-optimized correlation map.

[0134] Average Pooling can capture the average distribution of global features, while Max Pooling can highlight the most significant features. The generation steps include:

[0135] The channel-optimized correlation graph is subjected to channel-by-channel average pooling and maximum pooling operations to obtain two two-dimensional feature maps.

[0136] The two feature maps are stacked to form a feature representation containing important spatial information.

[0137] The stacked feature map is passed into a small convolution kernel (usually 1×1 convolution) for convolution operation to further learn which spatial positions are critical. The convolved feature map is then restricted to the range of (0,1) by the Sigmoid activation function to generate a spatial attention weight map. The generated spatial attention weight map is multiplied element by element with the channel-optimized correlation map in the spatial dimension. In this way, the weight of each spatial position is adjusted according to its importance, highlighting the feature information of the key spatial positions.

[0138] S134. According to the cosine similarity principle, the similarity between the feature vectors in the final correlation graphs of the two branches is calculated to obtain the feature vector and the similarity judgment result.

[0139] It should be noted that cosine similarity is used to measure the similarity between two feature vectors, and similarity is determined by comparing their directions rather than their sizes. In a twin network, the feature maps extracted by the two branches represent the features of different inputs. By calculating their cosine similarity, we can determine how close the two images are in the feature space. Calculate the cosine similarity of the two normalized feature vectors to get a similarity score. The closer this score is to 1, the more similar the input features of the two branches are; the closer it is to -1, the greater the difference.

[0140] S14. According to the feature vector and the similarity judgment result, the feature vector that meets the preset similarity threshold is selected, and the selected feature vector is enhanced through the multi-layer perceptron structure to obtain a multi-level feature map.

[0141] It should be noted that if the similarity score of the feature vector is greater than the preset similarity threshold, these feature vectors are considered to be valuable and highly similar features. Then, through screening, those feature vectors with high correlation in the feature space are retained, thereby improving the matching or detection accuracy of the model.

[0142] The feature vector is nonlinearly mapped through a multi-layer perceptron to improve the expressiveness of the features, enabling them to better capture complex patterns and relationships. The processing steps include:

[0143] Input layer: Input the filtered feature vector into the MLP.

[0144] Hidden layer: Through multiple hidden layers and nonlinear activation functions, the feature vector is transformed and enhanced layer by layer. The depth of the hidden layer and the number of neurons depend on the specific task requirements.

[0145] Output layer: Generates enhanced feature vectors that are more expressive and can adapt to more complex patterns and tasks.

[0146] After multi-layer perceptron enhancement processing, a new set of feature vectors is obtained. These feature vectors can be reorganized into multi-level feature maps. The generated multi-level feature maps capture different aspects of the input features at different levels, so that the model has better discrimination and generalization capabilities. The multi-level feature maps can be used for subsequent target detection, classification or matching tasks to improve the overall performance of the model.

[0147] S2, based on the feature pyramid network of the fusion coordinate attention mechanism, the feature information of different levels in the multi-level feature map is fused and processed by the feature map splicing method and the feature map enhancement method to obtain the feature output map;

[0148] It should be noted that feature fusion is a key step in processing multi-scale target detection tasks, especially in the field of remote sensing images, where the target scales are diverse and the background is complex. The Feature Pyramid Network (FPN) effectively fuses the rich semantic information of high-level feature maps with the fine detail information of low-level feature maps through top-down paths and lateral connections, such as Figure 4 As shown, Figure 4 Here, Residual represents the residual module, BN represents batch normalization, and Re-weight represents weight redistribution.

[0149] As a preferred implementation, the feature pyramid network based on the fusion coordinate attention mechanism, and the feature information of different levels in the multi-level feature map are fused and processed by the feature map splicing method and the feature map enhancement method to obtain the feature output map, including the following steps:

[0150] S21, based on the feature pyramid network, a top-down path is constructed according to the multi-level feature map, and the feature maps of different levels are spliced ​​by horizontal connection;

[0151] As a preferred implementation, the expression for constructing a top-down path based on a feature pyramid network and concatenating feature maps of different levels by horizontal connection is as follows:

[0152] FPN(F i )=UpSample(Conv(Concat(F i+1 ,LateralConn(F i ))));

[0153] In the formula, FPN(F i ) represents the concatenated feature map; UpSample represents the upsampling operation, which is used to adjust the feature map size to the same as the previous layer; Conv represents the product operation, which is used to adjust the feature dimension; Concat represents the feature concatenation operation; LateralConn represents the lateral connection operation, which is used to obtain the features of the current layer; F i Represents the feature map of the i-th layer.

[0154] S22, using the coordinate attention mechanism to enhance the attention of each layer of the spliced ​​feature map, generate an attention map, and perform element-by-element multiplication operation on the generated attention map and the corresponding feature map to obtain an enhanced feature map;

[0155] It should be noted that the coordinate attention mechanism enhances feature expression by capturing important information of feature maps in spatial dimensions and introducing coordinate information. Specifically, it generates attention maps along the horizontal and vertical directions respectively, and then applies these attention maps to the original feature maps, thereby enhancing the key areas and suppressing the non-key areas.

[0156] As a preferred implementation, the coordinate attention mechanism is used to enhance the attention of each layer of the spliced ​​feature map to generate an attention map, and the generated attention map is element-by-element multiplied with the corresponding feature map to obtain the expression of the enhanced feature map:

[0157]

[0158] In the formula, CA(F) represents the enhanced feature map; F represents the concatenated feature map; σ represents the activation function, which is used to map the value of the attention map to the interval (0,1), indicating the weights of different positions; AvgPool h Indicates an average pooling operation along the horizontal direction; AvgPool w Indicates that an average pooling operation is performed along the vertical direction to obtain spatial information in the horizontal and vertical directions. Concat means concatenating the average pooling results in the horizontal and vertical directions to generate a two-dimensional attention map. Represents element-wise multiplication.

[0159] Embedding the coordinate attention mechanism in FPN can further improve the expressiveness of multi-scale feature maps. Specifically, after each layer of feature map of FPN is generated, the coordinate attention mechanism is applied to enhance it. This process can be expressed as:

[0160] Fi_CA = CA (Fi_FPN);

[0161] In the formula, Fi_FPN represents the feature map obtained by FPN through the bottom-up path (extracting features), the top-down path (up-sampling features) and the lateral connection (fusion features), and Fi_CA represents the enhanced feature map obtained by applying the coordinate attention mechanism CA to the i-th layer feature map of FPN.

[0162] The whole process first generates multi-scale feature maps through FPN, and then applies the coordinate attention mechanism to enhance the feature maps of each scale. In this way, the combination of FPN and CA not only retains the advantages of FPN in feature fusion, but also further improves the expression ability of feature maps through the coordinate attention mechanism, thereby improving the detection accuracy of multi-scale targets and the target recognition ability in complex backgrounds.

[0163] S23. The feature maps of all levels after enhancement are fused through a pre-configured fusion strategy to obtain a final feature output map.

[0164] It should be noted that the enhanced feature maps of all levels are fused according to the selected fusion method. This usually involves tensor operations such as splicing, weighted summation, etc. During the fusion process, they are made compatible through appropriate operations (such as upsampling, downsampling). After the fusion operation, a final feature output map containing information of all levels and all scales is obtained.

[0165] S3. The detection head of the FCOS algorithm performs target classification, center prediction and bounding box regression analysis on the feature output image, and determines the position of the remote sensing image target based on the analysis results to obtain the target detection result.

[0166] The detection head of the FCOS algorithm integrates target classification, bounding box regression and Center-ness prediction. Its core innovation is that the classification branch uses the Generalized Focal Loss (GFL) loss function, which solves the problem of small target sample imbalance through a dynamic quality evaluation mechanism, unifies the classification and positioning tasks, and improves the model's detection accuracy for small targets and difficult-to-classify samples. At the same time, combined with the advantages of multi-scale feature fusion, anchor-free design, and Center-ness branch to reduce false detections, the FCOS algorithm has shown efficient, accurate and extensive application prospects in the field of remote sensing image target detection, and has become a powerful tool for processing complex remote sensing image tasks.

[0167] As a preferred embodiment, the detection head includes a target classification branch, a bounding box regression branch and a center prediction branch;

[0168] A generalized focal loss function is used in the target classification branch;

[0169] The bounding box regression branch is implemented by predicting the distance from each point on the feature output graph to the four sides of the true bounding box;

[0170] The center prediction branch predicts points on the feature output graph and reduces the weight of the target center prediction box by multiplying the predicted value with the target classification score.

[0171] Specifically, if Figure 5 As shown in the figure, the detailed processing of the detection head incorporates a variety of professional terms and formulas. The following is a detailed expansion of the process, including the Generalized Focal Loss (GFL) loss function, multi-scale feature fusion, anchor-free box design, and Center-ness prediction.

[0172] The detection head is a highly integrated structure, which mainly includes three parts: classification, bounding box regression and center-ness prediction. These parts work together to achieve accurate detection of objects in remote sensing images.

[0173] The bounding box regression is achieved by predicting the distance from each point on the feature map to the four sides of the ground-truth bounding box. Let the point on the feature map be (x, y), and its corresponding ground-truth bounding box be (x0, y0, x1, y1) (upper left and lower right corner coordinates), then the regression target is four distance values ​​(l, t, r, b), which represent the distance from the point to the left, upper, right, and lower boundaries of the bounding box respectively. These distance values ​​are calculated using the following formula:

[0174] l=x-x0, t=y-y0, r=x1-x, b=y1-y;

[0175] In order to suppress low-quality prediction boxes far from the center of the target, FCOS introduces a Center-ness prediction branch. For a point (x, y) on the feature map, its corresponding Center_ness value is calculated by the following formula:

[0176]

[0177] The Center_ness value indicates how close the point is to the center of the target. The value is between 0 and 1. When the point (x, y) is close to the center of the target, the Center-ness value is close to 1; otherwise, it is close to 0. During the test phase, the predicted Center-ness value is multiplied by the classification score to reduce the weight of the predicted box far from the center of the target and improve the accuracy of the detection.

[0178] The present invention adopts the Generalized Focal Loss (GFL) loss function in the classification branch to solve the problem of small target sample imbalance and unify the classification and positioning tasks. GFL includes two parts: Quality Focal Loss (QFL) and Distribution Focal Loss (DFL).

[0179] Quality Focal Loss (QFL): To address the inconsistency between the classification score and the detection box quality score during training and inference, QFL combines the classification score and the detection box quality score (such as IoU) to form a joint score. Its loss function is:

[0180] QFL(y,σ)=-a t (1-σ t ) γ log(σ t );

[0181] Among them, y is the label (continuous value, indicating IoU), σ is the predicted value, and a t and γ are adjustment parameters, σ t It is the result of the predicted value after passing the Sigmoid function.

[0182] Distribution Focal Loss (DFL): To address the problem that bounding box regression representation is not flexible enough, DFL changes the bounding box regression value from a single fixed value to an arbitrary probability distribution within a certain range. Its loss function is based on cross entropy and aims to optimize the arbitrary probability distribution of the bounding box.

[0183] The present invention uses FPN (Feature Pyramid Network) to perform multi-scale feature fusion to improve the detection capability of objects of different sizes. Feature maps of different scales are predicted separately and different regression distance ranges are set. For example, in FPN, feature maps of lower layers are responsible for detecting smaller objects, while feature maps of higher layers are responsible for detecting larger objects.

[0184] This paper integrates Generalized Focal Loss loss function, multi-scale feature fusion, anchor-free design and Center-ness prediction and other innovative points to achieve efficient and accurate detection of targets in remote sensing images. These technologies not only solve the problem of small target sample imbalance, but also improve the detection accuracy of the model for small targets and difficult-to-classify samples, providing strong support for processing complex remote sensing image tasks. The specific verification work is as follows:

[0185] (i) In the present invention, the DOTA dataset is selected for performance verification. The full name of the dataset is "A Large-scale Dataset for Object Detection in Aerial Images". It is a large-scale dataset designed for aerial image target detection. It is derived from Google Maps and brings together aerial images from different sensors and platforms. It contains a total of 2806 high-resolution images, each of which is about 4000×4000 pixels in size and rich in details. The dataset covers 15 common target categories, such as airplanes, bridges, small and large vehicles, ships, various sports fields and facilities, etc., which is extensive and diverse. In terms of annotation, the DOTA dataset uses an arbitrary quadrilateral to accurately mark each instance, and the vertices are arranged clockwise. This innovative method can more accurately capture the direction and shape of the target, providing strong support for the research on target detection in aerial images. In order to make full use of this dataset, it is divided into training set, validation set and test set in a ratio of 2:1:1 to ensure the rationality of model training and evaluation. In addition, in order to enhance the generalization ability of the model, a variety of data enhancement techniques including flipping, rotation, and cropping are applied to effectively increase the diversity and scale of the dataset.

[0186] (ii) In order to comprehensively evaluate the performance of the proposed model in the task of remote sensing image target detection, a comparative experiment was conducted with a series of classic and mainstream target detection models; these models include Deformable DETR, TOOD, FoveaBox and RetinaNet, which are widely recognized and have excellent performance in the field of target detection; the experiment was conducted on the DOTA dataset, focusing on comparing the performance of each model in terms of average accuracy, number of parameters and amount of computation; through comparative analysis, the purpose is to verify the superiority of the present invention in multi-scale remote sensing image detection, especially its ability to balance detection accuracy and efficiency.

[0187] (iii) In order to further explore the impact of the key components of this method (ConvNeXt module, GA attention mechanism, GFL loss function) on the overall performance, a series of ablation experiments were designed; first, the backbone network was replaced from ConvNeXt to ResNet50 to evaluate the impact of different backbone networks on the model performance; then, the GA attention mechanism of the feature fusion part was gradually removed to observe the changes in the model's feature extraction and fusion capabilities; finally, the GFL loss function was removed to verify its role in optimizing model training and improving detection accuracy; through this series of ablation experiments, the contribution of each component to the performance of this model was deeply analyzed, providing strong support for the optimization and improvement of the model.

[0188] In summary, with the help of the above technical scheme of the present invention, the present invention uses an efficient convolutional neural network as the backbone of the FCOS target detection framework for feature extraction. The network deeply mines the multi-level and multi-scale features of remote sensing images through multi-layer convolution, normalization and attention mechanism, and constructs a feature pyramid rich in contextual information. Subsequently, the feature pyramid network is combined with an innovative coordinate attention mechanism to achieve the fusion of high-level semantic information and low-level detail information, and generate an enhanced multi-scale feature map to cope with the diversity of target scales in remote sensing images. The GFL loss function is used to solve the imbalance problem of small target samples, unify classification and positioning tasks, and combine multi-scale fusion with anchor-free frame design to effectively reduce false detections, thereby showing efficient, accurate and wide application potential in the field of remote sensing image target detection. The present invention uses twin multi-layer perceptrons to enhance preliminary features, and can expand feature space through nonlinear transformations, so that the model can have a deeper understanding of complex image patterns. Multi-level feature map generation can capture a wider range of details and hierarchical information, and provide multi-level support for subsequent classification or detection tasks, thereby improving the recognition and discrimination capabilities of the model. The dual-branch structure of the twin network realizes feature matching and similarity discrimination, and can accurately compare the differences between different feature maps, which is conducive to identifying similar objects in remote sensing images. It further enhances feature differences through pixel-by-pixel correlation operations and attention mechanisms, and uses channel attention and spatial attention to optimize the initial correlation map, so that the model can automatically focus on important channels and key spatial areas, thereby suppressing redundant information.

[0189] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program code.

[0190] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A remote sensing image target detection method based on improved FCOS, characterized in that: The method comprises the following steps: S1. Obtain a remote sensing target image, perform convolution and pooling operations on the remote sensing target image based on a pre-trained convolutional neural network to generate preliminary features, and use a twin multi-layer perceptron to enhance the preliminary features to obtain a multi-level feature map; S2, based on the feature pyramid network of the fusion coordinate attention mechanism, the feature information of different levels in the multi-level feature map is fused and processed by the feature map splicing method and the feature map enhancement method to obtain the feature output map; S3. The detection head of the FCOS algorithm performs target classification, center prediction and bounding box regression analysis on the feature output image, and determines the position of the remote sensing image target based on the analysis results to obtain the target detection result.

2. The improved FCOS remote sensing image target detection method according to claim 1, characterized in that: The method of acquiring a remote sensing target image, performing convolution and pooling operations on the remote sensing target image based on a pre-trained convolutional neural network to generate preliminary features, and enhancing the preliminary features using a twin multi-layer perceptron to obtain a multi-level feature map includes the following steps: S11, preprocessing the acquired remote sensing target image to obtain a preprocessed remote sensing target image; S12, using a pre-trained convolutional neural network to perform convolution and pooling operations on the pre-processed remote sensing target image, and extracting a preliminary feature map from the pre-processed remote sensing target image; S13, using the dual-branch structure of the twin network to perform feature matching and similarity discrimination on the extracted preliminary feature map, and obtain a feature vector and a similarity discrimination result; S14. According to the feature vector and the similarity judgment result, the feature vector that meets the preset similarity threshold is selected, and the selected feature vector is enhanced through the multi-layer perceptron structure to obtain a multi-level feature map.

3. The improved FCOS remote sensing image target detection method according to claim 2, characterized in that: The method of using a pre-trained convolutional neural network to perform convolution and pooling operations on the pre-processed remote sensing target image and extracting a preliminary feature map from the pre-processed remote sensing target image comprises the following steps: S121, performing a dot product operation on the preprocessed remote sensing target image through a convolution kernel in a convolutional neural network to obtain a convolved feature map; S122, using a normalization layer to normalize the convolutional features, and activating the normalized features through a nonlinear activation function to obtain an activated feature map; S123, performing a maximum pooling operation on the activated features through a maximum pooling method, extracting and storing a preliminary feature map after pooling.

4. The improved FCOS remote sensing image target detection method according to claim 2, characterized in that: The method of using the dual-branch structure of the twin network to perform feature matching and similarity discrimination on the extracted preliminary feature map to obtain the feature vector and similarity discrimination result includes the following steps: S131, flattening the extracted preliminary features, and inputting the flattened feature graph into each branch in the twin network; S132. In each branch, a pixel-by-pixel correlation operation is performed on the flattened feature map, and an initial correlation map is generated by feature decomposition; S133, based on the attention mechanism, the initial correlation graph is optimized using channel attention and spatial attention to obtain the final correlation graph; S134. According to the cosine similarity principle, the similarity between the feature vectors in the final correlation graphs of the two branches is calculated to obtain the feature vector and the similarity judgment result.

5. The improved FCOS remote sensing image target detection method according to claim 4, characterized in that: The principle expression of the pixel-by-pixel correlation operation is: Where M represents the initial correlation map output by the pixel-by-pixel correlation operation; M j Represents the result of the cross-correlation operation between the jth 1×1 convolution kernel and the search area feature map; K j Represents the jth 1×1 convolution kernel of the flattened feature map; S represents the feature map of the search area; * indicates convolution operation; H0×W0 represents the size of the feature map after flattening.

6. The improved FCOS remote sensing image target detection method according to claim 4, characterized in that: The method of optimizing the initial correlation graph using channel attention and spatial attention based on the attention mechanism to obtain the final correlation graph includes the following steps: S1331. Calculate the variance of each channel of the initial correlation graph according to the channel attention mechanism; S1332, normalizing the variance of each channel of the initial correlation graph so that each channel has zero mean and unit variance, and scaling and translating the normalized feature graph through affine transformation parameters; S1333, based on the pre-learned weight parameters and the normalized feature map, a channel attention weight map is generated using a Sigmoid activation function, and the weight map is multiplied by the initial correlation map channel by channel to obtain a channel-optimized correlation map; S1334. Use the spatial attention mechanism to perform spatial weighted processing on the channel-optimized correlation graph to obtain the final correlation graph.

7. The improved FCOS remote sensing image target detection method according to claim 1, characterized in that: The feature pyramid network based on the fusion coordinate attention mechanism fuses the feature information of different levels in the multi-level feature map through feature map splicing and feature map enhancement to obtain the feature output map, including the following steps: S21, based on the feature pyramid network, a top-down path is constructed according to the multi-level feature map, and the feature maps of different levels are spliced ​​by horizontal connection; S22, using the coordinate attention mechanism to enhance the attention of each layer of the spliced ​​feature map, generate an attention map, and perform element-by-element multiplication operation on the generated attention map and the corresponding feature map to obtain an enhanced feature map; S23. The feature maps of all levels after enhancement are fused through a pre-configured fusion strategy to obtain a final feature output map.

8. The improved FCOS remote sensing image target detection method according to claim 7, characterized in that: The expression of constructing a top-down path based on the feature pyramid network and concatenating feature maps of different levels by horizontal connection is as follows: FPN(F i )=UpSample(Conv(Concat(F i+1 ,LateralConn(F i )))); In the formula, FPN(F i ) represents the concatenated feature map; UpSample indicates upsampling operation; Conv represents the product operation; Concat represents feature concatenation operation; LateralConn represents a lateral connection operation; F i Represents the feature map of the i-th layer.

9. The improved FCOS remote sensing image target detection method according to claim 8, characterized in that: The coordinate attention mechanism is used to enhance the attention of each layer of the spliced ​​feature map to generate an attention map, and the generated attention map is multiplied element by element with the corresponding feature map to obtain the expression of the enhanced feature map: In the formula, CA(F) represents the enhanced feature map; F represents the concatenated feature map; σ represents the activation function; AvgPool h Indicates that the average pooling operation is performed along the horizontal direction; AvgPool w Indicates that the average pooling operation is performed along the vertical direction; Concat means concatenating the average pooling results in the horizontal and vertical directions; Represents element-wise multiplication.

10. The improved FCOS remote sensing image target detection method according to claim 1, characterized in that: The detection head includes a target classification branch, a bounding box regression branch and a center prediction branch; A generalized focal loss function is used in the target classification branch; The bounding box regression branch is implemented by predicting the distance from each point on the feature output graph to the four sides of the true bounding box; The center prediction branch predicts points on the feature output graph and reduces the weight of the target center prediction box by multiplying the predicted value with the target classification score.

Images