An improved FCOS remote sensing image target detection method

Abstract

The application provides an improved remote sensing image target detection method of FCOS, and relates to the technical field of target detection, and the method comprises the following steps: performing convolution and pooling operation on a remote sensing target image based on a pre-trained convolutional neural network, generating preliminary features, and using a twin multilayer perception machine to perform enhancement processing on the preliminary features; the feature information of different levels in the multi-level feature map is fused by the feature map splicing method and the feature map enhancement method; the position of the remote sensing image target is determined according to the analysis result, and the target detection result is obtained. Through the feature pyramid network combined with the innovative coordinate attention mechanism, the application realizes the fusion of high-level semantic information and low-level detail information, generates an enhanced multi-scale feature map, and responds to the diversity of target scales in the remote sensing image, combines multi-scale fusion and anchor-free design, effectively reduces false detection, and thus shows high efficiency, accuracy and wide application potential in the field of remote sensing image target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, in particular, relates to an improved FCOS remote sensing image target detection method. BACKGROUND

[0002] Traditional target detection methods, such as Faster R-CNN, SSD and YOLO series, are mainly based on anchor-based mechanism, which relies on predefined anchor boxes to generate candidate regions and adjusts them to generate prediction results. However, anchor boxes introduce multiple problems, including sensitivity, difficulty in processing targets with large scale differences, small target detection problems, and imbalance between positive and negative samples. In the field of remote sensing images, ground object targets have varying sizes, dense distribution, large aspect ratio, and uncertain orientation, which makes traditional anchor-based methods face serious challenges in handling these complex scenarios.

[0003] In recent years, anchor-free target detection methods have gradually become a research hotspot. They avoid the complex calculations and hyperparameter adjustments brought by anchor boxes, simplify model design, and improve generalization ability. FCOS (Fully Convolutional One-Stage Object Detection) as a typical anchor-free target detection method, directly predicts at each location on the feature map, achieving efficient target detection. Considering the special properties of remote sensing images, such as high resolution, large scene, complex background, etc., traditional target detection methods often fail to achieve ideal detection results.

[0004] In view of the problems in the related art, no effective solutions have been proposed so far. SUMMARY

[0005] Therefore, the present application provides an improved FCOS remote sensing image target detection method to solve the above-mentioned problems.

[0006] In order to solve the above problems, the specific technical scheme adopted by the present application is as follows:

[0007] An improved FCOS remote sensing image target detection method, the method comprising the following steps:

[0008] S1, obtaining a remote sensing target image, performing convolution and pooling operations on the remote sensing target image based on a pre-trained convolutional neural network to generate preliminary features, and using a twin multi-layer perceptron to enhance the preliminary features to obtain multi-level feature maps;

[0009] S2, a feature pyramid network based on a fusion coordinate attention mechanism, and the feature information of different levels in the multi-level feature map is fused by means of feature map splicing and feature map enhancement, so as to obtain a feature output map;

[0010] S3, the detection head of the FCOS algorithm is used to perform target classification, center degree prediction and boundary box regression analysis on the feature output map, and the position of the remote sensing image target is determined according to the analysis result, so as to obtain a target detection result.

[0011] Preferably, the remote sensing target image is obtained, the convolutional neural network is pre-trained, the convolution and pooling operations are performed on the remote sensing target image, the preliminary features are generated, and the multi-layer perceptron is used to enhance the preliminary features, so as to obtain the multi-level feature map including the following steps:

[0012] S11, the obtained remote sensing target image is preprocessed to obtain a preprocessed remote sensing target image;

[0013] S12, the preprocessed remote sensing target image is respectively subjected to convolution and pooling operations by using a pre-trained convolutional neural network, and a preliminary feature map is extracted from the preprocessed remote sensing target image;

[0014] S13, the extracted preliminary feature map is subjected to feature matching and similarity discrimination by using a double-branch structure of a twin network, so as to obtain a feature vector and a similarity discrimination result;

[0015] S14, according to the feature vector and the similarity discrimination result, the feature vector satisfying the preset similarity threshold is screened, and the screened feature vector is enhanced by using a multi-layer perceptron structure, so as to obtain a multi-level feature map.

[0016] Preferably, the preprocessed remote sensing target image is respectively subjected to convolution and pooling operations by using a pre-trained convolutional neural network, and a preliminary feature map is extracted from the preprocessed remote sensing target image including the following steps:

[0017] S121, the preprocessed remote sensing target image is subjected to dot product operation by using a convolution kernel in the convolutional neural network, so as to obtain a feature map after convolution;

[0018] S122, the normalized feature is obtained by using a normalization layer to perform normalization operation on the feature after convolution, and the feature after normalization is activated by using a nonlinear activation function, so as to obtain an activated feature map;

[0019] S123, the activated feature is subjected to maximum pooling operation by using a maximum pooling method, so as to extract and store the preliminary feature map after pooling.

[0020] Preferably, the twin network double-branch structure is used to perform feature matching and similarity discrimination on the extracted preliminary feature map to obtain a feature vector and a similarity discrimination result, including the following steps:

[0021] S131, performing feature map flattening processing on the extracted preliminary feature, and inputting the flattened feature map into each branch of the twin network;

[0022] S132, performing pixel-by-pixel correlation operation on the flattened feature map in each branch, and generating an initial correlation map through feature decomposition;

[0023] S133, based on the attention mechanism, using channel attention and spatial attention to optimize the initial correlation map to obtain a final correlation map;

[0024] S134, according to the cosine similarity principle, calculating the similarity between the feature vectors in the final correlation map in the two branches to obtain a feature vector and a similarity discrimination result.

[0025] Preferably, the principle expression of the pixel-by-pixel correlation operation is:

[0026]

[0027] In the formula, M represents the initial correlation map output by the pixel-by-pixel correlation operation;

[0028] M j represents the result of the cross-correlation operation of the jth 1x1 convolution kernel and the feature map of the search area;

[0029] K j represents the jth 1x1 convolution kernel of the flattened feature map;

[0030] S represents the feature map of the search area;

[0031] * represents the convolution operation;

[0032] H0xW0 represents the size of the flattened feature map.

[0033] Preferably, the optimization processing of the initial correlation map based on the attention mechanism, using channel attention and spatial attention to obtain the final correlation map includes the following steps:

[0034] S1331, according to the channel attention mechanism, calculating the variance of each channel of the initial correlation map;

[0035] S1332, normalizing the variance of each channel of the initial correlation map to make each channel have zero mean and unit variance, and scaling and translating the normalized feature map through affine transformation parameters;

[0036] S1333, based on the pre-learned weight parameters and the normalized feature map, a channel attention weight map is generated using a Sigmoid activation function, and the weight map is multiplied with the initial correlation map channel by channel to obtain a channel-optimized correlation map;

[0037] S1334, the spatial attention mechanism is used to perform spatial weighting processing on the channel-optimized correlation map to obtain a final correlation map.

[0038] Preferably, the feature pyramid network based on the fusion coordinate attention mechanism fuses and processes different levels of feature information in the multi-level feature map through feature map splicing and feature map enhancement to obtain a feature output map, including the following steps:

[0039] S21, based on the feature pyramid network, and according to the multi-level feature map, a top-down path is constructed, and different levels of feature maps are spliced through horizontal connection;

[0040] S22, the coordinate attention mechanism is used to perform attention enhancement on each layer of the spliced feature map to generate an attention map, and the generated attention map is multiplied with the corresponding feature map element by element to obtain an enhanced feature map;

[0041] S23, all levels of the enhanced feature map are fused through a pre-configured fusion strategy to obtain a final feature output map.

[0042] Preferably, the expression of the feature pyramid network based on the feature pyramid network and the multi-level feature map constructing a top-down path and splicing different levels of feature maps through horizontal connection is as follows:

[0043] FPN(F i )=UpSample(Conv(Concat(F i+1 ,LateralConn(F i ))));

[0044] In the formula, FPN(F i ) represents the spliced feature map;

[0045] UpSample represents the up-sampling operation;

[0046] Conv represents the multiplication operation;

[0047] Concat represents the feature splicing operation;

[0048] LateralConn represents the horizontal connection operation;

[0049] F i represents the i-th layer feature map.

[0050] Preferably, the coordinate attention mechanism is used to perform attention enhancement on each layer of the spliced feature map, to generate an attention map, and to perform element-wise multiplication operation on the generated attention map and the corresponding feature map, so as to obtain an enhanced feature map, and the expression of the enhanced feature map is as follows:

[0051]

[0052] In the formula, CA(F) represents the enhanced feature map;

[0053] F represents the feature map after splicing;

[0054] σ represents an activation function;

[0055] AvgPool h represents an average pooling operation along the horizontal direction;

[0056] AvgPool w represents an average pooling operation along the vertical direction;

[0057] Concat represents splicing of the average pooling results along the horizontal direction and the vertical direction;

[0058] represents element-wise multiplication.

[0059] Preferably, the detection head comprises a target classification branch, a bounding box regression branch and a center degree prediction branch;

[0060] The generalized focal loss function is used in the target classification branch;

[0061] The bounding box regression branch is realized by predicting the distance from each point on the feature output map to the four edges of the real bounding box;

[0062] The center degree prediction branch is to predict the points on the feature output map, and to reduce the weight of the target center prediction frame by multiplying the predicted value and the target classification score.

[0063] The present application has the following beneficial effects:

[0064] 1. The application uses an efficient convolutional neural network as the backbone of the FCOS target detection framework for feature extraction. The network deeply mines multi-level and multi-scale features of remote sensing images through multiple layers of convolution, normalization and attention mechanism, and constructs a feature pyramid rich in context information. Subsequently, through the feature pyramid network combined with the innovative coordinate attention mechanism, the fusion of high-level semantic information and low-level detail information is realized, and an enhanced multi-scale feature map is generated to cope with the diversity of target scales in remote sensing images. The GFL loss function is used to solve the small target sample imbalance problem, unify the classification and positioning tasks, and effectively reduce false detection by combining multi-scale fusion and anchor-free design, thereby showing high efficiency, accuracy and wide application potential in the field of remote sensing image target detection.

[0065] 2. The application uses a twin multi-layer perception to enhance the initial features, which can expand the feature space through nonlinear transformation, so that the model can better understand complex image patterns. The multi-level feature map generation can capture more extensive detail and hierarchical information, providing multi-level support for subsequent classification or detection tasks, thereby improving the recognition and discrimination ability of the model. The feature matching and similarity discrimination are realized through the double-branch structure of the twin network, which can accurately compare the differences between different feature maps, and is conducive to identifying similar objects in remote sensing images. The feature difference is further enhanced through pixel-by-pixel correlation operation and attention mechanism, and the initial correlation map is optimized using channel attention and spatial attention, so that the model can automatically focus on important channels and key spatial regions, thereby suppressing redundant information. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor. In the drawings:

[0067] Figure 1 is a flowchart of a remote sensing image target detection method for improving FCOS according to an embodiment of the present application;

[0068] Figure 2 is a flowchart of feature extraction, feature fusion, classification and regression in a remote sensing image target detection method for improving FCOS according to an embodiment of the present application;

[0069] Figure 3 is a feature extraction process diagram in a remote sensing image target detection method for improving FCOS according to an embodiment of the present application;

[0070] Figure 4It is a feature fusion process GA attention mechanism schematic diagram of an improved FCOS remote sensing image target detection method according to an embodiment of the present application.

[0071] Figure 5 It is a detection head schematic diagram in an improved FCOS remote sensing image target detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0073] According to an embodiment of the present application, an improved FCOS remote sensing image target detection method is provided.

[0074] The present application will be further described in conjunction with the drawings and specific embodiments. Figure 1 As shown, the improved FCOS remote sensing image target detection method according to an embodiment of the present application includes the following steps:

[0075] S1, a remote sensing target image is acquired, a pre-trained convolutional neural network is used to perform convolution and pooling operations on the remote sensing target image to generate preliminary features, and a twin multilayer perception machine is used to perform enhancement processing on the preliminary features to obtain multi-level feature maps;

[0076] It should be noted that, as shown, Figure 2 By selecting a pre-trained ConvNeXt network (convolutional neural network) as the backbone network, the network stands out with its efficient and powerful feature extraction capability. ConvNeXt, through its innovative architecture design, combines stacked convolutional layers, normalization layers and possible attention mechanisms to perform multi-level, multi-scale feature extraction and representation learning on the input remote sensing image. This process not only preserves the shallow detail information of the image, but also deeply mines the high-level semantic features, constructing a feature pyramid containing rich contextual information. These features of the ConvNeXt network provide a solid feature foundation for the subsequent classification, regression and other tasks of FCOS, significantly improving the accuracy and efficiency of remote sensing image target detection.

[0077] The ConvNeXt network is an advanced convolutional neural network architecture that combines large kernel convolutions, layer normalization, MLP (Multi-Layer Perceptron) structures, and deep hierarchical design concepts. Through multi-level and multi-scale feature extraction and representation learning, it achieves efficient processing of input remote sensing images. The core lies in capturing more extensive context information using large kernel convolutions. Although the specific formulas of the ConvNeXt network are relatively complex, its core idea can be summarized as gradually extracting and transforming image features through a series of carefully designed convolutional and normalization layers, as well as MLP structures, and finally outputting high-level feature representations for classification or subsequent processing. The core formula of the process is as follows:

[0078] Y = ConvLargeKernel (X) → LayerNorm (Y) → MLP (Y) ;

[0079] In the formula, Y represents the feature representation after each step of processing;

[0080] X represents the input remote sensing target image;

[0081] ConvLargeKernel represents the large kernel convolution operation for extracting image features;

[0082] LayerNorm represents the layer normalization operation for stabilizing the training process;

[0083] MLP represents the multi-layer perceptron structure for further converting and enhancing features;

[0084] This process not only preserves the shallow detail information of the image, but also deeply mines the high-level semantic features, constructing a feature pyramid containing rich context information. These characteristics of the ConvNeXt network provide a solid feature foundation for the subsequent classification, regression, and other tasks of FCOS, significantly improving the accuracy and efficiency of remote sensing image target detection. The ConvNeXt structure used in the backbone network of the present application is shown in Figure 3 Figure 3 Down sample represents down sampling, GlobalAvg Pooling represents global average pooling, Linear represents linear transformation, and LN: Layer Normalization represents layer normalization.

[0085] ​The ConvNeXt module effectively expands the receptive field range by introducing a 7x7 large convolution kernel in its deep convolution layer, thereby capturing more global information. At the same time, the module uses a 2x2 convolution layer combined with a convolution kernel with a step of 2 to realize the downsampling operation in the spatial dimension. In order to stabilize the training process when the spatial resolution changes, ConvNeXt adds a LayerNorm (LN) normalization layer at the corresponding position, which aims to reduce gradient fluctuations and improve training stability. In addition, ConvNeXt also draws on the architectural design essence of Swin Transformer, such as hierarchical structure and reverse bottleneck, while reducing the use of activation functions and BatchNorm. This reduces the computational burden while effectively preventing the loss of small-scale target feature information, thereby improving the overall performance of the network. In Figure 3 The GELU activation function shown in the formula (1) is designed to incorporate the idea of random regularization. For each input x, it is processed to conform to the characteristics of the standard normal distribution N(0, 1), and through multiplication with the Bernoulli distribution, it not only maintains the probabilistic characteristics of the data, but also ensures the continuity of the dependence relationship between inputs. The specific form of the GELU activation function for input x can be expressed by the given formula, which further enhances the expressive ability and generalization performance of the network. The expression of the GELU activation function is:

[0086] GELU(x) = x * σ(1.702x);

[0087] where x represents the input; σ represents the Sigmoid function.

[0088] As a preferred embodiment, the step of obtaining the remote sensing target image, performing convolution and pooling operations on the remote sensing target image based on a pre-trained convolutional neural network, generating preliminary features, and performing enhancement processing on the preliminary features using a twin multilayer perceptron to obtain a multi-level feature map includes the following steps:

[0089] S11, preprocessing the obtained remote sensing target image to obtain a preprocessed remote sensing target image;

[0090] It should be noted that preprocessing the obtained remote sensing target image includes adjusting the image size, normalizing the pixel value, data augmentation, etc.

[0091] S12, using a pre-trained convolutional neural network to perform convolution and pooling operations on the preprocessed remote sensing target image, and extracting a preliminary feature map from the preprocessed remote sensing target image;

[0092] As a preferred embodiment, the step of using a pre-trained convolutional neural network to perform convolution and pooling operations on the preprocessed remote sensing target image, and extracting a preliminary feature map from the preprocessed remote sensing target image includes the following steps:

[0093] S121, perform dot product operation on the preprocessed remote sensing target image through the convolution kernel in the convolutional neural network to obtain a feature map after convolution;

[0094] It should be noted that in the convolutional neural network, the convolution kernel (also known as filter or weight) is a key component for feature extraction of input images. Convolution operation is achieved by sliding the convolution kernel over the input image and calculating the dot product of the convolution kernel and the local region of the image (usually called receptive field). This process can extract local features such as edges, textures and shapes in the image.

[0095] S122, normalize the features after convolution using a normalization layer, and activate the normalized features through a nonlinear activation function to obtain an activated feature map;

[0096] It should be noted that in the convolutional neural network, the normalization layer and the nonlinear activation function are two indispensable components in building deep networks. They each play an important role in feature processing and enhancing network representation ability. The main purpose of the normalization layer is to adjust the numerical range of the features after convolution, making them more stable and easy to process. Normalization can speed up the convergence of the model, improve training efficiency, and help prevent problems such as gradient vanishing or explosion. In deep learning, common normalization methods include batch normalization and layer normalization.

[0097] The nonlinear activation function is the key to introducing network nonlinearity. Without the nonlinear activation function, a multi-layer neural network will degenerate into a linear model and cannot handle complex nonlinear problems. Common nonlinear activation functions include ReLU (Rectified Linear Unit), GELU (Gaussian Error Linear Unit), Sigmoid and Tanh, etc. The normalized features are activated through the nonlinear activation function. This will convert the linear combination of features into a nonlinear representation, enhancing the network's representation ability.

[0098] S123, perform maximum pooling operation on the activated features through the maximum pooling method to extract and store the preliminary feature map after pooling.

[0099] It should be noted that maximum pooling is a commonly used downsampling technique in convolutional neural networks, and its main purpose is to reduce the size of the feature map while preserving the most important feature information. Through the maximum pooling operation, the dimension of the data can be reduced, the amount of calculation can be reduced, and to some extent, overfitting can be prevented, improving the generalization ability of the model.

[0100] S13, use the double-branch structure of the twin network to perform feature matching and similarity discrimination on the extracted preliminary feature map to obtain a feature vector and a similarity discrimination result;

[0101] It should be noted that the twin network is composed of two (or more) branches with the same structure and weights. Each branch independently processes the input feature map.

[0102] As a preferred embodiment, the feature matching and similarity discrimination of the extracted preliminary feature map using the dual-branch structure of the twin network includes the following steps:

[0103] S131, the extracted preliminary feature is flattened into a feature map, and the flattened feature map is input into each branch of the twin network;

[0104] Specifically, the preliminary feature map is converted from multi-dimensional (for example, three-dimensional: height x width x channel number) to one-dimensional vector. This usually involves arranging all pixel values of the feature map into a long vector in a certain order (such as row first or column first). The flattened feature map is input into each branch of the twin network respectively. If the feature map remains two-dimensional structure, it is directly input as two-dimensional data; if converted to one-dimensional vector, it is input as one-dimensional data.

[0105] S132, in each branch, the flattened feature map is subjected to pixel-by-pixel correlation operation, and an initial correlation map is generated by feature decomposition;

[0106] It should be noted that in each branch, the flattened feature map is subjected to pixel-by-pixel correlation operation, which can be achieved by using convolution kernel. The role of convolution kernel is to adjust the depth (i.e. channel number) of feature map without changing the spatial size, and can capture the linear and nonlinear relationship between features, so as to generate the initial correlation map. Specifically:

[0107] For a given flattened feature map and feature map of search area, pixel-by-pixel correlation operation can be completed by calculating the cross-correlation at each position between the two. Cross-correlation operation essentially measures the similarity between two signals, and when applied to feature map, it can help determine the matching degree between different features.

[0108] In this process, feature decomposition refers to the transformation of feature map by convolution kernel to extract more meaningful feature representation. Each convolution kernel corresponds to a specific aspect or pattern of the original feature map, and through the learning of these patterns, important information in the feature map can be better captured. Finally, the initial correlation map obtained through the above operation can be regarded as a quantification of the similarity between the original feature map and the feature map of the search area. The value at each position reflects the degree of feature matching at that position.

[0109] As a preferred embodiment, the principle expression of the pixel-by-pixel correlation operation is:

[0110]

[0111] M represents an initial correlation map output by the pixel-by-pixel correlation operation;

[0112] M j represents the result of the cross-correlation operation of the jth 1x1 convolution kernel and the feature map of the search region;

[0113] K j represents the jth 1x1 convolution kernel of the flattened feature map;

[0114] S represents the feature map of the search region;

[0115] * represents a convolution operation;

[0116] H0xW0 represents the size of the flattened feature map.

[0117] S133, based on the attention mechanism, the initial correlation map is optimized by using channel attention and spatial attention, and a final correlation map is obtained;

[0118] As a preferred embodiment, the step of optimizing the initial correlation map based on the attention mechanism to obtain the final correlation map includes the following steps:

[0119] S1331, according to the channel attention mechanism, the variance of each channel of the initial correlation map is calculated;

[0120] It should be noted that the specific purpose of calculating the variance of each channel of the initial correlation map according to the channel attention mechanism is to identify the difference of feature information on different channels, so as to better highlight important features. Variance represents the dispersion degree of data. For each channel in the feature map, variance calculation can reflect the degree of change of the feature value in the channel. The channel with larger variance usually contains more useful feature information, which can reflect the significant features or patterns of the target object. The channel with smaller variance may contain more redundant information or noise, and the feature information is less, and the contribution to target recognition is relatively small.

[0121] S1332, the variance of each channel of the initial correlation map is normalized to have zero mean and unit variance, and the normalized feature map is scaled and translated through affine transformation parameters;

[0122] It should be noted that the purpose of normalizing the variance of each channel of the initial correlation map is to standardize the feature distribution of the feature map, ensuring that each channel has zero mean and unit variance, thereby reducing the feature difference between channels. At the same time, the normalized feature map is scaled and translated by the affine transformation parameter, providing flexibility to the model, so that different channels can be weighted differently according to their importance.

[0123] Although normalization adjusts each channel to the same standard distribution, sometimes the response intensity of different channels is further adjusted. Affine transformation allows adaptive scaling and shifting of features in each channel before attention weighting.

[0124] Operation: The expressiveness of the feature map is enhanced by introducing scaling and shifting parameters.

[0125] The affine transformation formula is:

[0126] G = e x P + r;

[0127] In the formula, G represents the feature map after affine transformation, e represents the scaling factor, which is used to control the amplitude of the feature map, P represents the normalized channel, and r represents the translation factor, which is used to adjust the offset of the feature map.

[0128] S1333, based on the pre-learned weight parameters and the normalized feature map, a Sigmoid activation function is used to generate a channel attention weight map, and the weight map is multiplied with the initial correlation map channel by channel to obtain a channel-optimized correlation map;

[0129] It should be noted that the weight parameters are usually obtained by training a deep learning model (such as a convolutional neural network). These parameters can represent the importance or patterns of features, and are used to generate attention weight maps. The initial correlation map (which usually represents some similarity or correlation) is multiplied with the channel attention weight map channel by channel to emphasize important features and suppress unimportant features.

[0130] S1334, using a spatial attention mechanism to perform spatial weighting processing on the channel-optimized correlation map to obtain a final correlation map.

[0131] It should be noted that the main function of the spatial attention mechanism is to identify important spatial positions in the feature map, thereby highlighting the feature values of these positions and suppressing the influence of background or unimportant regions.

[0132] In the feature map, the pixel information at different spatial positions has different contribution degrees. Through spatial weighting processing, the focus of the model can be concentrated on the key regions in the image (such as the target region in a remote sensing image).

[0133] The spatial attention weight map is a weighted representation of the channel-optimized correlation map in the spatial dimension, which is usually generated by average pooling or max pooling on the channel-optimized correlation map.

[0134] Average Pooling can capture the average distribution of global features, and MaxPooling can highlight the most significant features. The generation step includes:

[0135] The channel-optimized correlation map is subjected to channel-wise average pooling and max pooling operations to obtain two two-dimensional feature maps.

[0136] Stacking the two feature maps forms a feature representation containing important spatial information.

[0137] The stacked feature map is passed through a small convolution kernel (usually 1x1 convolution) for convolution operation to further learn which spatial positions are critical. The convolutional feature map is then passed through a Sigmoid activation function to limit the weight value to the range (0, 1), thereby generating a spatial attention weight map. The generated spatial attention weight map is multiplied element-wise with the channel-optimized correlation map in the spatial dimension. In this way, the weight of each spatial position is adjusted according to its importance, highlighting the feature information of key spatial positions.

[0138] S134, according to the cosine similarity principle, the similarity between the feature vectors in the final correlation map in the two branches is calculated, and the feature vector and similarity discrimination result is obtained.

[0139] It should be noted that the cosine similarity is used to measure the degree of similarity between two feature vectors, and the similarity is determined by comparing their directions rather than their sizes. In a twin network, the feature maps extracted by the two branches represent the features of different inputs. By calculating their cosine similarity, the closeness of the two images in the feature space can be determined. The cosine similarity of the two normalized feature vectors is calculated to obtain a similarity score. The closer this score is to 1, the more similar the input features of the two branches are; the closer it is to -1, the greater the difference.

[0140] S14, according to the feature vector and similarity discrimination result, the feature vectors that meet the preset similarity threshold are screened, and the screened feature vectors are enhanced by a multi-layer perceptron structure to obtain multi-level feature maps.

[0141] It should be noted that if the similarity score of the feature vector is greater than the preset similarity threshold, it is considered that these feature vectors are valuable and have high similarity. Then through screening, those feature vectors with high correlation in the feature space are retained, thereby improving the matching or detection accuracy of the model.

[0142] The feature vectors are nonlinearly mapped by a multi-layer perceptron, enhancing the expressiveness of the features and enabling them to better capture complex patterns and relationships. The processing steps include:

[0143] Input layer: The filtered feature vectors are input into the MLP.

[0144] Hidden layers: The feature vectors are transformed and enhanced layer by layer through multiple hidden layers and nonlinear activation functions. The depth and number of neurons in the hidden layers depend on the specific task requirements.

[0145] Output layer: The enhanced feature vectors are generated, which are more expressive and can adapt to more complex patterns and tasks.

[0146] After the multi-layer perceptron enhancement process, a new set of feature vectors is obtained. These feature vectors can be reorganized into a multi-level feature map. The generated multi-level feature map captures different aspects of information in the input features at different levels, enabling the model to have better discrimination and generalization capabilities. The multi-level feature map can be used for subsequent target detection, classification, or matching tasks, improving the overall performance of the model.

[0147] S2, a feature pyramid network based on a fusion coordinate attention mechanism, and through feature map splicing and feature map enhancement, the feature information in the multi-level feature map at different levels is fused to obtain a feature output map;

[0148] It should be noted that feature fusion is a key step in processing multi-scale target detection tasks, especially in the field of remote sensing images, where target scales are diverse and backgrounds are complex. The feature pyramid network (Feature Pyramid Network, FPN) effectively fuses the rich semantic information of high-level feature maps and the fine detailed information of low-level feature maps through a top-down path and horizontal connection, as shown in Figure 4 Residual indicates a residual module, BN indicates batch normalization (Batch Normalization), and Re-weight indicates weight redistribution. Figure 4

[0149] As a preferred embodiment, the feature pyramid network based on the fusion coordinate attention mechanism, and through the feature map splicing and feature map enhancement, the feature information in the multi-level feature map at different levels is fused to obtain a feature output map, including the following steps:

[0150] S21, based on the feature pyramid network, and according to the multi-level feature map, a top-down path is constructed, and different levels of feature maps are spliced through horizontal connection; ​

[0151] As a preferred embodiment, the feature pyramid network is used, and a top-down path is constructed according to multi-level feature maps, and the expression of splicing of feature maps of different levels is performed by means of lateral connection:

[0152] FPN(F i )=UpSample(Conv(Concat(F i+1 ,LateralConn(F i ))));

[0153] In the formula, FPN(F i ) represents the spliced feature map; UpSample represents the up-sampling operation, which is used to adjust the size of the feature map to be the same as the previous layer; Conv represents the convolution operation, which is used to adjust the feature dimension; Concat represents the feature splicing operation; LateralConn represents the lateral connection operation, which is used to obtain the feature of the current layer; and F i represents the feature map of the i-th layer.

[0154] S22, using a coordinate attention mechanism to perform attention enhancement on each layer of the spliced feature map, generating an attention map, and performing an element-wise multiplication operation on the generated attention map and the corresponding feature map to obtain an enhanced feature map;

[0155] It should be noted that the coordinate attention mechanism captures important information of the feature map in the spatial dimension and introduces coordinate information to enhance the feature expression. Specifically, it generates attention maps along the horizontal and vertical directions respectively, and then applies these attention maps to the original feature map, thereby enhancing the key regions and suppressing the non-key regions.

[0156] As a preferred embodiment, the expression of using a coordinate attention mechanism to perform attention enhancement on each layer of the spliced feature map, generating an attention map, and performing an element-wise multiplication operation on the generated attention map and the corresponding feature map to obtain an enhanced feature map is as follows:

[0157]

[0158] In the formula, CA(F) represents the enhanced feature map; F represents the spliced feature map; σ represents an activation function, which is used to map the value of the attention map to the interval (0, 1), indicating the weight of different positions; AvgPool h represents an average pooling operation along the horizontal direction; AvgPool w represents an average pooling operation along the vertical direction to obtain spatial information in the horizontal and vertical directions, and Concat represents splicing of the average pooling results of the horizontal and vertical directions to generate a two-dimensional attention map; represents an element-wise multiplication.

[0159] The coordinate attention mechanism is embedded in the FPN, which can further improve the expression ability of multi-scale feature maps. Specifically, after the generation of each layer feature map of the FPN, the coordinate attention mechanism is applied for enhancement. This process can be represented as:

[0160] Fi_CA=CA(Fi_FPN);

[0161] In the formula, Fi_FPN represents the feature map obtained by the FPN through the steps of bottom-up path (extracting features), top-down path (up-sampling features), and horizontal connection (fusing features), Fi_CA represents the enhanced feature map obtained by applying the coordinate attention mechanism CA to the i-th layer feature map of the FPN, and Fi_FPN represents the enhanced feature map obtained by applying the coordinate attention mechanism CA to the i-th layer feature map of the FPN.

[0162] The whole process first generates multi-scale feature maps through the FPN, and then applies the coordinate attention mechanism to each scale of the feature map for enhancement. In this way, the combination of FPN and CA not only retains the advantages of FPN in feature fusion, but also further improves the expression ability of the feature map through the coordinate attention mechanism, thereby improving the detection accuracy of multi-scale targets and the target recognition ability in complex background.

[0163] S23, fuse all enhanced feature maps of all levels according to the pre-configured fusion strategy to obtain a final feature output map.

[0164] It should be noted that all enhanced feature maps of all levels are fused according to the selected fusion method. This usually involves tensor operations such as splicing, weighted summation, etc. In the fusion process, they are made compatible through appropriate operations (such as up-sampling, down-sampling). After the fusion operation, a final feature output map containing all levels and all scale information is obtained.

[0165] S3, the feature output map is subjected to target classification, center degree prediction and bounding box regression analysis by the detection head of the FCOS algorithm, and the position of the remote sensing image target is determined according to the analysis result to obtain the target detection result.

[0166] The FCOS algorithm's detection head integrates target classification, bounding box regression, and center-ness prediction. Its core innovation lies in employing the Generalized Focal Loss (GFL) loss function in the classification branch. This loss function addresses the imbalance problem of small target samples through a dynamic quality assessment mechanism, unifying the classification and localization tasks and improving the model's detection accuracy for small targets and difficult-to-classify samples. Furthermore, combining the advantages of multi-scale feature fusion, anchor-free design, and the center-ness branch's reduction of false detections, the FCOS algorithm demonstrates high efficiency, accuracy, and broad application prospects in remote sensing image target detection, becoming a powerful tool for handling complex remote sensing image tasks.

[0167] In a preferred embodiment, the detection head includes a target classification branch, a bounding box regression branch, and a centrality prediction branch;

[0168] The generalized focus loss function is used in the target classification branch;

[0169] The bounding box regression branch is achieved by predicting the distance from each point on the feature output map to the four sides of the true bounding box.

[0170] The centrality prediction branch predicts points on the feature output map and reduces the weight of the target center prediction box by multiplying the predicted value by the target classification score.

[0171] Specifically, such as Figure 5 As shown, the detection head section incorporates various technical terms and formulas in its detailed processing. The following is a detailed expansion of this process, including the Generalized Focal Loss (GFL) loss function, multi-scale feature fusion, anchor-free design, and center-ness prediction.

[0172] The detection head is a highly integrated structure, mainly consisting of three parts: classification, bounding box regression, and center-ness prediction. These parts work together to achieve accurate detection of targets in remote sensing images.

[0173] Bounding box regression is achieved by predicting the distances from each point on the feature map to the four edges of the ground-truth bounding box. Let a point on the feature map be (x, y), and its corresponding ground-truth bounding box be (x0, y0, x1, y1) (coordinates of the top-left and bottom-right corners). The regression target is four distance values ​​(l, t, r, b), representing the distances from the point to the left, top, right, and bottom edges of the bounding box, respectively. These distance values ​​are calculated using the following formula:

[0174] l = x - x0, t = y - y0, r = x1 - x, b = y1 - y;

[0175] To suppress low-quality prediction boxes far from the center of the target, FCOS introduces a Center-ness prediction branch. For a point (x, y) on the feature map, its corresponding Center_ness value is calculated by the following formula:

[0176]

[0177] The Center_ness value represents the proximity of the point to the center of the target, and the value is between 0 and 1. When the point (x, y) is close to the center of the target, the Center-ness value is close to 1; otherwise, it is close to 0. In the test stage, the predicted Center-ness value is multiplied by the classification score, thereby reducing the weight of the prediction box far from the center of the target and improving the accuracy of detection.

[0178] The present application adopts a Generalized Focal Loss (GFL) loss function in the classification branch to solve the small target sample imbalance problem and unify the classification and positioning tasks. GFL includes two parts: Quality Focal Loss (QFL) and Distribution Focal Loss (DFL).

[0179] Quality Focal Loss (QFL): To solve the problem that the classification score and the detection box quality score are inconsistent in training and inference, QFL combines the classification score and the detection box quality score (such as IoU) to form a joint score. Its loss function form is:

[0180] QFL (y, s) = -a t (1 - s t ) γ log (s t );

[0181] Where y is the label (continuous value, representing IoU), s is the predicted value, a t and g are adjustment parameters, s t is the result of the predicted value after the Sigmoid function.

[0182] Distribution Focal Loss (DFL): To solve the problem that the boundary box regression representation is not flexible enough, DFL changes the regression value of the boundary box from a single determined value to an arbitrary probability distribution within a certain range. Its loss function form is based on cross-entropy, aiming to optimize the arbitrary probability distribution of the boundary box.

[0183] The present application uses FPN (Feature Pyramid Network) for multi-scale feature fusion to improve the detection ability of different size targets. Different scale feature maps are predicted respectively, and different regression distance ranges are set. For example, in FPN, the lower layer feature map is responsible for detecting smaller targets, and the higher layer feature map is responsible for detecting larger targets.

[0184] The present application realizes efficient and accurate detection of targets in remote sensing images by integrating Generalized Focal Loss loss function, multi-scale feature fusion, anchor-free design and Center-ness prediction. These technologies not only solve the problem of small target sample imbalance, but also improve the detection accuracy of small targets and difficult classification samples, providing strong support for complex remote sensing image tasks. The specific verification work is as follows:

[0185] (i) In the present application, DOTA dataset is selected for performance verification. The full name of the dataset is "A Large-scale Dataset for Object Detection in Aerial Images", which is a large-scale dataset designed for aerial image target detection. It is derived from Google Maps and gathers aerial images from different sensors and platforms, totaling 2806 high-resolution images, each with a size of about 4000x4000 pixels and rich details. The dataset covers 15 common target categories, such as airplanes, bridges, small and large vehicles, ships, various sports fields and facilities, etc., which are extensive and diverse. In labeling, the DOTA dataset uses arbitrary quadrilaterals to accurately mark each instance, with the vertices arranged in a clockwise direction. This innovative approach can more accurately capture the direction and shape of the target, providing strong support for target detection research in aerial images. To fully utilize this dataset, it is divided into training, validation and test sets in the ratio of 2:1:1 to ensure the rationality of model training and evaluation. In addition, to enhance the generalization ability of the model, a variety of data augmentation techniques including flipping, rotation and cropping are applied, effectively increasing the diversity and size of the dataset.

[0186] (ii) To comprehensively evaluate the performance of the model proposed in this paper in the task of remote sensing image target detection, comparative experiments were conducted with a series of classic and mainstream target detection models, including Deformable DETR, TOOD, FoveaBox and RetinaNet, which have wide recognition and excellent performance in the field of target detection. The experiment was conducted on the DOTA dataset, focusing on comparing the performance of each model in terms of average accuracy, parameter quantity and computational quantity. Through comparative analysis, the superiority of the present application in multi-scale remote sensing image detection, especially in the balance of detection accuracy and efficiency, is verified.

[0187] (iii) To further explore the impact of key components (ConvNeXt module, GA attention mechanism, and GFL loss function) on the overall performance of this method, a series of ablation experiments were designed. First, the backbone network was replaced from ConvNeXt to ResNet50 to evaluate the impact of different backbone networks on model performance. Next, the GA attention mechanism in the feature fusion part was gradually removed to observe the changes in the model's feature extraction and fusion capabilities. Finally, the GFL loss function was removed to verify its role in optimizing model training and improving detection accuracy. Through this series of ablation experiments, the contribution of each component to the performance of this model was analyzed in depth, providing strong support for the optimization and improvement of the model.

[0188] In summary, by utilizing the above-mentioned technical solution of this invention, this invention employs a highly efficient convolutional neural network as the backbone of the FCOS target detection framework for feature extraction. This network, through multi-layer convolution, normalization, and attention mechanisms, deeply mines multi-level and multi-scale features of remote sensing images, constructing a feature pyramid rich in contextual information. Subsequently, by combining the feature pyramid network with an innovative coordinate attention mechanism, it achieves the fusion of high-level semantic information and low-level detail information, generating enhanced multi-scale feature maps to address the diversity of target scales in remote sensing images. The GFL loss function is used to solve the imbalance problem of small target samples, unifying classification and localization tasks. Furthermore, by combining multi-scale fusion and anchor-free design, false detections are effectively reduced, thus demonstrating efficient, accurate, and broad application potential in the field of remote sensing image target detection. This invention uses a Siamese multilayer perceptron to enhance initial features. It can expand the feature space through nonlinear transformation, enabling the model to understand complex image patterns more deeply. The generation of multi-level feature maps can capture a wider range of details and hierarchical information, providing multi-level support for subsequent classification or detection tasks, thereby improving the model's recognition and discrimination capabilities. Feature matching and similarity discrimination are achieved through the dual-branch structure of the Siamese network, which can accurately compare the differences between different feature maps, which is beneficial for identifying similar objects in remote sensing images. Feature differences are further enhanced through pixel-wise correlation operations and attention mechanisms. Channel attention and spatial attention are used to optimize the initial correlation map, enabling the model to automatically focus on important channels and key spatial regions, thereby suppressing redundant information.

[0189] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.

[0190] The above-described specific embodiments further illustrate the objects, technical solutions, and beneficial effects of the present application. It should be understood that the above-described specific embodiments are merely examples of the present application and are not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An improved remote sensing image target detection method of FCOS, characterized in that, The method comprises the following steps: S1, acquiring a remote sensing target image, performing convolution and pooling operations on the remote sensing target image based on a pre-trained convolutional neural network to generate preliminary features, and performing enhancement processing on the preliminary features using a twin multilayer perception machine to obtain a multi-level feature map; S2, based on a feature pyramid network of a fusion coordinate attention mechanism, and through a feature map splicing method and a feature map enhancement method, the feature information of different levels in the multi-level feature map is fused to obtain a feature output map; S3, the feature output map is subjected to target classification, center degree prediction and bounding box regression analysis through a detection head of the FCOS algorithm, and the position of the remote sensing image target is determined according to the analysis result to obtain a target detection result.

2. The improved FCOS remote sensing image target detection method according to claim 1, characterized in that, The acquisition of the remote sensing target image, the convolution and pooling operations on the remote sensing target image based on the pre-trained convolutional neural network to generate preliminary features, and the enhancement processing on the preliminary features using the twin multilayer perception machine to obtain the multi-level feature map comprise the following steps: S11, preprocessing the acquired remote sensing target image to obtain a preprocessed remote sensing target image; S12, performing convolution and pooling operations on the preprocessed remote sensing target image using the pre-trained convolutional neural network, and extracting a preliminary feature map from the preprocessed remote sensing target image; S13, using the double-branch structure of the twin network to perform feature matching and similarity discrimination on the extracted preliminary feature map to obtain a feature vector and a similarity discrimination result; S14, according to the feature vector and the similarity discrimination result, screening the feature vectors satisfying the preset similarity threshold, and enhancing the screened feature vectors through the multilayer perception machine structure to obtain the multi-level feature map.

3. The improved FCOS remote sensing image target detection method according to claim 2, characterized in that, The convolution and pooling operations on the preprocessed remote sensing target image using the pre-trained convolutional neural network, and the extraction of the preliminary feature map from the preprocessed remote sensing target image comprise the following steps: S121, performing dot product operation on the preprocessed remote sensing target image through the convolution kernel in the convolutional neural network to obtain a convolution feature map; S122, performing normalization operation on the convolution feature using the normalization layer, and activating the normalized feature through the nonlinear activation function to obtain an activated feature map; S123, performing maximum pooling operation on the activated feature to extract and store the pooled preliminary feature map.

4. The improved FCOS remote sensing image target detection method according to claim 2, characterized in that, The use of the double-branch structure of the twin network to perform feature matching and similarity discrimination on the extracted preliminary feature map to obtain a feature vector and a similarity discrimination result comprises the following steps: S131, flattening the extracted preliminary feature into a feature map, and inputting the flattened feature map into each branch of the twin network; S132, performing pixel-by-pixel correlation operation on the flattened feature map in each branch to generate an initial correlation map through feature decomposition; S133, based on the attention mechanism, optimizing the initial correlation map using channel attention and spatial attention to obtain a final correlation map; S134, according to the cosine similarity principle, the similarity between the feature vectors in the final correlation graph in the two branches is calculated, and the similarity discrimination result of the feature vectors is obtained.

5. The improved FCOS remote sensing image target detection method according to claim 4, characterized in that, The principle expression of the pixel-by-pixel correlation operation is: In the formula, M represents the initial correlation graph output by the pixel-by-pixel correlation operation; M j represents the result of the cross-correlation operation between the jth 1x1 convolution kernel and the search region feature map; K j a jth 1 x 1 convolution kernel representing a flattened feature map; S represents the feature graph of the search area; * represents the convolution operation; H0*W0 represents the size of the flattened feature graph after processing.

6. The improved FCOS remote sensing image target detection method according to claim 4, characterized in that, The final correlation graph obtained by optimizing the initial correlation graph based on the attention mechanism includes the following steps: S1331, according to the channel attention mechanism, the variance of each channel of the initial correlation graph is calculated; S1332, the variance of each channel of the initial correlation graph is normalized to have zero mean and unit variance, and the normalized feature graph is scaled and translated through affine transformation parameters; S1333, based on the pre-learned weight parameters and the normalized feature graph, a Sigmoid activation function is used to generate a channel attention weight map, and the weight map is multiplied with the initial correlation graph channel by channel to obtain a channel-optimized correlation graph; S1334, the spatial weighting mechanism is used to process the channel-optimized correlation graph to obtain the final correlation graph.

7. The improved FCOS remote sensing image target detection method according to claim 1, characterized in that, The feature pyramid network based on the fusion coordinate attention mechanism fuses and processes different levels of feature information in the multi-level feature graph through feature graph splicing and feature graph enhancement to obtain a feature output graph, including the following steps: S21, based on the feature pyramid network, and according to the multi-level feature graph, a top-down path is constructed, and different levels of feature graphs are spliced through horizontal connection; S22, the coordinate attention mechanism is used to enhance the attention of each layer of feature graph after splicing to generate an attention map, and the generated attention map is multiplied with the corresponding feature graph element by element to obtain an enhanced feature graph; S23, all levels of feature graphs after enhancement are fused according to the pre-configured fusion strategy to obtain the final feature output graph.

8. The improved FCOS remote sensing image target detection method according to claim 7, characterized in that, The expression of splicing different levels of feature graphs through horizontal connection based on the feature pyramid network and according to the multi-level feature graph constructing a top-down path is: FPN(F i ) = UpSample(Conv(Concat(F i+1 , LateralConn(F i ))) ); In the formula, FPN(F i ) represents the feature map after splicing; UpSample represents the up-sampling operation; Conv represents the multiplication operation; Concat represents the feature splicing operation; LateralConn represents the horizontal connection operation; F i denotes the feature map of the i-th layer.

9. The improved FCOS remote sensing image target detection method according to claim 8, characterized in that, The expression of enhancing the attention of each layer of feature graph after splicing through the coordinate attention mechanism to generate an attention map, and multiplying the generated attention map with the corresponding feature graph element by element to obtain an enhanced feature graph is: In the formula, CA(F) represents the enhanced feature graph; F represents the feature graph after splicing; σ represents the activation function; AvgPool h represents an average pooling operation along the horizontal direction; AvgPool w represents an average pooling operation along the vertical direction; Concat represents the average pooling result of the splicing horizontal direction and vertical direction; represents element-wise multiplication.

10. The improved FCOS remote sensing image target detection method according to claim 1, characterized in that, The detection head includes a target classification branch, a bounding box regression branch, and a center degree prediction branch; The generalized focal loss function is used in the target classification branch; The bounding box regression branch is realized by predicting the distance from each point on the feature output graph to the four edges of the real bounding box; The center degree prediction branch is to predict the points on the feature output graph, and reduce the weight of the target center prediction box by multiplying the predicted value with the target classification score.

Images