A remote sensing image target detection method based on improved YOLOv9s

Abstract

The application relates to the technical field of computer vision and artificial intelligence, and provides a remote sensing image target detection method based on an improved YOLOv9s, which comprises the following steps: acquiring a remote sensing image and performing a pretreatment operation; introducing a kernel selection feature fusion (KSFF) module into a neck network of a YOLOv9s model, and replacing at least one Concat structure in the original model; adding a cross-space multi-scale attention (CSMA) module in the neck network; replacing at least one SPPELAN structure in the original model with a parallel pooling feature modulation (PPFM) module in the neck network; and training, testing and evaluating the improved YOLOv9s network. Through the comprehensive application of network structure improvement, data pretreatment optimization, a semi-supervised learning strategy and an advanced training method, the improved YOLOv9s model has higher detection precision in a remote sensing image target detection task.

Description

A Remote Sensing Image Target Detection Method Based on Improved YOLOv9s Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to a remote sensing image target detection method based on an improved YOLOv9s. Background Technology

[0002] Remote sensing image target detection is a core technology in fields such as land monitoring, military reconnaissance, environmental protection, and traffic control. It uses computer vision algorithms to identify and locate targets such as aircraft, ships, bridges, and buildings from high-resolution remote sensing images, providing crucial data support for decision-making. With the development of deep learning technology, target detection models based on convolutional neural networks (CNNs) have become the mainstream solution in this field. Among them, the YOLO series models are widely used in remote sensing image tasks due to their balance between detection accuracy and inference speed. The YOLOv9s model, in particular, has significant advantages in embedded devices and real-time detection scenarios thanks to its lightweight backbone and efficient neck network design.

[0003] However, the unique characteristics of remote sensing images (such as atmospheric interference, large differences in target scale, a high proportion of small targets, and scarce labeled data) result in the following key shortcomings in existing technologies:

[0004] First, image preprocessing capabilities are insufficient, resulting in limited atmospheric interference suppression. Remote sensing images are easily affected by atmospheric factors such as clouds, haze, and aerosols, leading to blurred target features and reduced contrast. Existing preprocessing methods mainly fall into two categories: one is based on atmospheric physical models (such as dark channel priors and atmospheric scattering models), which can initially correct atmospheric interference but has poor adaptability to complex scenes (such as local thick clouds and non-uniform haze), easily resulting in over-correction or incomplete correction; the other is based on pure deep learning (such as the U-Net dehazing network), which relies on a large amount of labeled data and does not fully incorporate the prior knowledge of physical models, easily resulting in the loss of original ground feature details in the corrected image. In addition, existing fusion strategies mostly use fixed weights to fuse the original and corrected images, failing to dynamically adjust according to local image quality (such as contrast and edge sharpness), making it difficult to simultaneously address the processing effects of target areas (such as buildings with sharp edges) and interference areas (such as cloud-covered areas).

[0005] Second, there are design flaws in the neck network feature fusion and attention mechanism. The neck network of YOLOv9s uses SPPELAN module, Concat splicing, and upsampling structure. Its core flaws are: low feature fusion accuracy, lack of attention mechanism, and insufficient multi-scale context capture.

[0006] Third, the performance of small target detection is insufficient to meet practical needs. In remote sensing images, small targets (such as those with a pixel size <32×32) typically account for more than 40%. These targets have low feature dimensionality, little detail, and are easily drowned out by background noise. Existing YOLOv9s models rely on shallow features of the backbone network for small target detection. However, shallow features have weak semantic information, and the neck network lacks a dedicated feature enhancement mechanism for small targets. This results in a mAP (mean accuracy) for small targets generally being below 75%, which cannot meet the high-precision requirements for identifying small facilities in land monitoring.

[0007] Fourth, the model is highly dependent on data and it is difficult to balance training efficiency with model performance. Remote sensing image annotation requires professional personnel to complete the work in conjunction with geographic information, which is costly and time-consuming. This results in a limited number of high-quality annotated datasets (such as DIOR and SIMD), making the model prone to overfitting. Existing semi-supervised training strategies mostly use "weak enhancement + pseudo-labels" without combining style transfer to generate synthetic samples, resulting in limited data augmentation effects. At the same time, in model training, pure FP32 precision training is inefficient, pure FP16 precision is prone to gradient vanishing, and there is a lack of robust training against remote sensing image noise (such as sensor noise and cloud interference), resulting in poor generalization ability of the model in real-world scenarios.

[0008] In summary, existing remote sensing image target detection methods have significant shortcomings in terms of image preprocessing accuracy, multi-scale feature utilization, small target detection performance, data dependence, and training efficiency. There is an urgent need for a remote sensing image target detection method based on improved YOLOv9s. Summary of the Invention

[0009] This invention provides a remote sensing image target detection method based on an improved YOLOv9s. By optimizing the preprocessing process, neck network structure, sample generation strategy and training mechanism, it solves problems such as insufficient atmospheric interference suppression, low feature fusion accuracy, high false negative rate for small targets and strong data dependence. It achieves the goals of improving the accuracy, enhancing robustness, reducing data dependence and balancing efficiency and complexity in remote sensing image target detection.

[0010] This invention provides a remote sensing image target detection method based on an improved YOLOv9s, comprising the following steps:

[0011] Step 1: Acquire remote sensing images and perform preprocessing operations;

[0012] Step 2: Introduce a Kernel Selective Feature Fusion (KSFF) module into the neck network of the YOLOv9s model, replacing at least one Concat structure in the original model; the Kernel Selective Feature Fusion (KSFF) module extracts multi-scale features through multiple convolutional kernels and combines channel attention weighting to achieve accurate fusion;

[0013] Step 3: Add a cross-spatial multi-scale attention (CSMA) module to the neck network; the cross-spatial multi-scale attention (CSMA) module is used to enhance the correlation of cross-spatial features to focus on the target region;

[0014] Step 4: Replace at least one SPPELAN structure in the original model with a Parallel Pooling Feature Modulation (PPFM) module in the neck network; the Parallel Pooling Feature Modulation (PPFM) module captures long-range dependencies through global branches and extracts local features through local branches;

[0015] Step 5: Train, test, and evaluate the improved YOLOv9s network built based on Steps 2 to 4.

[0016] Furthermore, the preprocessing operations in step one include routine preprocessing and image enhancement preprocessing. Routine preprocessing includes: scaling the input image to a fixed size; normalizing the pixel values ​​of the input image; and performing a fill operation on the input image, where the fill area is filled with grayscale values, and the specific grayscale value for the grayscale fill is 128.

[0017] The steps of image enhancement preprocessing are as follows:

[0018] The image after conventional preprocessing is processed based on the atmospheric physics model to generate preliminary correction parameters and preliminary corrected image. The implementation of the atmospheric physics model includes: combining the dark channel prior and the atmospheric scattering coefficient back estimation method to jointly estimate the atmospheric transport map, and estimating the atmospheric light vector by screening the high-brightness pixel region of the image.

[0019] The initial correction parameters are optimized and corrected using a neural network; the neural network is a convolutional neural network with channel and spatial attention mechanisms; the channel attention mechanism is used to focus on spectral channels with severe atmospheric interference, and the spatial attention mechanism is used to locate areas covered by cloud shadows or haze.

[0020] The original image, the pre-corrected image, and the optimized corrected image are adaptively fused to output a pre-processed image. The weights of the adaptive fusion are dynamically calculated based on the quantized values ​​of local contrast and edge sharpness of the image: the original image is given higher weights for areas with high contrast and sharp edges, and the corrected image is given higher weights for areas with severe atmospheric interference.

[0021] Furthermore, the neck network is configured to receive multi-scale feature maps extracted by the backbone network. The neck network includes a SPPELAN module, an upsample module, a concatenation module, a RepNCSPELAN4 module, and a switchable convolution (AConv) module. At least one concatenation module is replaced by a KSFF module, a CSMA module is added after the upsample module, and at least one SPPELAN module is replaced by a PPFM module. The KSFF module, CSMA module, and PPFM module work together in the neck network.

[0022] Furthermore, the KSFF module implements kernel selection and feature fusion in the following ways:

[0023] The input feature map is fed into multiple parallel convolutional layers with different kernel sizes to extract multi-scale features;

[0024] The obtained multi-scale features are spliced ​​and fused.

[0025] A channel attention mechanism is applied to the fused features to generate weight coefficients corresponding to each convolutional branch;

[0026] Based on the weight coefficients, the multi-scale features are weighted and fused to obtain the output feature map of the KSFF module.

[0027] Furthermore, the CSMA module achieves cross-space attention modeling through the following steps:

[0028] At the channel dimension, global average pooling is performed on the input feature map to capture global contextual information and generate channel attention weights;

[0029] In the spatial dimension, convolution operations are performed on the input feature map to capture local spatial details and generate spatial attention weights;

[0030] By using an aggregation function, channel attention weights and spatial attention weights are fused together to generate a cross-spatial attention graph;

[0031] The cross-spatial attention map is multiplied with the original input feature map to output an enhanced feature map.

[0032] Furthermore, the PPFM module achieves multi-scale context modulation through the following steps:

[0033] The input feature map is fed into a 1×1 convolutional layer to expand the number of channels;

[0034] The expanded feature map is input into both the global and local branches simultaneously.

[0035] The global branch captures long-range dependencies through global average pooling and depthwise separable convolutions;

[0036] Local branches capture local salient features through max pooling and standard convolution;

[0037] The output features of the global branch and the local branch are fused, and the number of channels is adjusted through a 1×1 convolutional layer to output a modulated feature map.

[0038] Furthermore, in step five, the improved YOLOv9s network is trained using the DIOR or SIMD remote sensing image dataset; the mean accuracy mAP@0.5 and mAP@0.5:0.95 are used as evaluation metrics for the model's detection accuracy.

[0039] Furthermore, before performing step one, the following steps are also included:

[0040] Step A: Sample generation step, using natural image datasets and unlabeled remote sensing images, to generate synthetic remote sensing training samples through a style transfer model; wherein, the style transfer model adopts a pre-trained CycleGAN model or a pre-trained AdaIN model;

[0041] Step B: Sample mixing step, the synthetic remote sensing training samples are mixed with real labeled remote sensing data to obtain mixed training data; the mixing ratio of synthetic remote sensing training samples to real labeled remote sensing data is 3:7.

[0042] Step C: Model pre-training step. A semi-supervised consistency training strategy is adopted, using mixed training data to pre-train the improved YOLOv9s model. The semi-supervised consistency training strategy includes: performing strong augmentation and weak augmentation processing on the same batch of training data to obtain strong augmentation version data and weak augmentation version data; inputting the strong augmentation version data and weak augmentation version data into the improved YOLOv9s model respectively to obtain the corresponding strong augmentation prediction results and weak augmentation prediction results; calculating the consistency loss between the strong augmentation prediction results and weak augmentation prediction results, and updating the parameters of the improved YOLOv9s model based on the consistency loss.

[0043] Furthermore, the improved detection head of the YOLOv9s network further includes a rotating box regression branch and a density map estimation branch;

[0044] The rotation box regression branch is configured to predict the orientation angle of the target. The rotation box regression branch uses the sliding window principle to predict the orientation angle, and its output format is the parameter set (cx,cy,w,h,θ) of the rotation rectangle, where cx represents the x-coordinate of the center point of the rotation rectangle, cy represents the y-coordinate of the center point of the rotation rectangle, w represents the width of the rotation rectangle, h represents the height of the rotation rectangle, and θ represents the rotation angle of the rotation rectangle.

[0045] The density map estimation branch is configured to generate a target density map of the remote sensing image to count the targets; the density map estimation branch adopts a lightweight sub-network structure, and the density map estimation loss is the mean squared error loss;

[0046] The total loss function of the improved YOLOv9s network is composed of a weighted sum of classification loss, bounding box regression loss, angle regression loss, and density map estimation loss.

[0047] Furthermore, the model training process in step five employs an optimization strategy that combines dynamic mixed precision training with knowledge distillation, specifically including:

[0048] The FP16 precision and FP32 precision are dynamically switched based on the gradient norm of the current iteration. When the gradient norm is greater than a preset threshold, FP32 precision is enabled to avoid gradient vanishing. When the gradient norm is less than or equal to a preset threshold, FP16 precision is used to accelerate the training process. The preset threshold for the gradient norm is 1e-4.

[0049] Knowledge distillation is implemented using the original YOLOv9-1 model as the teacher model and the improved YOLOv9s model as the student model. The distillation loss function includes classification probability distillation loss and feature distillation loss. The classification probability distillation loss is calculated using KL divergence. The feature distillation loss is calculated using MSE loss. Feature distillation is performed on the output features of the KSFF and PPFM modules of the neck network, and the consistency of feature distribution between the student model and the teacher model is constrained by L2 distance.

[0050] A perturbation generator is added after the output features of the backbone network, and adversarial training is performed by minimizing the detection loss and maximizing the perturbation loss; the noise includes, but is not limited to, cloud interference and sensor noise.

[0051] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0052] First, by introducing a kernel-selective feature fusion (KSFF) module and a cross-spatial multi-scale attention (CSMA) module into the YOLOv9s neck network, and replacing the SPPELAN structure with a parallel pooling feature modulation (PPFM) module, we achieved accurate fusion of multi-scale features, enhanced cross-spatial feature correlation, and effective capture of long-range dependence and local features, thereby improving the detection accuracy of various targets in remote sensing images, especially small targets, dense targets, and targets in complex backgrounds.

[0053] Secondly, conventional preprocessing and image enhancement preprocessing methods based on atmospheric physics models and neural network optimization can effectively correct atmospheric interference (such as haze and cloud shadows), and retain details of clear areas in the original image through adaptive fusion strategies. This makes the model more adaptable and robust to remote sensing images with different imaging conditions and different levels of atmospheric interference, and reduces detection errors caused by image quality issues.

[0054] Third, by generating synthetic remote sensing training samples through style transfer models and combining them with semi-supervised consistency training strategies, we can make full use of unlabeled or insufficiently labeled remote sensing data, alleviate the problem of scarce remote sensing image labeled data, reduce the dependence on a large amount of manually labeled data, and improve the model's generalization ability and detection performance of unseen targets.

[0055] Fourth, knowledge distillation using YOLOv9-1 as the teacher model improves the detection performance of the improved YOLOv9s (student model) by transferring the feature knowledge of classification probabilities and key modules (KSFF, PPFM); adversarial training enhances the model's resistance to common remote sensing image noise such as cloud interference and sensor noise, thereby improving the overall robustness of the model.

[0056] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0057] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0058] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0059] Figure 1 is a schematic diagram of the steps of a remote sensing image target detection method based on improved YOLOv9s;

[0060] Figure 2 is a schematic diagram of the method steps for kernel selection and feature fusion implemented by the KSFF module;

[0061] Figure 3 is a schematic diagram of the steps involved in implementing multi-scale context modulation using the PPFM module. Detailed Implementation

[0062] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0063] This invention provides a remote sensing image target detection method based on an improved YOLOv9s, as shown in Figure 1, comprising the following steps:

[0064] Step 1: Acquire remote sensing images and perform preprocessing operations;

[0065] Step 2: Introduce a Kernel Selective Feature Fusion (KSFF) module into the neck network of the YOLOv9s model, replacing at least one Concat structure in the original model; the Kernel Selective Feature Fusion (KSFF) module extracts multi-scale features through multiple convolutional kernels and combines channel attention weighting to achieve accurate fusion;

[0066] Step 3: Add a cross-spatial multi-scale attention (CSMA) module to the neck network; the cross-spatial multi-scale attention (CSMA) module is used to enhance the correlation of cross-spatial features to focus on the target region;

[0067] Step 4: Replace at least one SPPELAN structure in the original model with a Parallel Pooling Feature Modulation (PPFM) module in the neck network; the Parallel Pooling Feature Modulation (PPFM) module captures long-range dependencies through global branches and extracts local features through local branches;

[0068] Step 5: Train, test, and evaluate the improved YOLOv9s network built based on Steps 2 to 4.

[0069] The following examples illustrate each step: Suppose we are monitoring crop diseases and pests in a certain area and using the improved YOLOv9s model to identify diseased plants.

[0070] Step 1: Acquire remote sensing images and perform preprocessing: Use a high-resolution remote sensing camera mounted on a drone to acquire remote sensing images of crops in the area; since the actual images may have problems such as uneven lighting and noise interference, preprocessing is required.

[0071] Step 2: Introduce a Kernel Selective Feature Fusion (KSFF) module into the neck network of the YOLOv9s model, replacing at least one Concat structure in the original model. In this crop pest and disease monitoring scenario, features at different scales are crucial for identifying pest-infested plants. The KSFF module extracts multi-scale features through multiple convolutional kernels. For example, a 3×3 convolutional kernel is used to capture local details of pest-infested plants, such as the shape of lesions on leaves; a 5×5 convolutional kernel is used to obtain features over a relatively large area, such as the distribution of pests and diseases in farmland; then, combined with channel attention weighting, higher weights are given to feature channels related to pests and diseases to achieve accurate fusion. For example, in images, channels related to the color and texture of pest-infested plants will receive special attention, which can more accurately fuse important features at different scales and improve the model's ability to capture features of pest-infested plants.

[0072] Step 3: Add a Cross-Spatial Multi-Scale Attention (CSMA) module to the neck network: In large-area farmland remote sensing images, there is a lot of background information that is not related to the identification of pests and diseases. The CSMA module is used to enhance the cross-spatial feature correlation to focus on the target area. For example, in the image, even if the pests and diseases plants only occupy a small area, the model can strengthen the feature connection between the pests and diseases plants and the surrounding environment at different spatial scales through the CSMA module, so that the model pays more attention to the plant areas that may have pests and diseases, ignores large areas of normal crops and other irrelevant backgrounds, and thus more accurately locates and identifies pests and diseases plants.

[0073] Step 4: Replace at least one SPPELAN structure in the original model with a Parallel Pooling Feature Modulation (PPFM) module in the neck network: In the example of crop pest and disease monitoring, the PPFM module captures long-range dependencies through global branches, which can grasp the overall distribution of pests and diseases from the entire farmland image range, such as understanding whether pests and diseases are concentrated or scattered throughout the region; local branches extract local features to extract specific symptom features of a particular plant with pests and diseases.

[0074] Step 5: Train, test, and evaluate the improved YOLOv9s network built in Steps 2 to 4: Collect a large amount of labeled remote sensing image data of crops, including images of normal plants and plants affected by pests and diseases, and divide them into training and testing sets; use the training set to train the improved YOLOv9s network, adjust the model parameters, and allow the model to learn the characteristic patterns of plants affected by pests and diseases; for example, after multiple iterations of training, the model gradually masters the image features of different pests and diseases at different growth stages. After training, use the testing set to test the model and calculate the model's recognition accuracy, recall, and other indicators. If the model achieves a recognition accuracy of 90% and a recall of 85% for plants affected by pests and diseases in the testing set, it indicates that the model has good performance in this crop pest and disease monitoring task and can accurately identify plants affected by pests and diseases in practical applications.

[0075] The working principle of the above technical solution is as follows: First, remote sensing images are acquired by a high-resolution remote sensing camera mounted on a drone. Since the actual captured images have problems such as uneven lighting and noise interference, preprocessing is used to optimize the image quality, provide better data for subsequent model processing, and improve the model's ability to extract image features.

[0076] Secondly, in the neck network of the YOLOv9s model, at least one Concat structure is replaced with a KSFF module. Features at different scales are crucial for target detection. The KSFF module uses multiple convolutional kernels to extract multi-scale features, such as 3×3 convolutional kernels to capture local details and 5×5 convolutional kernels to obtain features over a larger range. Combined with channel attention weighting, higher weights are given to feature channels related to the target to achieve accurate fusion, enabling the model to capture target features more accurately.

[0077] Next, a CSMA module was added to the neck network. Addressing the presence of a large amount of irrelevant background information in remote sensing images, this module enhances cross-spatial feature correlation, focusing on the target region. Even if the target occupies a small portion of the image, it strengthens the feature connections between the target and its surrounding environment at different spatial scales, allowing the model to focus more on the target region and ignore irrelevant background, thus improving the accuracy of target localization and recognition. At least one SPPELAN structure was replaced in the neck network with a PPFM module. The PPFM module captures long-range dependencies through global branches, enabling a holistic understanding of target distribution; local branches extract local features, specifically extracting symptom features for individual targets, comprehensively improving the model's understanding and detection capabilities of target features.

[0078] Finally, a large amount of labeled remote sensing image data was collected and divided into training and test sets. The improved YOLOv9s network was trained using the training set, and the model parameters were adjusted to enable the model to learn the characteristic patterns of diseased and pest-infested plants. Multiple iterations of training were conducted to enable the model to master the image characteristics of different diseases and pests at different growth stages. After training, the model was tested using the test set, and the recognition accuracy, recall, and other indicators were calculated to evaluate the model performance. If the recognition accuracy reaches 90% and the recall reaches 85% in the test set, it indicates that the model performs well on this task and can accurately identify diseased and pest-infested plants in practice.

[0079] The beneficial effects of the above technical solution are as follows: By introducing the KSFF module, the solution provided in this embodiment achieves accurate fusion of multi-scale features, solving the problem that the traditional Concat structure is prone to losing key information during feature fusion; the addition of the CSMA module enhances the cross-spatial feature correlation, enabling the model to focus on the target region more accurately and reduce background interference; the PPFM module, through the parallel design of global and local branches, captures long-range dependencies and local details at the same time, improving the ability to represent targets in complex scenes.

[0080] In one embodiment, the preprocessing operation in step one includes conventional preprocessing and image enhancement preprocessing. Conventional preprocessing is as follows: scaling the input image to a fixed size of 640×640; normalizing the pixel values ​​of the input image with a normalization range of [0,1]; and performing a filling operation on the input image, wherein the filling area is filled with grayscale values, and the specific grayscale value of the grayscale filling is 128.

[0081] The steps of image enhancement preprocessing are as follows:

[0082] The image is preprocessed using an atmospheric physics model to generate preliminary correction parameters and a preliminary corrected image. The atmospheric physics model is implemented by combining dark channel priors with atmospheric scattering coefficient back-calculation methods to jointly estimate the atmospheric transmission map and estimate the atmospheric light vector by selecting high-brightness pixel regions in the image. For example, for a remote sensing image under hazy weather, the dark channel prior assumes that the local minimum value in the non-sky region approaches 0. The dark channel map is obtained by calculating the minimum value in the 5×5 neighborhood. Then, the atmospheric transmission map is back-calculated by combining the empirical value of atmospheric scattering coefficient β = 1.2. At the same time, the top 0.1% of the pixels in the image are selected and their average RGB value is taken as the atmospheric light vector (e.g., [190, 200, 210]).

[0083] The initial correction parameters are optimized using a neural network. This neural network is a convolutional neural network with channel and spatial attention mechanisms. The channel attention mechanism focuses on spectral channels with severe atmospheric interference, while the spatial attention mechanism locates areas covered by clouds or haze. For example, for an image containing three RGB channels, the channel attention module obtains the weights of each channel feature through global average pooling. If the blue channel is more severely affected by haze (stronger atmospheric scattering of blue light), it is given a higher weight (e.g., 0.6), while the weights of the red and green channels are 0.3 and 0.1, respectively. The spatial attention module concatenates the feature maps after performing max pooling and average pooling on the channel dimensions, and then generates a spatial attention map through a 7×7 convolution. The attention weight of cloud-covered areas (areas with lower gray values) is enhanced to above 0.8, while the weight of clear sky areas remains below 0.2.

[0084] The original image, the pre-corrected image, and the optimized corrected image are adaptively fused to output a preprocessed image. The weights for this adaptive fusion are dynamically calculated based on the quantized values ​​of local image contrast and edge sharpness: the local contrast threshold is 0.3, and the edge sharpness threshold is 50. Regions with high contrast and sharp edges are assigned higher weights to the original image; regions with severe atmospheric interference are assigned higher weights to the corrected image. For example, if the local contrast of a building edge region in the image is calculated to be 0.4 (above the threshold of 0.3), and the edge sharpness calculated using the Sobel operator is 80 (above the threshold of 50), then the weight for this region is 0.7 for the original image, 0.2 for the pre-corrected image, and 0.1 for the optimized corrected image. Conversely, in the image of a heavily smog-affected sky region, the local contrast is 0.2 (below the threshold of 0.3), and the edge sharpness is 30 (below the threshold of 50), so the weight for the original image is 0.1, the weight for the pre-corrected image is 0.2, and the weight for the optimized corrected image is 0.7. Finally, a weighted average is used to obtain the fused, clear image.

[0085] The working principle of the above technical solution is as follows: First, conventional preprocessing is performed, including scaling to a fixed size: To facilitate subsequent processing and model calculation, input images of different sizes are adjusted to a uniform specification to ensure that all images are operated on under the same data dimension. For example, convolutional neural networks usually require input images to have fixed length and width. Next, pixel values ​​are normalized: Mapping image pixel values ​​to a specific range (such as [0,1] or [-1,1]) can speed up model convergence, reduce the risk of gradient vanishing or exploding during model training, and make it easier for the model to learn the features in the data. Then, a filling operation (grayscale value filling) is performed: After scaling the image, the image size may not meet the requirements of subsequent processing. Through the filling operation, grayscale values ​​are used to fill the blank areas to maintain the integrity of the image structure and avoid information loss in subsequent processing due to size issues.

[0086] Next, image enhancement preprocessing is performed: Based on the atmospheric physical model, the atmospheric transmission map is jointly estimated by combining the dark channel prior and the atmospheric scattering coefficient back-calculation method. The dark channel prior utilizes the characteristic that in haze-free images, most non-sky areas have at least one color channel with a very low pixel value to estimate the atmospheric transmission map; the atmospheric scattering coefficient back-calculation method derives relevant parameters based on the principle of atmospheric scattering. The atmospheric transmission map is obtained by combining these two methods. At the same time, atmospheric light vectors are estimated by selecting high-brightness pixel areas in the image. These high-brightness pixels usually include information from areas that are greatly affected by the atmosphere, such as the sky. The atmospheric light vectors are estimated using this information. Based on these operations, preliminary correction parameters and a preliminary corrected image are generated. The preliminary corrected image is the result of preliminary correction of the original image to the atmospheric effects.

[0087] Next, a convolutional neural network with channel and spatial attention mechanisms is used to optimize the initial correction parameters. The channel attention mechanism focuses on the spectral channels with severe atmospheric interference. Since different spectral channels are affected by atmospheric interference to varying degrees, focusing can make the network pay more attention to the information of the channels with greater interference, thereby improving the correction effect. The spatial attention mechanism locates the areas covered by cloud shadows or haze, and can focus on processing specific interference areas in the image, thereby optimizing the correction parameters and further improving the correction effect.

[0088] Finally, the original image, the pre-corrected image, and the optimized image are adaptively fused. The weights are dynamically calculated based on the quantized values ​​of local contrast and edge sharpness. For regions with high contrast and sharp edges, it indicates that the region is less affected by atmospheric interference, so the original image is given a higher weight to preserve the details and features of the original image. For regions with severe atmospheric interference, the corrected image is given a higher weight to improve the quality of the region by leveraging the advantages of the corrected image. The final output is a pre-processed image that retains the effective information of the original image and enhances the correction of regions affected by atmospheric interference.

[0089] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the quality of remote sensing images can be effectively improved through a multi-step preprocessing process, laying a good data foundation for subsequent target detection tasks; the scaling, normalization, and padding operations in conventional preprocessing ensure that the model can stably receive and process input data; the image enhancement preprocessing step improves the overall visual effect and feature clarity of the image; the optimization of correction parameters by the convolutional neural network with channel and spatial attention mechanisms further improves the accuracy of correction; adaptive fusion of the original image and images at different correction stages, dynamically allocating weights based on local contrast and edge clarity, achieves an organic combination of preserving original information and enhancing interference areas.

[0090] In one embodiment, the neck network is configured to receive multi-scale feature maps extracted by the backbone network. The neck network includes a SPPELAN module, an upsample module, a concatenation module, a RepNCSPELAN4 module, and a switchable convolution (AConv) module. At least one concatenation module is replaced by a KSFF module, a CSMA module is added after the upsample module, and at least one SPPELAN module is replaced by a PPFM module. The KSFF, CSMA, and PPFM modules work collaboratively in the neck network. The KSFF module has convolutional kernel combinations of 3×3, 5×5, and 7×7, while the PPFM module uses 1×1 convolutions to expand the number of channels from 256 to 512, and the depthwise separable convolutional kernel size is 3×3.

[0091] The working principle of the above technical solution is as follows: the original SPPELAN module is replaced by the PPFM module. The PPFM module is designed for specific feature processing needs and, compared with the SPPELAN module, may have more efficient or better suited methods in terms of feature aggregation and scale processing. By replacing the module, the feature processing flow is optimized to improve the processing effect on multi-scale feature maps. The upsampling module increases the resolution of the low-resolution feature map, and the subsequently added CSMA module performs attention adjustment on the channel or spatial dimensions of the upsampled feature map to highlight key feature information, helping subsequent modules to better utilize these features. Features: Some splicing modules are replaced by the KSFF module. The splicing module generally merges feature maps of different scales or different dimensions according to a specific dimension. The KSFF module adds a more intelligent fusion strategy when merging features, such as considering the importance weight of features, to improve the quality of feature fusion. The KSFF module, CSMA module and PPFM module work together. The PPFM module preprocesses some features first, the CSMA module optimizes the upsampled features, and the KSFF module fuses features more effectively. Together, they optimize the neck network's processing of multi-scale feature maps of the backbone network, providing better feature representation for subsequent tasks such as object detection and classification.

[0092] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, through the collaborative optimization of the PPFM module, CSMA module and KSFF module, the neck network can more accurately capture the feature information of targets at different scales in remote sensing images, and improve the feature extraction capability of targets with small targets, dense targets and targets in complex backgrounds.

[0093] In one embodiment, as shown in Figure 2, the KSFF module implements kernel selection and feature fusion in the following manner:

[0094] The input feature map is fed into multiple parallel convolutional layers with different kernel sizes to extract multi-scale features;

[0095] The obtained multi-scale features are spliced ​​and fused.

[0096] A channel attention mechanism is applied to the fused features to generate weight coefficients corresponding to each convolutional branch;

[0097] Based on the weight coefficients, the multi-scale features are weighted and fused to obtain the output feature map of the KSFF module;

[0098] Assuming the input feature map is an RGB image of size 224×224×64, the KSFF module is implemented as follows: Three parallel convolutional branches are set up, using 3×3, 5×5, and 7×7 convolutional kernels respectively (each containing 64 output channels, stride 1, padding = 1, 2, 3 to maintain size), extracting three sets of 224×224×64 multi-scale feature maps F1, F2, and F3; F1, F2, and F3 are concatenated along the channel dimension to obtain a 224×224×64 image. The 4×192 concatenated feature F_concat is then applied sequentially: global average pooling (generating a 1×1×192 vector), two fully connected layers (hidden layer dimension 96, ReLU activation, output layer dimension 3), and Softmax normalization to obtain weight coefficients W1, W2, and W3 (e.g., [0.5, 0.3, 0.2]). Element-wise weighted summation is then performed on F1, F2, and F3 according to their weights to output a 224×224×64 feature map.

[0099] The working principle of the above technical solution is as follows: First, the input feature map is fed into multiple convolutional layers in parallel. Each convolutional layer has a different kernel size. Since the receptive fields of convolutional kernels of different sizes are different, small convolutional kernels focus on local details, while large convolutional kernels can capture a wider range of contextual information. In this way, the input features can be extracted from different scales, thereby enriching the expression of features.

[0100] Secondly, the feature maps of different scales obtained in the previous step are stitched together. This stitching operation allows feature information of different scales to be concentrated in one feature map, providing comprehensive feature data for subsequent processing and creating conditions for the model to utilize multi-scale information.

[0101] Then, a channel attention mechanism is applied to the spliced ​​and fused feature map. This mechanism can automatically learn the importance of each channel feature and generate weight coefficients corresponding to each convolutional branch (i.e., convolutional layer branches with different kernel sizes). The weight coefficients reflect the contribution of different channel features to the final result, and channels with high importance will receive larger weights.

[0102] Then, based on the generated weight coefficients, the multi-scale features are weighted and fused. This weighted fusion method is not a simple addition, but a weighting based on the importance of the channels, so that important features play a greater role in the fusion. Finally, the output feature map of the KSFF module is obtained. This output feature map effectively integrates multi-scale information and highlights key features.

[0103] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, features can be extracted in parallel through multi-scale convolution kernels, and the features at different scales can be dynamically weighted by combining the channel attention mechanism, which effectively balances the fusion effect of local details and global context information.

[0104] In one embodiment, the CSMA module implements cross-spatial attention modeling through the following steps:

[0105] At the channel dimension, global average pooling is performed on the input feature map to capture global contextual information and generate channel attention weights;

[0106] In the spatial dimension, convolution operations are performed on the input feature map to capture local spatial details and generate spatial attention weights;

[0107] By using an aggregation function, channel attention weights and spatial attention weights are fused together to generate a cross-spatial attention graph;

[0108] The cross-spatial attention map is multiplied with the original input feature map to output an enhanced feature map.

[0109] The working principle of the above technical solution is as follows: Global average pooling is a statistical method. First, the input feature map is globally average pooled in the channel dimension, which is equivalent to aggregating all spatial location information on each channel into a single value. This allows the model to obtain the global context information of each channel in the entire feature map, understand the importance of each channel in the whole, and then generate channel attention weights. These weights reflect the contribution of different channels to the global information expression.

[0110] Next, convolution operations are performed in the spatial dimension. The convolution kernel slides on the feature map, and each convolution operation only focuses on a local region. In this way, local spatial details in the input feature map can be captured, such as local textures and edges in the image. Then, spatial attention weights are generated, which represent the importance of different spatial locations.

[0111] Next, the channel attention weights and spatial attention weights are fused using an aggregation function. The aggregation function takes into account both types of weights, so that the fused weights can reflect the spatial local importance, thereby generating a cross-spatial attention map. This cross-spatial attention map comprehensively reflects the distribution of important information in the feature map in both channel and spatial dimensions.

[0112] Next, the cross-space attention map is multiplied with the original input feature map. During the multiplication process, the features of the corresponding positions in the original feature map will be enhanced at positions with high weights in the cross-space attention map, while the features of the corresponding positions in the original feature map will be weakened at positions with low weights. This results in an enhanced feature map that highlights important features and suppresses unimportant features, thereby improving the model's ability to capture and express important information and improving model performance.

[0113] The beneficial effects of the above technical solution are as follows: By using the solution provided in this embodiment, channel attention weights are generated through global average pooling, which can effectively aggregate global context information of channels and clarify the contribution of each channel to the expression of global information; spatial attention weights are generated by performing convolution operations in the spatial dimension, which can accurately capture local spatial details, such as texture and edge information; cross-spatial attention map generated by fusing the two weights with the aggregation function reflects both the global importance of channels and the local spatial importance. Multiplying it with the original input feature map can enhance important features and suppress unimportant features, improve the model's ability to capture and express important information, effectively improve the accuracy and efficiency of remote sensing image target detection, and make the model perform better in remote sensing image detection tasks with complex backgrounds and multi-scale targets.

[0114] In one embodiment, as shown in Figure 3, the PPFM module implements multi-scale context modulation through the following steps:

[0115] The input feature map is fed into a 1×1 convolutional layer to expand the number of channels;

[0116] The expanded feature map is input into both the global and local branches simultaneously.

[0117] The global branch captures long-range dependencies through global average pooling and depthwise separable convolutions;

[0118] Local branches capture local salient features through max pooling and standard convolution;

[0119] The output features of the global branch and the local branch are fused, and the number of channels is adjusted through a 1×1 convolutional layer to output a modulated feature map.

[0120] The working principle of the above technical solution is as follows: The PPFM module realizes multi-scale context modulation. First, the input feature map passes through a 1×1 convolutional layer. The role of the 1×1 convolutional layer is to expand the number of channels by linearly combining the channel dimensions through matrix operations without changing the size of the feature map. This step can increase the expressive power of the features, allowing the model to learn more different types of feature information. The feature map after expanding the channels is simultaneously fed to the global branch and the local branch. This branch structure design allows the module to obtain context information from different scales in parallel.

[0121] Next, in the global branch, global average pooling is first used. Global average pooling averages the values ​​at all spatial locations of each channel to obtain a global feature vector that only contains channel-dimensional information. This step allows the model to consider the entire feature map as a whole, ignoring spatial details, thereby capturing long-range dependencies. For example, in image classification tasks, global average pooling can obtain the overall feature representation of the image, not limited to local details, to determine which category the image belongs to. After global average pooling, depthwise separable convolution is used. Depthwise separable convolution consists of depthwise convolution and pointwise convolution. Depthwise convolution performs convolution operations on each channel separately, considering only spatial dimension feature extraction. Pointwise convolution uses 1×1 convolution to linearly combine the channels output by depthwise convolution, adjusting the number of channels. Compared with standard convolution, this convolution method reduces computation while further mining important features in long-range dependencies, refining the global features.

[0122] Next, the local branch first uses max pooling. Max pooling selects the maximum value within a given window, which can highlight the salient features of the local region. For example, in an image, it can preserve salient features such as edges and corners, while reducing the spatial size of the feature map and reducing computational cost. After max pooling, standard convolution is applied. Standard convolution performs convolution operations by sliding the convolution kernel on the feature map. It can further extract local spatial features while preserving salient local features, explore the relationships between different local features, and generate representative local feature representations.

[0123] Finally, the features output from the global and local branches are fused. This fusion operation combines global long-range dependencies with local salient features, enabling the model to consider contextual information at different scales simultaneously. Then, the number of channels is adjusted through a 1×1 convolutional layer. On the one hand, the fused features are linearly transformed to adapt to the feature dimension requirements of subsequent tasks. On the other hand, feature information can be further integrated to output a modulated feature map. This feature map contains multi-scale context-modulated information, providing richer and more representative features for subsequent tasks (such as object detection, semantic segmentation, etc.).

[0124] The beneficial effects of the above technical solution are as follows: By adopting the solution provided in this embodiment, the global long-range dependency relationship and local salient features can be effectively integrated through the multi-branch structure design of the PPFM module, so that when the model processes remote sensing images, it can not only capture the overall distribution pattern of the target, but also accurately locate local detailed features.

[0125] In one embodiment, in step five, the improved YOLOv9s network is trained using a DIOR or SIMD remote sensing image dataset; the mean accuracy mAP@0.5 and mAP@0.5:0.95 are used as evaluation metrics for the model's detection accuracy.

[0126] The working principle of the above technical solution is as follows: DIOR (Digital Sensing Image Target Detection Dataset) or SIMD (Small Scale Infrared and Visible Target Detection) remote sensing image datasets provide rich samples for model training. These datasets contain remote sensing images of various scenes and resolutions, and the target objects in the images have been labeled. The labeling information clarifies the key information such as the category and location of the target in the image. For example, the DIOR dataset may cover a variety of typical ground targets, such as buildings, roads, and vehicles. Through a large amount of such sample data, the improved YOLOv9s network can learn the feature patterns of different targets in remote sensing images, such as the appearance, shape, and relationship with the surrounding environment of the target, thereby improving the ability to detect targets in remote sensing images.

[0127] The improved YOLOv9s has been optimized in terms of network structure, feature extraction method, and loss function. For example, the backbone network structure has been optimized to make it more suitable for handling the characteristics of large size and complex scenes of remote sensing images, and can more effectively extract deep features in the images. By training the improved network using these datasets, the network can use the rich information in the dataset to adjust its own parameters to better adapt to the target detection task of remote sensing images.

[0128] Mean accuracy (mAP) is a metric for comprehensively evaluating the performance of object detection models. mAP@0.5 indicates the accuracy in the intersection-over-union (IoU) ratio. mAP@0.5 is the mAP value calculated when the IoU (Intersection over Union) threshold is 0.5. IoU measures the overlap between the model's predicted bounding box and the ground truth bounding box. IoU = (area of ​​intersection between predicted and ground truth boxes) / (area of ​​union between predicted and ground truth boxes). When IoU is greater than or equal to 0.5, the prediction is considered a correct detection. mAP@0.5 is obtained by averaging the average precision (AP) of all classes at this IoU threshold. AP is calculated based on the area under the recall and precision curves. Recall measures the proportion of ground truth objects that the model can correctly detect, while precision measures the proportion of samples predicted as positive by the model that are actually positive. The mAP@0.5 metric reflects the overall detection accuracy of the model for different classes of objects under a relatively lenient IoU threshold. In practical applications, when the accuracy requirements for detection are not extremely strict and a certain degree of bounding box deviation is allowed, mAP@0.5 can quickly evaluate the model's performance in common situations.

[0129] mAP@0.5:0.95 calculates mAP values ​​at different IoU thresholds (from 0.5 to 0.95, with a step size of 0.05) and averages these mAP values. This metric considers different levels of IoU strictness and provides a more comprehensive evaluation of the model's performance. In real-world scenarios, different applications have different requirements for detection accuracy. Some applications may have extremely high requirements for bounding box accuracy, while others can tolerate a certain degree of deviation. mAP@0.5:0.95 can comprehensively reflect the model's performance at different levels of IoU strictness and provide a more detailed evaluation of the model's robustness and accuracy in object detection.

[0130] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, the diverse remote sensing image samples contained in the DIOR or SIMD dataset can provide sufficient and representative training data support for the improved YOLOv9s network, enabling the network to fully learn the feature patterns of different targets in remote sensing images.

[0131] In one embodiment, before performing step one, the following steps are also included:

[0132] Step A: Sample generation step, using natural image datasets and unlabeled remote sensing images, to generate synthetic remote sensing training samples through a style transfer model; wherein, the style transfer model adopts a pre-trained CycleGAN model or a pre-trained AdaIN model;

[0133] Step B: Sample mixing step, the synthetic remote sensing training samples are mixed with real labeled remote sensing data to obtain mixed training data; the mixing ratio of synthetic remote sensing training samples to real labeled remote sensing data is 3:7.

[0134] Step C: Model pre-training step. A semi-supervised consistency training strategy is adopted, using mixed training data to pre-train the improved YOLOv9s model. The semi-supervised consistency training strategy includes: performing strong enhancement and weak enhancement processing on the same batch of training data to obtain strong enhancement version data and weak enhancement version data. Strong enhancement is: a random cropping ratio of 20% and a color jitter amplitude of ±15%. Weak enhancement is: a rotation angle range of ±5°. The strong enhancement version data and weak enhancement version data are respectively input into the improved YOLOv9s model to obtain the corresponding strong enhancement prediction results and weak enhancement prediction results. The consistency loss between the strong enhancement prediction results and weak enhancement prediction results is calculated, and the parameters of the improved YOLOv9s model are updated based on the consistency loss.

[0135] In practical applications, assume there are 1000 unlabeled high-resolution natural landscape images (containing scenes such as mountains, rivers, and buildings) and 500 labeled real remote sensing images (labeled as buildings, roads, and water bodies). In the sample generation step, a pre-trained CycleGAN model is first used as a style transfer model to transfer the content of the natural landscape images to the style of the remote sensing images. For example, given a natural landscape image containing a river, the CycleGAN model learns the texture and tone (such as dark blue and smooth texture) of the water in the real remote sensing image to generate a synthetic river remote sensing image with a visual style consistent with the real remote sensing image, generating a total of 300 synthetic remote sensing training samples to ensure that they cover the same category distribution as the real labeled data. In the sample mixing step, the 300 synthetic remote sensing training samples are mixed with 700 real labeled remote sensing data at a mixing ratio of 3:7, resulting in a total of 1000 mixed training data images. In the model pre-training step, a batch of data (e.g., 64 images) is randomly selected from the mixed training data, and each image is augmented separately. Strong and weak enhancement methods are used. The image is randomly cropped by 20% (e.g., a 205×205 region is randomly cropped from a 1024×1024 image and resized back to its original size). Simultaneously, ±15% color dithering is applied (e.g., brightness increases by 15%, contrast decreases by 10%), and the image is randomly rotated within ±5° (e.g., 3° ​​clockwise). The strongly enhanced and weakly enhanced versions of the data are then input into the improved YOLOv9s model to obtain the strongly enhanced prediction result (e.g., building bounding box coordinates, confidence score 0.8). 5) Compare the weak augmentation prediction results (e.g., the bounding box coordinates of the same building, confidence level 0.92); calculate the consistency loss between the two prediction results (e.g., using CIoU loss to measure the difference in bounding boxes, combined with the KL divergence of classification probabilities). If the loss value is 0.12 (above the threshold), then update the convolutional layer weights and bias parameters of the improved YOLOv9s model through backpropagation to make the model more consistent with the input and output of different augmentations; repeat the above process until the model's mAP (mean accuracy) on the validation set no longer improves, thus completing the pre-training.

[0136] The working principle of the above technical solution is as follows: Natural image datasets usually have rich visual features and diversity, while unlabeled remote sensing images contain specific remote sensing scene information. The role of the style transfer model is to transfer the style features of natural images to unlabeled remote sensing images to generate synthetic remote sensing training samples.

[0137] CycleGAN is an unsupervised image-to-image translation model based on Generative Adversarial Networks (GANs). It can learn the mapping relationship between two image domains, enabling the generated image to retain content information while possessing the style of the target domain. The pre-trained CycleGAN model has already learned a certain image style transfer capability, and here it is used to transfer the style of natural images to remote sensing images.

[0138] The AdaIN (Adaptive Instance Normalization) model is also used for style transfer. It achieves the separation and fusion of content and style by replacing the instance normalization statistics of the content image with the instance normalization statistics of the style image, thereby generating an image with a specific style. Using the pre-trained AdaIN model, the style of natural images can also be transferred to unlabeled remote sensing images to generate synthetic remote sensing training samples.

[0139] Although synthetic remote sensing training samples are generated through style transfer, they may differ from real remote sensing scenes. Real labeled remote sensing data has accurate annotation information and reflects the actual remote sensing target situation. Mixing synthetic remote sensing training samples with real labeled remote sensing data can combine the advantages of both. Synthetic samples increase the diversity of data, while real labeled data provides accurate annotation references. The resulting mixed training data can better train the model, enabling the model to learn more comprehensive remote sensing image features and annotation information.

[0140] This invention employs a semi-supervised consistency training strategy for model pre-training. First, the same batch of training data undergoes both strong and weak enhancement processing. Strong enhancement involves drastic transformations such as large rotations, cropping, and color changes, aiming to teach the model more robust and invariant features. Weak enhancement involves relatively minor transformations, such as small-angle rotations and slight brightness adjustments, preserving the main features of the image. This results in strongly and weakly enhanced versions of the data. These versions are then input into an improved YOLOv9s model. Due to the different degrees of transformation in the input data, the model... The model outputs corresponding strong and weak enhancement prediction results. Then, the consistency loss between the strong and weak enhancement prediction results is calculated. The consistency loss measures the consistency of the prediction results under different enhancement levels. If the prediction results are inconsistent under different enhancement levels, it indicates that the model is more sensitive to image transformations and its generalization ability may be insufficient. Next, the parameters of the improved YOLOv9s model are updated based on the consistency loss. Through the backpropagation algorithm, the model parameters are updated in the direction of reducing the consistency loss, so that the model can output relatively consistent prediction results under different enhancement levels, thereby improving the model's generalization ability and stability.

[0141] The beneficial effects of the above technical solution are as follows: by adopting the solution provided in this embodiment, through the semi-supervised consistency training strategy, it is possible to fully mine the feature information contained in the unlabeled data while utilizing a small amount of labeled data, effectively reducing the model's dependence on large-scale labeled data.

[0142] In one embodiment, the detection head of the improved YOLOv9s network further includes a rotating box regression branch and a density map estimation branch;

[0143] The rotation box regression branch is configured to predict the orientation angle of the target. The rotation box regression branch uses the sliding window principle to predict the orientation angle, and its output format is the parameter set (cx,cy,w,h,θ) of the rotation rectangle, where cx represents the x-coordinate of the center point of the rotation rectangle, cy represents the y-coordinate of the center point of the rotation rectangle, w represents the width of the rotation rectangle, h represents the height of the rotation rectangle, and θ represents the rotation angle of the rotation rectangle.

[0144] The density map estimation branch is configured to generate a target density map of the remote sensing image to count the targets; the density map estimation branch adopts a lightweight sub-network structure, and the density map estimation loss is the mean squared error loss;

[0145] The total loss function of the improved YOLOv9s network is composed of a weighted sum of classification loss, bounding box regression loss, angle regression loss, and density map estimation loss.

[0146] The working principle of the above technical solution is as follows: The rotation box regression branch predicts the orientation angle of the target. In practical application scenarios, such as remote sensing image detection, the target object may appear at different angles. Accurately predicting its orientation angle is crucial for accurate identification and localization. Mechanical detection is carried out using the sliding window principle. The sliding window slides on the image with a fixed step size. The image features in each window are used to predict the parameters of the rotation rectangle. This allows the target to be detected at different positions, thereby covering the entire image. The output is a set of parameters of the rotation rectangle ((cx,cy,w,h,θ)), where (cx) and (cy) determine the position of the center point of the rotation rectangle in the image coordinate system, (w) and (h) represent the width and height of the rotation rectangle, respectively, and (θ) represents the angle of rotation of the rotation rectangle around the center point. This set of parameters can accurately depict the position and orientation of the target object on the image.

[0147] The main task of the density map estimation branch is to generate a target density map of the remote sensing image to count the targets. In scenarios where it is necessary to count the number of targets, such as counting vehicles, ships, and other objects in a specific area, the density map estimation branch can infer the distribution of the number of targets based on the pixel values ​​of the generated density map. Specifically, it adopts a lightweight sub-network structure, which can effectively generate the density map while ensuring computational efficiency. The lightweight structure helps to reduce the consumption of computing resources. The density map estimation loss uses mean squared error loss, which measures the difference between the predicted density map and the true density map. By minimizing this loss, the predicted density map is made as close as possible to the true situation, thereby improving the accuracy of target counting.

[0148] The improved YOLOv9s network's total loss function is composed of a weighted average of classification loss, bounding box regression loss, angle regression loss, and density map estimation loss. Classification loss measures the model's accuracy in classifying target objects, ensuring the model can correctly identify different categories. Excessive classification loss indicates a significant bias in classification, requiring parameter adjustments to improve accuracy. Bounding box regression loss helps the model accurately predict the bounding box position and size of target objects; a smaller loss means the predicted bounding box is closer to the true bounding box, improving target detection accuracy. Angle regression loss measures the predicted orientation angle for the rotated bounding box regression branch, balancing the difference between the predicted and true angles to ensure accurate target orientation prediction. Density map estimation loss, through mean squared error loss, makes the generated density map closer to reality, achieving accurate target counting.

[0149] Weighting these different losses is to balance the importance of different tasks during training. Different application scenarios may have different emphases on classification, localization, angle prediction and target counting. By adjusting the weights, the model can achieve better performance on each task, thereby improving the detection effect of the YOLOv9s network as a whole.

[0150] The beneficial effects of the above technical solution are as follows: By adding a rotation box regression branch and a density map estimation branch to the detection head of the improved YOLOv9s network, the solution provided in this embodiment can effectively cope with the characteristics of multi-angle distribution of targets in remote sensing images and improve the accuracy of target positioning.

[0151] In one embodiment, the model training process in step five employs an optimization strategy that combines dynamic mixed precision training with knowledge distillation, specifically including:

[0152] The FP16 precision and FP32 precision are dynamically switched based on the gradient norm of the current iteration. When the gradient norm is greater than a preset threshold, FP32 precision is enabled to avoid gradient vanishing. When the gradient norm is less than or equal to a preset threshold, FP16 precision is used to accelerate the training process. The preset threshold for the gradient norm is 1e-4.

[0153] Knowledge distillation was implemented using the original YOLOv9-1 model as the teacher model and the improved YOLOv9s model as the student model. The distillation loss function included classification probability distillation loss and feature distillation loss. The classification probability distillation loss was calculated using KL divergence. The temperature coefficient for classification probability distillation was 10. The feature distillation loss was calculated using MSE loss. Feature distillation was performed on the output features of the KSFF and PPFM modules of the neck network, and the consistency of feature distribution between the student model and the teacher model was constrained by L2 distance.

[0154] A perturbation generator is added after the output features of the backbone network, and adversarial training is performed by minimizing the detection loss and maximizing the perturbation loss; the noise includes, but is not limited to, cloud interference and sensor noise.

[0155] The working principle of the above technical solutions is as follows: In deep learning training, data precision affects training speed and memory usage, and also relates to the stability of gradient calculation; FP16 (half-precision floating-point number) has the advantages of small memory usage and fast calculation speed, but in some cases, such as when the gradient norm is large, it may cause gradient vanishing due to insufficient precision, affecting model convergence; FP32 (single-precision floating-point number) has high calculation precision, but it has a large memory usage and relatively slow calculation speed.

[0156] An optimization strategy combining dynamic mixed precision training and knowledge distillation is adopted to dynamically switch precision based on the gradient norm of the current iteration. When the gradient norm is greater than a preset threshold, FP32 precision is enabled to avoid gradient vanishing and ensure the stability of model training by leveraging its high precision characteristics. When the gradient norm is less than or equal to the preset threshold, FP16 precision is used to accelerate the training process by leveraging its speed advantage. This improves training efficiency while ensuring the model training effect.

[0157] Knowledge distillation is the process of transferring knowledge learned by the teacher model to the student model. Typically, the teacher model is larger and performs better, while the student model is relatively smaller and lighter. This invention uses the original YOLOv9-1 model as the teacher model and the improved YOLOv9s as the student model. The distillation loss function consists of classification probability distillation loss and feature distillation loss. The classification probability distillation loss is calculated using Kullback-Leibler divergence, which measures the difference between two probability distributions. In knowledge distillation, it measures the difference in classification probability distributions between the student model and the teacher model, prompting the student model to learn the classification knowledge from the teacher model. The feature distillation loss is calculated using MSE (mean squared error) loss, targeting the output features of the KSFF and PPFM modules in the neck network. L2 distance is used to constrain the consistency of feature distributions between the student model and the teacher model. This ensures that the student model not only learns the classification results of the teacher model but also learns the knowledge of the teacher model's intermediate layer feature representations, thus improving the performance of the student model.

[0158] A perturbation generator is added after the backbone network outputs features, and adversarial training is employed. The adversarial training process involves minimizing the detection loss and maximizing the perturbation loss. The detection loss is used to optimize the model's target detection performance, while the perturbation loss is generated by adding noise (such as cloud interference and sensor noise) through the perturbation generator. Through this adversarial training, the model can adapt to various noise interferences while learning to detect targets, enhancing the model's robustness to noise and improving its generalization ability in complex real-world environments.

[0159] The beneficial effects of the above technical solution are as follows: the solution provided in this embodiment can significantly improve training efficiency while ensuring the stability of model training. Dynamic mixed precision training realizes intelligent switching of precision through real-time monitoring of gradient norm, which not only gives full play to the advantages of FP16 in accelerating training and saving memory, but also ensures the accuracy of gradient calculation when gradient fluctuation is large by using FP32, thus balancing training speed and numerical stability.

[0160] To verify the effectiveness of this invention, comparative experiments were conducted using the DIOR dataset and the SIMD dataset. The results included performance comparison data of the model on the DIOR dataset, data on the impact of modules on model performance, and performance comparison data of the object detection model, as shown in Figures 1, Tables 2 and 3. The DIOR dataset includes 20 remote sensing target categories (e.g., aircraft, bridges, ships), with a total of 23,463 images and a resolution of 800×800 pixels. The SIMD dataset includes 15 categories, with a total of 5,000 images and a resolution of 1024×768 pixels. Both datasets have a uniform resolution of 640×640 and are divided into training, validation, and test sets in a 7:1:2 ratio. The training parameters were: initial learning rate of 0.01 (cosine annealing decay), batch size of 16, training epochs of 100, and the optimizer AdamW (weight decay of 0.0005).

[0161]

[0162] Table 1: Performance Comparison Data of Each Model on the DIOR Dataset

[0163] As shown in Table 1, the mAP@0.5 of the model of this invention is 4.4 percentage points higher than that of the original YOLOv9s, and the mAP@0.5:0.95 is 5.3 percentage points higher, proving that the improved scheme effectively improves the detection accuracy. At the same time, GFLOPS is reduced by 2.6% compared with the original model, and the inference speed is only reduced by 2 FPS, which is better than existing models such as Faster R-CNN and YOLOv8s, and meets the requirements of real-time remote sensing detection.

[0164] Table 2: Impact data of modules on model performance (based on the original YOLOv9s dataset as the baseline)

[0165] As shown in Table 2, when KSFF, CSMA, and PPFM modules are used individually, they can all improve the mAP@0.5 of the baseline model by 1.3 to 1.8 percentage points, proving the effectiveness of each module acting alone. When the three are used together, the mAP@0.5 is improved by 4.4 percentage points, and the mAP@0.5 of small targets is improved by 7.4 percentage points, which is much higher than the improvement of a single module. This proves the synergistic effect of the three modules and solves the defect of insufficient utilization of multi-scale features in existing neck networks.

[0166]

[0167] Table 3: Performance Comparison Data of Object Detection Models (Based on SIMD Dataset)

[0168] As shown in Table 3, the improved YOLOv9s outperforms the original YOLOv9s in mAP@0.5 for small, medium and large target detection. The improvement in small target detection performance is the most significant, reaching 7.4 percentage points. This solves the core problem of blurred features and easy missed detection of small targets in remote sensing images, and further verifies the practicality of the model in complex remote sensing scenarios.

[0169] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A remote sensing image target detection method based on an improved YOLOv9s, characterized in that, Includes the following steps: Step 1: Acquire remote sensing images and perform preprocessing operations; Step 2: Introduce a kernel selection feature fusion module into the neck network of the YOLOv9s model, replacing at least one Concat structure in the original model; The kernel selection feature fusion module extracts multi-scale features through multiple convolutional kernels and combines them with channel attention weighting to achieve accurate fusion; Step 3: Add a cross-spatial multi-scale attention module to the neck network; A cross-spatial multi-scale attention module is used to enhance cross-spatial feature correlation to focus on the target region; Step 4: Replace at least one SPPELAN structure in the original model with a parallel pooling feature modulation module in the neck network; The parallel pooling feature modulation module captures long-range dependencies through global branches and extracts local features through local branches; Step 5: Train, test, and evaluate the improved YOLOv9s network built based on Steps 2 to 4; The neck network is configured to receive multi-scale feature maps extracted by the backbone network, and the neck network includes a SPPELAN module, an upsampling module, a splicing module, and a RepNCSPELAN4 module. The network consists of blocks and switchable convolutional modules; at least one concatenation module is replaced by a kernel-selective feature fusion module, a cross-spatial multi-scale attention module is added after the upsampling module, and at least one SPPELAN module is replaced by a parallel pooling feature modulation module; the kernel-selective feature fusion module, the cross-spatial multi-scale attention module, and the parallel pooling feature modulation module work collaboratively in the neck network; the kernel-selective feature fusion module achieves kernel selection and feature fusion by: inputting the input feature map into multiple parallel convolutional layers with different kernel sizes to extract multi-scale features; concatenating and fusing the obtained multi-scale features; and so on. The fused features are subjected to a channel attention mechanism to generate weight coefficients corresponding to each convolutional branch. Based on these weight coefficients, multi-scale features are weighted and fused to obtain the output feature map of the kernel selection feature fusion module. The cross-spatial multi-scale attention module implements cross-spatial attention modeling through the following steps: In the channel dimension, global average pooling is performed on the input feature map to capture global contextual information and generate channel attention weights; in the spatial dimension, convolution operations are performed on the input feature map to capture local spatial details and generate spatial attention weights; the channel attention weights and spatial attention weights are fused through an aggregation function. The system generates a cross-spatial attention map; multiplies the cross-spatial attention map with the original input feature map to output an enhanced feature map; the parallel pooling feature modulation module implements multi-scale context modulation through the following steps: inputting the input feature map into a 1×1 convolutional layer to expand the number of channels; inputting the expanded feature map into both the global branch and the local branch; capturing long-range dependencies in the global branch through global average pooling and depthwise separable convolution; capturing local salient features in the local branch through max pooling and standard convolution; fusing the output features of the global branch and the local branch, adjusting the number of channels through a 1×1 convolutional layer, and outputting the modulated feature map.

2. The remote sensing image target detection method based on improved YOLOv9s according to claim 1, characterized in that, Step 1's preprocessing operations include routine preprocessing and image enhancement preprocessing. Routine preprocessing involves: scaling the input image to a fixed size; normalizing the pixel values ​​of the input image; and performing a fill operation on the input image, where the fill area is filled with grayscale values, specifically grayscale values ​​of 128. The image enhancement preprocessing steps involve: processing the image after routine preprocessing based on an atmospheric physics model to generate preliminary correction parameters and a preliminary corrected image. The implementation of the atmospheric physics model includes: jointly estimating the atmospheric transport map by combining dark channel priors and atmospheric scattering coefficient back-calculation methods, and estimating atmospheric transport by selecting high-brightness pixel regions in the image. The light vector is used to optimize the initial correction parameters using a neural network. This neural network is a convolutional neural network with channel and spatial attention mechanisms. The channel attention mechanism focuses on spectral channels with severe atmospheric interference, while the spatial attention mechanism locates areas covered by clouds or haze. The original image, the initially corrected image, and the optimized image are adaptively fused to output a preprocessed image. The weights of the adaptive fusion are dynamically calculated based on the quantized values ​​of local image contrast and edge sharpness: higher weights are assigned to the original image for areas with high contrast and sharp edges, and higher weights are assigned to the corrected image for areas with severe atmospheric interference.

3. The remote sensing image target detection method based on improved YOLOv9s according to claim 1, characterized in that, In step five, the improved YOLOv9s network is trained using DIOR or SIMD remote sensing image datasets; The mean accuracy mAP@0.5 and mAP@0.5:0.95 were used as evaluation indicators for model detection accuracy.

4. The remote sensing image target detection method based on improved YOLOv9s according to claim 1, characterized in that, Before performing step one, the following steps are also included: Step A: Sample generation step, using natural image datasets and unlabeled remote sensing images, synthetic remote sensing training samples are generated through a style transfer model; wherein, the style transfer model adopts a pre-trained CycleGAN model or a pre-trained AdaIN model; Step B: Sample mixing step, the synthetic remote sensing training samples are mixed with real labeled remote sensing data to obtain mixed training data; the mixing ratio of synthetic remote sensing training samples to real labeled remote sensing data is 3:7; Step C: Model pre-training step, a semi-supervised consistency training strategy is adopted, and the improved YOLOv9s model is pre-trained using the mixed training data; wherein, the semi-supervised consistency training strategy includes: performing strong enhancement processing and weak enhancement processing on the same batch of training data respectively to obtain strong enhancement version data and weak enhancement version data; inputting the strong enhancement version data and weak enhancement version data into the improved YOLOv9s model respectively to obtain the corresponding strong enhancement prediction results and weak enhancement prediction results; calculating the consistency loss between the strong enhancement prediction results and weak enhancement prediction results, and updating the parameters of the improved YOLOv9s model based on the consistency loss.

5. The remote sensing image target detection method based on improved YOLOv9s according to claim 1, characterized in that, The improved YOLOv9s network's detection head further includes a rotation box regression branch and a density map estimation branch. The rotation box regression branch is configured to predict the target's orientation angle. It employs a sliding window principle to predict the orientation angle, and its output format is a set of parameters (cx, cy, w, h, θ) of the rotation rectangle, where cx represents the x-coordinate of the center point, cy represents the y-coordinate, w represents the width, h represents the height, and θ represents the rotation angle. The density map estimation branch is configured to generate a target density map of the remote sensing image to count targets. It uses a lightweight sub-network structure, and the density map estimation loss is a mean squared error loss. The total loss function of the improved YOLOv9s network is a weighted sum of the classification loss, bounding box regression loss, angle regression loss, and density map estimation loss.

6. The remote sensing image target detection method based on improved YOLOv9s according to claim 1, characterized in that, Step 5's model training process employs an optimization strategy combining dynamic mixed precision training and knowledge distillation. Specifically, this includes: dynamically switching between FP16 and FP32 precision based on the gradient norm of the current iteration; enabling FP32 precision to avoid gradient vanishing when the gradient norm exceeds a preset threshold; using FP16 precision to accelerate training when the gradient norm is less than or equal to a preset threshold (1e-4); implementing knowledge distillation using the original YOLOv9-1 model as the teacher model and the improved YOLOv9s model as the student model; the distillation loss function includes classification probability distillation loss and feature distillation loss; the classification probability distillation loss is calculated using KL divergence; the feature distillation loss is calculated using MSE loss; feature distillation is performed on the output features of the KSFF and PPFM modules of the neck network, and L2 distance is used to constrain the consistency of feature distribution between the student and teacher models; a perturbation generator is added after the backbone network output features, and adversarial training is conducted by minimizing detection loss and maximizing perturbation loss; noise includes, but is not limited to, cloud interference and sensor noise.

Images