Marine small target detection method and system based on low-altitude remote sensing image enhancement

Abstract

The invention provides an offshore small target detection method and system based on low-altitude remote sensing image enhancement, and relates to the technical field of target detection. Obtaining spectrum features of the to-be-detected low-altitude remote sensing image, and quantifying a rain potential feature value and a haze feature value based on a shielding mask; comparing the rain vigor characteristic value with a rain vigor threshold value, and if the rain vigor characteristic value is greater than the rain vigor threshold value, triggering rain removal processing: through a multi-scale Transform network, in combination with implicit neural representation, learning rain degradation characteristics and carrying out cross-scale characteristic interaction, and removing rain stripe interference to obtain a rain removal image; comparing the haze feature value with a haze threshold value, and if the haze feature value is greater than the haze threshold value, triggering defogging processing: filtering low-frequency haze information through a histogram self-attention mechanism and retaining high-frequency image details to obtain a defogged image; and inputting the processed image into a small target detection model, and identifying and obtaining an offshore tiny target detection result, thereby effectively solving the problem of insufficient detection robustness caused by low-altitude remote sensing image degradation.

Description

A Method and System for Small Target Detection at Sea Based on Low-Altitude Remote Sensing Image Enhancement Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a method and system for detecting small maritime targets based on low-altitude remote sensing image enhancement. Background Technology

[0002] Low-altitude remote sensing platforms (represented by unmanned aerial vehicles) are the core technological carriers for tasks such as maritime monitoring and environmental surveys. They acquire remote sensing images of sea surface areas through airborne imaging equipment and then rely on target detection algorithms to identify small maritime targets such as buoys, small vessels, and personnel on the water. These targets typically account for less than 1% of the pixels in the images, but they directly affect the efficiency of tasks such as maritime safety and emergency rescue. With the large-scale deployment of low-altitude platforms, the technical requirement for rapid response and accurate identification of small maritime targets has become a core bottleneck in the implementation of current maritime intelligent monitoring systems.

[0003] To address this need, academia and industry have developed two main technological approaches: one is to improve image quality in harsh environments through image enhancement algorithms, mitigating texture loss caused by rain, fog, and sea surface reflections; the other is to develop lightweight small target detection models (such as the RemDet model optimized for UAV scenarios) to balance detection accuracy and computational efficiency. However, the complex degradation of maritime scenes and the characteristics of extremely small targets mean that the actual performance of existing technologies still falls significantly short of mission requirements.

[0004] The shortcomings of existing solutions are as follows: traditional image enhancement methods struggle to overcome the coupling degradation caused by rain / fog and sea surface reflections; after a single de-raining or de-fogging operation, the recovery of image contrast and details is limited, directly restricting the quality of feature extraction for subsequent detection; simultaneously, mainstream small target detection models are not optimized for the obfuscation characteristics of the sea surface background and tiny targets, making extremely small targets easily interfered with by wave textures, resulting in large localization errors; furthermore, there is a conflict between the computational cost of high-precision detection models and the real-time requirements of low-altitude platforms, while lightweight models sacrifice detection robustness under adverse weather conditions. The combination of these problems makes it difficult for current systems to stably support small target detection tasks under complex sea conditions. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a method and system for detecting small targets at sea based on low-altitude remote sensing image enhancement. By quantizing spectral features, it achieves accurate identification of rain and fog interference. Combined with targeted image enhancement and optimized detection models, it balances real-time performance and accuracy in small target detection under complex sea conditions, effectively solving the problem of insufficient robustness in detection caused by low-altitude remote sensing image degradation.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, the present invention provides a method for detecting small maritime targets based on low-altitude remote sensing image enhancement, comprising: acquiring the spectral features of the low-altitude remote sensing image to be detected; quantizing rainfall feature values ​​and haze feature values ​​based on a masking layer; comparing the rainfall feature values ​​with a rainfall threshold; if the rainfall feature value is greater than the rainfall threshold, triggering deraining processing; the deraining processing involves learning rain degradation features and performing cross-scale feature interaction through a multi-scale Transformer network combined with implicit neural representation to remove rain streak interference and obtain a derained image; specifically, downsampling the rain-containing image to obtain multi-scale features, which are then input into the encoders corresponding to the Transformer-U-Net branches; establishing a bidirectional information flow between features of different scales through a cross-domain feature decoupling module, and performing cross-scale feature decoupling... The system interacts with the image to obtain fused features. Normalized pixel coordinates of the rain-containing image are input into a latent neural branch, generating rain pattern features based on an implicit function. A cross-domain feature decoupling module then uses the rain pattern features as a reference to remove rain pattern information from the fused features, obtaining structural features. In the decoder stage, the structural features are upsampled to restore resolution, resulting in a de-rained image. The haze feature value is compared with a haze threshold; if the haze feature value is greater than the haze threshold, dehazing is triggered. The dehazing process uses a histogram self-attention mechanism to filter out low-frequency haze information while retaining high-frequency image details, resulting in a dehazed image. If the rainfall feature value is not greater than the rainfall threshold and the haze feature value is not greater than the haze threshold, the low-altitude remote sensing image to be detected is used as a pass-through image. The processed image or pass-through image is input into a small target detection model to identify and obtain the detection results for small targets at sea.

[0007] As an optional implementation, the acquisition of the spectral features of the low-altitude remote sensing image to be detected specifically includes: performing grayscale processing on the low-altitude remote sensing image to be detected to obtain a grayscale image, and converting the grayscale image to the frequency domain through a fast Fourier transform to obtain a frequency domain complex matrix and amplitude spectrum, which are used as spectral features.

[0008] As an alternative implementation method, the rainfall potential characteristic value is quantified based on the shielding mask. Specifically, a cross-shaped shielding mask is constructed. The cross-shaped shielding mask removes background interference by shielding the low-frequency region at the center of the spectrum and the horizontal and vertical axis regions. The ratio of the energy of the non-axial high-frequency region after shielding to the total energy of the entire spectrum is calculated as the quantified rainfall potential characteristic value.

[0009] As an alternative implementation method, the haze feature value is quantified based on the shielding mask. Specifically, a shielding mask in the central low-frequency region is constructed, and the ratio of the spectral energy in the central low-frequency region to the total energy of the entire spectrum is calculated as the quantified haze feature value.

[0010] As an alternative implementation, the dehazing process involves filtering out low-frequency haze information and retaining high-frequency image details using a histogram self-attention mechanism to obtain a dehazed image. Specifically, this includes: inputting the hazy image into a lightweight CNN to extract an initial spatial feature map; using a histogram self-attention mechanism, dividing the feature map pixels into multiple histogram bins according to brightness values, and performing self-attention calculations between the bins to obtain an enhanced feature map; inputting the enhanced feature map into a convolutional network to obtain a transmittance map, and combining it with the estimated atmospheric light values ​​to invert the dehazed image based on an atmospheric scattering model.

[0011] As an optional implementation, the step of inputting the processed image or through-pass image into the small target detection model to identify and obtain the detection result of small targets at sea specifically includes: inputting the processed image or through-pass image into the model, extracting multi-scale features through ResNet-50; the processed image is one of three types of images: an image after only de-raining processing, an image after only de-hazing processing, or an image after joint de-raining and de-hazing processing; using the inter-scale feature weighting aggregation module, combined with the small target density map, dynamically allocating attention weights to enhance the perception ability of small targets, and performing cross-scale global context modeling of multi-scale features; using a dynamic loss weighting mechanism to dynamically weight and fine-tune the bounding box to obtain the detection result of small targets at sea.

[0012] Secondly, this invention provides a small marine target detection system based on low-altitude remote sensing image enhancement, comprising: a feature calculation module configured to acquire spectral features of the low-altitude remote sensing image to be detected, and quantize rainfall feature values ​​and haze feature values ​​based on a masking layer; a rainfall recognition and processing module configured to compare the rainfall feature values ​​with a rainfall threshold, and if the rainfall feature values ​​are greater than the rainfall threshold, trigger deraining processing; the deraining processing involves learning rain degradation features and performing cross-scale feature interaction through a multi-scale Transformer network combined with implicit neural representation to remove rain streak interference and obtain a derained image; specifically, downsampling the rain-containing image to obtain multi-scale features, which are then input into the encoders corresponding to the Transformer-U-Net branches; a cross-domain feature decoupling module is used to establish a bidirectional information flow between features of different scales, and cross-scale feature interaction is performed to obtain fused features; the rain-containing image is then processed... Normalized pixel coordinates are input into an implicit neural branch to generate rain pattern features based on an implicit function. Then, through a cross-domain feature decoupling module, the rain pattern features are used as a reference to remove rain pattern information from the fused features, resulting in structural features. In the decoder stage, the structural features are upsampled to restore resolution, yielding a de-rained image. A haze recognition and processing module is configured to compare haze feature values ​​with a haze threshold. If the haze feature value is greater than the haze threshold, dehazing processing is triggered. This dehazing processing uses a histogram self-attention mechanism to filter out low-frequency haze information while retaining high-frequency image details, resulting in a dehazed image. A pass-through module is configured to use the low-altitude remote sensing image to be detected as a pass-through image if the rainfall feature value is not greater than the rainfall threshold and the haze feature value is not greater than the haze threshold. A small target detection module is configured to input the processed image or the pass-through image into a small target detection model to identify and obtain the detection results for small targets at sea.

[0013] As an optional implementation, the acquisition of the spectral features of the low-altitude remote sensing image to be detected specifically includes: performing grayscale processing on the low-altitude remote sensing image to be detected to obtain a grayscale image, and converting the grayscale image to the frequency domain through a fast Fourier transform to obtain a frequency domain complex matrix and amplitude spectrum, which are used as spectral features.

[0014] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in the first aspect.

[0015] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in the first aspect.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention is based on the spectral feature quantization mechanism of the shielding mask, which can quickly identify the type and intensity of rain and fog interference without labeling samples, providing a precise basis for subsequent enhancement; secondly, the rain removal process combines multi-scale Transformer and implicit neural representation to efficiently remove rain patterns while avoiding the loss of target details, and the fog removal process uses the histogram self-attention mechanism to accurately filter out fog interference and restore image contrast, which lays the foundation for subsequent detection work; finally, the small target detection model specifically enhances the perception and localization capabilities of small targets, effectively solving the problem of target confusion in complex sea background.

[0017] (2) This invention adopts the cascade integration of CFD-DRNet and ABL-TENet, and realizes the systematic and efficient processing of rain and fog coupling degradation in low-altitude coastal images through an adaptive cascade strategy, which significantly improves the image clarity, contrast and usability, and comprehensively solves the problem of multiple degradation.

[0018] (3) The IAFA-Module introduced in the MLPF-DETR effectively fuses fine-grained features and highlights small target features. It inherits the lightweight advantage of RemDet, significantly enhances small target features, and solves the problem that small target features are easily diluted in multi-level fusion. It also overcomes the problem of high false negative rate of detectors for small targets at sea.

[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0021] Figure 1 is a flowchart of the main process of a method for detecting small targets at sea based on low-altitude remote sensing image enhancement provided by an embodiment of the present invention; Figure 2 is a schematic diagram of an adaptive meteorological gating module based on frequency domain features provided by an embodiment of the present invention; Figure 3 is a schematic diagram of the overall structure of the CFD-DRNet network provided by an embodiment of the present invention; Figure 4 is a schematic diagram of the overall structure of the ABL-TENet network provided by an embodiment of the present invention; Figure 5 is a schematic diagram of the overall structure of the MLPF-DETR network provided by an embodiment of the present invention. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] Example 1, as shown in Figure 1, discloses a method for detecting small maritime targets based on low-altitude remote sensing image enhancement, comprising the following steps: S1: Obtaining the spectral features of the low-altitude remote sensing image to be detected, and quantizing the rainfall feature value and haze feature value based on a masking layer; S2: Comparing the rainfall feature value with a rainfall threshold. If the rainfall feature value is greater than the rainfall threshold, deraining processing is triggered. The deraining processing involves learning rain degradation features and performing cross-scale feature interaction through a multi-scale Transformer network combined with implicit neural representation to remove rain streak interference and obtain a derained image. Specifically, the rain-containing image is downsampled to obtain multi-scale features, which are then input into the encoders corresponding to the Transformer-U-Net branches. A bidirectional information flow between features of different scales is established through a cross-domain feature decoupling module to perform cross-scale feature interaction and obtain a fused image. The process involves: S1) Inputting the normalized pixel coordinates of the rain-containing image into an implicit neural branch to generate rain pattern features based on an implicit function; then, using a cross-domain feature decoupling module, removing rain pattern information from the fused features to obtain structural features, and upsampling the structural features in the decoder stage to restore resolution, resulting in a de-rained image; S2) Comparing the haze feature value with the haze threshold. If the haze feature value is greater than the haze threshold, dehazing is triggered. The dehazing process involves filtering out low-frequency haze information and retaining high-frequency image details using a histogram self-attention mechanism to obtain a dehazed image; S3) If the rainfall feature value is not greater than the rainfall threshold and the haze feature value is not greater than the haze threshold, the low-altitude remote sensing image to be detected is used as a pass-through image; S4) Inputting the processed image or pass-through image into a small target detection model to identify and obtain the detection results of small targets at sea.

[0024] Next, referring to Figure 1, a detailed description will be given of a method for detecting small targets at sea based on low-altitude remote sensing image enhancement disclosed in this embodiment.

[0025] To achieve meteorological interference perception and preprocessing of small targets in low-altitude scenarios with lower computational resources under unlabeled sample conditions, this embodiment proposes a zero-sample, low-power meteorological perception mechanism based on frequency domain energy spectrum characteristics. By performing pre-screening before the main process of small target detection, meteorological interference is filtered out in advance, reducing the computational load of subsequent detection modules while realizing the identification and preliminary suppression of meteorological degradation such as rain and fog.

[0026] Specifically, as shown in Figure 2, the original image containing the small target is acquired and then converted to grayscale. The resulting grayscale image is... The frequency domain is transformed using Fast Fourier Transform (FFT), and then spectral centering is performed to extract spectral features, resulting in a frequency domain complex matrix. and amplitude spectrum : ; Where u represents the index of the frequency component in the horizontal direction in the frequency domain, v represents the index of the frequency component in the vertical direction in the frequency domain, H represents the height of the original grayscale image, and W represents the width of the original grayscale image.

[0027] Based on the frequency domain complex matrix Further extract rainfall and haze feature values.

[0028] 1. Rainfall characteristics

[0029] Rain ripples exhibit high-frequency components in the frequency domain that are non-axial (i.e., not horizontal or vertical). To quantify this feature, a directional mask region is constructed. This region shields the low-frequency region at the center of the spectrum (radius). Inside; for example, The range is 1 / 16 to 1 / 8 of the frequency domain image dimension) and the horizontal and vertical axis regions (width is (The cross-shaped area).

[0030] The formula for calculating the characteristic value of rainfall intensity is as follows: ;in, This formula calculates the proportion of high-frequency energy in a specific direction to the total spectral energy, and can effectively characterize the intensity of rain streaks.

[0031] This embodiment targets rainy weather. By constructing a cross-shaped shielding mask to remove background interference and calculating the proportion of non-axial high-frequency energy to lock in rain patterns with directional characteristics, the directional features of rain patterns can be accurately extracted, effectively distinguishing rain patterns from small target details and improving the robustness of small target recognition in rain and fog images.

[0032] 2. Haze characteristic values

[0033] Haze causes the loss of high-frequency information in images, with energy highly concentrated in the low-frequency region. Constructing a central low-frequency region... (radius is) A circular region; for example, The range is from 1 / 8 to 1 / 4 of the frequency domain image dimension. The formula for calculating haze feature values ​​is as follows: This ratio reflects the degree to which spectral energy is concentrated at the center; the larger the ratio, the more severe the visibility attenuation.

[0034] This embodiment targets foggy weather and quantifies the degree of high-frequency detail attenuation by calculating the ratio of the central low-frequency region to the full spectrum energy. This quantifies the degree of high-frequency detail loss caused by fog and haze, providing a precise basis for subsequent enhancement of small target features and improving detection accuracy in foggy weather scenarios.

[0035] Then, threshold judgments were made based on the rainfall characteristic value and the haze characteristic value respectively.

[0036] Specifically, setting rainfall thresholds and smog threshold The control signal is generated using the following dual-threshold logic: rain switch signal ( ): ;when When =1, the rain removal network branch is activated.

[0037] Fog switch signal ( ): ;when When =1, the defogging network branch is activated.

[0038] According to the switch status and Automatically plan image flow direction This achieves the optimal balance between computational efficiency and image quality. ;in, For the input image, and These represent the rain removal and dehazing functions controlled by the switch, respectively. When the switch is 0, the function performs only an identity mapping, directly outputting the input image as a pass-through image; when the switch is 1, the corresponding deep learning network enhancement is performed.

[0039] This embodiment utilizes Fast Fourier Transform (FFT) to map the image to the frequency domain and designs a dual decoupled discrimination algorithm based on a spectral mask, which can identify rain and fog interference without large-scale data training. This orthogonal discrimination logic not only reduces the computational complexity of the pre-sensing to the millisecond level, but also realizes a dynamic inference path through an independent dual-switch design.

[0040] Specifically, the deraining and defogging network branches are not a simple "either / or" choice, but rather a serial structure triggered on demand. In clear weather, the deraining and defogging network branches are not needed; for single weather disturbances, the system only activates the corresponding branch; and when facing complex rain-fog coupled degradation scenarios, the system allows both switches to be activated simultaneously, causing the image to flow sequentially through the deraining and defogging modules for joint processing. This on-demand computation mechanism ensures all-weather detection accuracy while avoiding unnecessary computation, significantly improving the drone's endurance and inference speed under normal conditions.

[0041] Based on the above, four processing methods are applied to the image: no processing, deraining alone, dehazing alone, or a combination of deraining and dehazing.

[0042] (a) The image that activates the rain removal network branch is regarded as a rainy image. The rainy image is input into the Cross-Domain Feature Decoupling De-raining Network (CFD-DRNet).

[0043] Traditional methods that directly learn the mapping from rainy to dry images are prone to losing image details. CFD-DRNet, however, leverages the global modeling capabilities of the Transformer to transform the input rainy image into a single model. Decoupled into structural backbone features in feature space (Low-frequency information, carrying the target structure) and rain pattern residual characteristics (High-frequency information, carrying rain patterns).

[0044] Building upon CFD-DRNet, this paper aims to more precisely characterize the rain ripple residual features. To address the non-uniform distribution characteristics of rain patterns and avoid excessive erosion of target details in traditional rain pattern separation, this embodiment introduces Implicit Neural Rain Stroke Representation. This utilizes a small neural network. pixel coordinates Directly mapped to spatially varying rain streak feature maps This representation can continuously and finely simulate the morphology of non-uniformly distributed rain streaks, avoiding overfitting to discrete raindrops. The network performs joint optimization through minimization, ensuring that key target information in the image is preserved to the maximum extent during removal.

[0045] Specifically, as shown in Figure 3, a rain-containing image is acquired, and normalized pixel coordinates corresponding to its size are generated based on the rain-containing image. .

[0046] (1) Input rainy image Three scale features were obtained through bilinear downsampling: , , For example, these are 1 / 4, 1 / 2, and 1 times the original image, respectively.

[0047] (2) The images at the three scales are fed into the corresponding encoders in the Transformer-U-Net branch. The encoders 1, 2 and 3 have depths of 1, 2 and 3, respectively, to process rain pattern information of different granularities.

[0048] Features at different scales are input into the CDFD module, and bidirectional connections are established between features at adjacent scales to achieve coarse-to-fine and fine-to-coarse information flow. Bidirectional connection refers to... and Transition characteristics obtained by multiplying elements one by one and then passing the sigmoid function Then the transition features Feedback and Fusion features are obtained from information streams of different scales. The formula is as follows: ; ;in, It is the Sigmoid function. It is element-wise multiplication. It's channel splicing.

[0049] (3) Implicit neural branches (INR branches) in pixel coordinates As input, implicit functions are constructed using a multilayer perceptron (MLP). The MLP learns and fits the continuous distribution of rain streaks, outputting rain streak feature vectors. A multi-scale rain streak generator performs dimensional transformation and scale mapping on the rain streak feature vectors output by the MLP, generating rain streak feature maps. ;in, It is the generated rain pattern feature map.

[0050] (4) Introduce a cross-domain feature decoupling module, which receives two inputs: the fused features output through the aforementioned cross-scale interaction steps. Rain ripple feature map from INR branch A reverse attention mask with values ​​ranging from (0, 1) is constructed. This mask acts on the fused features, adaptively suppressing the rain ripple response through a gating mechanism. That is, CDFD uses the rain ripple features generated by the INR branch as a reference, performing a logical "subtraction" at the feature level, first removing the rain ripple component to obtain the structural features. Then, clean features The image is then handed over to the decoder for restoration. The decoupling formula is defined as: ;in, It is the Sigmoid activation function. This indicates element-wise multiplication.

[0051] (5) Structural features after decoupling The decoder upsamples the image step by step to restore its resolution. The final output layer generates a rain-free image through convolution. .

[0052] Among them, a hybrid loss function is used to balance pixel-level accuracy and structural similarity: In the formula, , , These are the weighting coefficients. Use Charbonnier loss to enhance training stability; To mitigate frequency domain loss and improve the quality of high-frequency detail recovery in images; To mitigate edge loss, preserve image structural information; The INR reconstruction loss constrains the output of the implicit representation.

[0053] As an implementation method, to ensure that the model achieves accurate rain stripe separation and target structure preservation in the rain removal task of rainy images, this embodiment adopts the following targeted training settings to adapt to the characteristics of the network architecture: the datasets selected are Rain200L and Rain200H, two typical rainy image datasets, covering different rain intensity scenarios to improve the model's generalization ability; the input images are uniformly adjusted to a standard size of 256×256 to ensure both the effectiveness of feature extraction and training efficiency; the optimizer uses the Adam optimizer, with an initial learning rate set to 0.0001, which is gradually decayed to 0.000001 through a cosine annealing strategy to balance the convergence speed in the early stage of model training with the accuracy of parameter fine-tuning in the later stage; in terms of the number of training epochs, 200 epochs are set for both the Rain200H and Rain200L datasets to ensure that the model fully learns the rain stripe feature distribution and structural feature mapping rules.

[0054] (ii) Dehazing Network Branch As shown in Figure 4, the image that activates the dehazing network branch is regarded as a foggy image. The foggy image is input into the Adaptive Brightness Localized-TransmissionEstimation Network (ABL-TENet).

[0055] To address the global contrast reduction and brightness imbalance caused by smog, ABL-TENet employs a U-Net-like structure, aiming to accurately estimate atmospheric transmittance. .

[0056] Based on ABL-TENet, in order to more accurately capture the dependencies between different brightness regions in the image and solve the problems of global brightness imbalance and insufficient contrast caused by haze, this embodiment introduces the "Histogram Self-Attention Mechanism". This mechanism divides the spatial pixels of the feature map X into B histogram bins according to their brightness values ​​(intensities). Self-attention calculation is performed between bins, rather than in the original space. This enables the network to efficiently capture the brightness dependencies over long distances and achieve global brightness balance and contrast restoration. This mechanism effectively solves the problem of overall whitening of the image caused by haze and provides a robust global context for transmittance estimation. Specifically: (1) Feature extraction: Input a hazy image Initial 2D spatial feature maps are extracted using a lightweight CNN. .

[0057] (2) Histogram self-attention mechanism: First, the feature map is... The spatial pixels are divided into B=32 histogram bins according to their brightness values; then, the feature map is... The feature vectors of all spatial pixels belonging to the same brightness box are aggregated to obtain the brightness box features. Histogram self-attention computation is performed between bins, rather than in the traditional spatial dimension. This allows the network to efficiently capture brightness dependencies over long distances, achieving global brightness equalization.

[0058] The brightness dependency refers to the statistical correlation and semantic association between pixels / features of different brightness levels in the feature map. For example, dark areas (low brightness boxes) and bright areas (high brightness boxes) may belong to different illuminated surfaces of the same object in a scene, and there is a structural dependency; feature changes in the medium brightness area will simultaneously affect the visual performance of both dark and bright areas, forming a cross-box brightness linkage.

[0059] This brightness dependency is modeled using the following self-attention formula: ;in, , , It is a Query, Key, and Value matrix grouped by brightness box. This represents the dimension of the key vector. This indicates transpose. This term is used to calculate the similarity weight between different brightness boxes, which directly reflects the brightness dependence strength between each brightness box; the network uses this weight to analyze the brightness box features. Weighted fusion is performed to achieve global interaction and adaptive correction of different brightness box characteristics, avoiding problems such as loss of detail, insufficient contrast, or overexposure and underexposure caused by local brightness imbalance.

[0060] The weighted and corrected brightness box features are then stitched together and their dimensions restored to reconstruct the initial feature map. The spatial and channel dimensions are used to output an enhanced feature map after global brightness equalization. .

[0061] (3) The network output is a transmittance map.

[0062] Using a convolutional network consisting of a 3×3 convolutional layer and a 1×1 convolutional layer, the feature map is enhanced. The feature map is mapped to a single channel to compress the feature dimension and integrate channel information.

[0063] Due to transmittance The physical meaning of transmittance is the proportion of light that penetrates the medium, and its value must be between (0, 1). Therefore, the Sigmoid activation function is used to perform nonlinear mapping on the single-channel feature map to obtain the final transmittance map. : ;in This represents the Sigmoid function.

[0064] Atmospheric light value A is obtained from the input hazy image. The dark channel prior method, which selects the brightest pixel, is used to estimate the value. The final restored image is obtained through inversion using an atmospheric scattering model, and the output is a dehazed image. : ;in, It is a small constant set for numerical stability.

[0065] Loss function: A joint loss function is used to simultaneously constrain the restored image and the intermediate transmittance map. ;in: Restored image With real images L1 loss between. It is a prediction of transmittance With true transmittance Smoothing L1 loss between them.

[0066] As one implementation method, to fully leverage the advantages of the histogram self-attention mechanism of the ABL-TENet network and accurately address the global contrast reduction and brightness imbalance caused by haze, while ensuring the accuracy of transmittance estimation and the quality of the restored image, this embodiment designs training settings adapted to the network architecture and task requirements: A synthetic haze dataset covering diverse haze scenarios is selected to provide sufficient sample support for the model to learn the mapping rules between haze features and transmittance; input images are uniformly adjusted to a standard size of 256×256 to balance the completeness of feature extraction and training efficiency; the optimizer uses AdamW to effectively alleviate gradient vanishing and weight decay problems, with an initial learning rate set to 0.0001, combined with the CosineAnnealing learning rate scheduling strategy to achieve dynamic adaptive adjustment of the learning rate, balancing the convergence speed in the early stages of model training with the accuracy of parameter optimization in the later stages; the training process is set to 200 epochs to ensure the model fully fits the data distribution, while a batch size of 16 is used to improve training stability and computational efficiency.

[0067] Furthermore, the processed image or the through image is input into the multi-level decoupled fusion target detection module for small target detection. The processed image is one of three types: an image that has only undergone deraining processing, an image that has only undergone dehazing processing, or an image that has undergone combined deraining and dehazing processing.

[0068] As shown in Figure 5, a Multi-Level Progressive Fusion DETR (MLPF-DETR) detector is adopted. Based on the efficient architecture principle of the RemDet model, it has been deeply innovated to adapt to the detection of small targets at sea. The enhanced high-quality, rain- and fog-free images or straight-through images are collectively referred to as… Using this as input, ResNet-50 is used to extract multi-scale features (P3, P4, P5). Specifically: (1) Inter-scale feature weighting aggregation module (IAFA-Module): MLPF-DETR inherits RemDet's efficient and lightweight feature extraction concept in UAV image processing, but in its hybrid encoder, it integrates an innovative inter-scale feature weighting aggregation module, which is used to efficiently aggregate the high-resolution fine-grained features extracted by CNN Backbone and the global semantic information captured by Transformer Encoder.

[0069] Meanwhile, the module introduces a dynamic reweighting mechanism based on small target density maps. For the high-resolution layer in the feature pyramid used to detect small targets, IAFA-Module assigns higher attention weights based on the small target density maps pre-calculated or dynamically generated during the training phase. This significantly enhances the encoder's ability to perceive small targets and overcomes the problem that small target features may be diluted in the feature fusion of the RemDet model.

[0070] Specifically, the multi-scale features (P3, P4, P5) output by the backbone network are used as input. First, a 1x1 convolution is used to unify the number of feature channels of P3, P4, and P5 to C. Then, a density regression branch consisting of a 1×1 convolutional layer and an activation function maps the multi-scale fused features after unifying the channels to a single-channel density response map.

[0071] During the training phase, based on the distribution of real small targets (e.g., pixel area <32x32) on the P3 layer feature map, a small target density heatmap is pre-calculated or dynamically generated. This map shows a high response value in areas where small targets are present. (Density map) After normalization, the weights are modulated into the self-attention calculation of the P3 layer features. Specifically, when calculating the query-key similarity matrix of the P3 features, a density map-based weight matrix is ​​introduced. : ;in, From density map This is generated by a small network, which enables the attention mechanism to significantly enhance its ability to perceive features in dense regions of small targets. This represents the output features obtained after the P3 layer features have been processed by this attention mechanism. , , These represent the query, key, and value corresponding to the features in layer P3, respectively. Indicates the size of the dimension. This indicates transpose.

[0072] The weighted P3 features, along with the P4 and P5 features, are fed into a standard Transformer encoder for cross-scale global context modeling to obtain the target features. The IAFA-Module ensures that the high-resolution P3 features, which are crucial for small targets, dominate the fusion process.

[0073] (2) Dynamic Loss Weighted and Target Localization Refinement Head (DLW-PLH) Design: DLW-PLH is a decoupled prediction head of MLPF-DETR, which decouples target classification, bounding box regression and localization refinement. This design is a further enhancement of RemDet's approach to optimizing localization accuracy for UAV images.

[0074] The target localization refinement head, after the initial bounding box is output by the conventional bounding box branch, takes the initial bounding box and its corresponding target features as input to further extract local spatial detail features, predict a set of fine coordinate offsets, and correct the initial bounding box with these offsets to improve target localization accuracy. Therefore, the target localization refinement head can be understood as an optimization module for bounding box regression. Decoupling refers to breaking the traditional structure where classification and localization tasks share feature extraction parameters, and computing the three sub-tasks of target classification, bounding box regression, and localization refinement independently and in parallel within the network architecture.

[0075] DLW-PLH introduces a dynamic loss weighting mechanism to address the imbalance of positive and negative samples and the difficulty in distinguishing between easy and difficult samples during training for small targets. It automatically increases the regression loss weight for small targets that are difficult to locate. More importantly, it includes a localization refinement branch specifically for correcting the localization of small targets in water, using additional higher-order pose parameters. Loss-guided localization addresses the challenge of inaccurate localization of small targets under complex sea conditions, improving final positioning accuracy. Specifically, a standard Transformer decoder is employed, but its prediction head is designed as a Dynamic Loss Weighted and Localization Refinement Head (DLW-PLH) to decouple and refine the task. A set of learnable object queries serves as input to the decoder, and the output is a rich feature representation of a target corresponding to each object query. The decoupled prediction head processes each target feature output by the decoder in parallel through three independent branches.

[0076] 1. Classification branch refers to a linear layer that outputs the confidence score for each category.

[0077] 2. Bounding box regression branch: An MLP that predicts the initial coordinates of the target bounding box. .

[0078] 3. A separate 3-layer feedforward network (FFN) takes the initial bounding box and object features from the regression branch as input to predict a fine-grained offset. This is used to fine-tune the initial bounding box. The final refined bounding box is: ;in, It is a scaling factor that controls the refinement step size. This represents the initial target bounding box output by the bounding box regression branch. This represents the fine offset predicted by the 3-layer feedforward network (FFN).

[0079] Output the final detection results.

[0080] Loss function: The loss function of MLPF-DETR is based on the bipartite graph matching and loss of standard DETR, and introduces dynamic weighting and refinement mechanism.

[0081] ;in, , , These are the weighting coefficients. It is a classification loss, ensuring that the model can correctly determine the category of the object within the bounding box; It is a localization refinement loss, which fine-tunes the initial regression bounding box, especially improving the high threshold localization accuracy for small targets; It is a bounding box regression loss, which aims to make the bounding boxes predicted by the model coincide as closely as possible with the ground truth boxes in terms of location and size. It's not a simple addition, but rather modulated through a dynamic weighting. This is based on the relationship between the predicted and ground truth bounding boxes. Value, dynamically calculate a weight : ;in It is a dynamic weighting factor. This is the intersection-over-union ratio (IoU) between the predicted bounding box and its matched ground truth bounding box, reflecting the real-time localization quality of the current box. γ is an adjustment factor used to control the degree of weight difference between easy and difficult samples. The larger the γ value, the more weighted the difficult samples (low... The greater the weight of ), the higher the amplification.

[0082] Applying the dynamic weighting factor α to the original regression loss yields the weighted regression loss function: For hard samples: when the predicted bounding box is inaccurate, the value is low, (1 The value close to 1 results in a dynamic weight α that is significantly greater than 1. This is equivalent to amplifying the regression loss of the sample, generating a larger gradient during backpropagation, thus forcing the model parameters to be adjusted to a greater extent, prioritizing the optimization of these poorly localized predictions. For easy samples: when the predicted bounding box localization is already very accurate, the IoU value is high, (1 As the value approaches 0, the dynamic weight α approaches 1. At this point, the regression loss remains essentially unchanged, and the model will not pay much attention to these learned samples.

[0083] In this embodiment, MLPF-DETR is built upon RemDet's efficient architecture principles, ensuring the model has real-time or ultra-real-time inference speed (high FPS), making it suitable for practical deployment on UAV platforms. Specifically, the localization refinement branch in the DLW-PLH module further optimizes RemDet's efficient localization, particularly improving the strictness of localization by precisely correcting the bounding boxes of small targets. Positioning accuracy below the threshold (mAP@0.75) is crucial for critical scenarios such as search and rescue.

[0084] As one implementation method, to fully leverage the performance advantages of MLPF-DETR in small target detection at sea, adapt the design logic of the inter-scale feature weighting aggregation module and the dynamic loss weighted localization refinement head, and specifically address issues such as the easy dilution of small target features at sea, high localization accuracy requirements, and unbalanced sample distribution, this embodiment has formulated the following refined training settings: The dataset selected is the SeaDronesSeeV2 dataset, which is specifically used for target detection of UAVs at sea, to provide accurate data support for the model to learn the adaptation rules of small target features at sea and complex sea surface backgrounds; the optimizer adopts AdamW with weight decay set to 0.0001 to effectively suppress model overfitting and ensure the stability of feature extraction and target detection; the initial learning rate is set to 0.0001, combined with cosine decay with hot restart. The fire strategy achieves dynamic decay of the learning rate, balancing convergence efficiency in the early stages of training with the accuracy of parameter fine-tuning in the later stages. A batch size of 16 is used during training to balance computational efficiency and gradient stability, with a uniform training duration of 200 epochs to ensure the model fully fits the data distribution. For data augmentation, strong enhancement techniques such as Mosaic stitching, random cropping, color dithering, and multi-scale training are used to simulate image changes under complex lighting and weather conditions at sea, significantly improving the model's robustness to small targets. In terms of matching strategy, the Hungarian algorithm is used for bipartite graph matching, but the weight of small target classification scores is deliberately increased in the cost matrix to ensure that more small targets are correctly assigned during the matching process, avoiding them being ignored due to their small scale. This, combined with the aforementioned loss function, achieves dual optimization of small target detection accuracy and localization accuracy.

[0085] This specific implementation achieves end-to-end collaboration between interference recognition, image enhancement, and accurate detection, significantly improving the accuracy and robustness of small target detection at sea under adverse weather conditions while ensuring the real-time requirements of low-altitude platforms.

[0086] Example 2 This example provides a small marine target detection system based on low-altitude remote sensing image enhancement, including: a feature calculation module configured to acquire the spectral features of the low-altitude remote sensing image to be detected, and quantize the rainfall feature value and haze feature value based on a masking layer; a rainfall recognition and processing module configured to compare the rainfall feature value with a rainfall threshold, and if the rainfall feature value is greater than the rainfall threshold, trigger deraining processing; the deraining processing is to learn rain degradation features and perform cross-scale feature interaction through a multi-scale Transformer network combined with implicit neural representation to remove rain streak interference and obtain a derained image; specifically, the rain-containing image is downsampled to obtain multi-scale features, which are respectively input into the encoder corresponding to the Transformer-U-Net branch; a cross-domain feature decoupling module is used to establish a bidirectional information flow between features of different scales, and cross-scale feature interaction is performed to obtain fused features; the rain-containing image is then processed. Normalized pixel coordinates are input into an implicit neural branch to generate rain pattern features based on an implicit function. Then, through a cross-domain feature decoupling module, the rain pattern features are used as a reference to remove rain pattern information from the fused features, resulting in structural features. In the decoder stage, the structural features are upsampled to restore resolution, yielding a de-rained image. A haze recognition and processing module is configured to compare haze feature values ​​with a haze threshold. If the haze feature value is greater than the haze threshold, dehazing processing is triggered. This dehazing processing uses a histogram self-attention mechanism to filter out low-frequency haze information while retaining high-frequency image details, resulting in a dehazed image. A pass-through module is configured to use the low-altitude remote sensing image to be detected as a pass-through image if the rainfall feature value is not greater than the rainfall threshold and the haze feature value is not greater than the haze threshold. A small target detection module is configured to input the processed image or the pass-through image into a small target detection model to identify and obtain the detection results for small targets at sea.

[0087] Example 3 This example provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in Example 1 above.

[0088] Example 4 This example provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in Example 1 above.

[0089] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0090] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting small maritime targets based on low-altitude remote sensing image enhancement, characterized in that, include: The spectral features of the low-altitude remote sensing image to be detected are obtained, and the rainfall and haze feature values ​​are quantified based on the masking. The rain intensity feature value is compared with the rain intensity threshold. If the rain intensity feature value is greater than the rain intensity threshold, rain removal processing is triggered. The rain removal processing is carried out by learning rain degradation features and performing cross-scale feature interaction through a multi-scale Transformer network combined with implicit neural representation to remove rain streak interference and obtain a rain-removed image. Specifically, the rain-containing image is downsampled to obtain multi-scale features, which are then input into the encoders corresponding to the Transformer-U-Net branches. A bidirectional information flow between features of different scales is established through a cross-domain feature decoupling module to perform cross-scale feature interaction and obtain fused features. The normalized pixel coordinates of the rain-containing image are input into the implicit neural branch, and rain pattern features are generated based on the implicit function. Then, through the cross-domain feature decoupling module, the rain pattern features are used as a reference to remove the rain pattern information in the fused features to obtain the structural features. In the decoder stage, the structural features are upsampled to restore the resolution and obtain the rain-removed image. The haze feature value is compared with the haze threshold. If the haze feature value is greater than the haze threshold, dehazing processing is triggered. The dehazing processing is to filter out low-frequency haze information and retain high-frequency image details through a histogram self-attention mechanism to obtain a dehazed image. If the rainfall feature value is not greater than the rainfall threshold and the haze feature value is not greater than the haze threshold, the low-altitude remote sensing image to be detected is used as a pass-through image. The processed image or pass-through image is input into the small target detection model to identify and obtain the detection results of small targets at sea.

2. The method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in claim 1, characterized in that, The acquisition of the spectral features of the low-altitude remote sensing image to be detected specifically includes: performing grayscale processing on the low-altitude remote sensing image to be detected to obtain a grayscale image, and converting the grayscale image to the frequency domain through a fast Fourier transform to obtain a frequency domain complex matrix and amplitude spectrum, which are used as spectral features.

3. The method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in claim 1, characterized in that, The rainfall potential characteristic value is quantified based on the shielding mask. Specifically, a cross-shaped shielding mask is constructed. The cross-shaped shielding mask removes background interference by shielding the low-frequency region at the center of the spectrum and the horizontal and vertical axis regions. The ratio of the energy of the non-axial high-frequency region after shielding to the total energy of the whole spectrum is calculated as the quantified rainfall potential characteristic value.

4. The method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in claim 1, characterized in that, The method for quantifying haze feature values ​​based on a shielding mask is as follows: a shielding mask is constructed in the central low-frequency region, and the ratio of the spectral energy in the central low-frequency region to the total energy of the entire spectrum is calculated as the quantified haze feature value.

5. The method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in claim 1, characterized in that, The dehazing process involves filtering out low-frequency haze information and retaining high-frequency image details using a histogram self-attention mechanism to obtain a dehazed image. Specifically, this includes: inputting the hazy image into a lightweight CNN to extract an initial spatial feature map; using a histogram self-attention mechanism, dividing the feature map pixels into multiple histogram bins according to brightness values, and performing self-attention calculations between the bins to obtain an enhanced feature map; inputting the enhanced feature map into a convolutional network to obtain a transmittance map, and combining it with the estimated atmospheric light values ​​to invert the dehazed image based on an atmospheric scattering model.

6. The method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in claim 1, characterized in that, The process of inputting the processed image or through-pass image into the small target detection model to identify and obtain the detection results of small targets at sea specifically includes: inputting the processed image or through-pass image into the model and extracting multi-scale features through ResNet-50; the processed image is one of three types of images: an image after only de-raining processing, an image after only de-hazing processing, or an image after joint de-raining and de-hazing processing; using the inter-scale feature weighting aggregation module, combined with the small target density map, dynamically allocating attention weights to enhance the perception ability of small targets, and performing cross-scale global context modeling of multi-scale features; and using a dynamic loss weighting mechanism to dynamically weight and fine-tune the bounding box to obtain the detection results of small targets at sea.

7. A small maritime target detection system based on low-altitude remote sensing image enhancement, characterized in that, include: The feature calculation module is configured to acquire the spectral features of the low-altitude remote sensing image to be detected, and quantify the rainfall feature value and haze feature value based on the shielding mask. The rainfall recognition and processing module is configured to compare rainfall feature values ​​with rainfall thresholds. If the rainfall feature value is greater than the rainfall threshold, rain removal processing is triggered. The rain removal processing involves learning rain degradation features and performing cross-scale feature interaction through a multi-scale Transformer network combined with implicit neural representations to remove rain streak interference and obtain a rain-removed image. Specifically, the rain-containing image is downsampled to obtain multi-scale features, which are then input into the encoders corresponding to the Transformer-U-Net branches. A cross-domain feature decoupling module is used to establish a bidirectional information flow between features of different scales, and cross-scale feature interaction is performed to obtain fused features. The normalized pixel coordinates of the rain-containing image are input into the implicit neural branch, and rain pattern features are generated based on the implicit function. Then, through the cross-domain feature decoupling module, rain ripple features are used as a reference to remove rain ripple information from the fused features to obtain structural features. In the decoder stage, the structural features are upsampled to restore resolution, resulting in a de-rained image. The haze recognition and processing module is configured to compare haze feature values ​​with haze thresholds. If the haze feature value is greater than the haze threshold, dehazing processing is triggered. The dehazing processing uses a histogram self-attention mechanism to filter out low-frequency haze information and retain high-frequency image details to obtain a dehazed image. The pass-through module is configured to use the low-altitude remote sensing image to be detected as a pass-through image if the rain intensity feature value is not greater than the rain intensity threshold and the haze feature value is not greater than the haze threshold. The small target detection module is configured to input the processed image or the pass-through image into the small target detection model to identify and obtain the detection results of small targets at sea.

8. The small maritime target detection system based on low-altitude remote sensing image enhancement as described in claim 7, characterized in that, The acquisition of the spectral features of the low-altitude remote sensing image to be detected specifically includes: performing grayscale processing on the low-altitude remote sensing image to be detected to obtain a grayscale image, and converting the grayscale image to the frequency domain through a fast Fourier transform to obtain a frequency domain complex matrix and amplitude spectrum, which are used as spectral features.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in any one of claims 1-6.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for detecting small maritime targets based on low-altitude remote sensing image enhancement as described in any one of claims 1-6.

Images