Unmanned aerial vehicle remote sensing defogging method based on dual-dimensional prior extraction and four-level adaptive collaborative fusion

Abstract

This application relates to a UAV remote sensing dehazing method based on two-dimensional prior extraction and four-level adaptive collaborative fusion. First, non-uniform fog remote sensing images are preprocessed. A semantic segmentation network extracts spatial fog concentration masks, and a feature regression network extracts regional atmospheric light and scattering coefficients, constructing a spatial and physical two-dimensional prior. Second, based on the prior, multi-path physical sensing image signal processing operators are constructed to enhance the image in parallel, acquiring multi-dimensional features. Dynamic collaborative fusion of these multi-path features is achieved through a feature fusion network using spatial feature modulation, adaptive receptive field allocation, and an attention mechanism. Finally, a dehazing main network with an encoder-decoder structure is used for image reconstruction, outputting a clear remote sensing image. A joint loss function is constructed and a physical consistency constraint is introduced. A training dataset is generated using a physical scattering model, enabling the model to accurately adapt to complex and non-uniform fog environments. This application improves remote sensing image dehazing while maintaining detail fidelity and global consistency.

Description

Technical Field

[0001] This application relates to the fields of image processing, computer vision, and UAV remote sensing, and in particular to a UAV remote sensing defogging method based on two-dimensional prior extraction and four-level adaptive collaborative fusion. Background Technology

[0002] With the rapid development of sensor and aviation technologies, drones have become an indispensable tool in many fields such as modern remote sensing mapping, environmental monitoring, and disaster emergency response due to their advantages such as strong maneuverability, wide field of view, and convenient deployment. The integration of drone remote sensing systems has greatly enhanced the ability to acquire data in large-area and complex environments.

[0003] However, when performing remote sensing missions in complex environments such as mountain valleys, over water, at dawn and dusk, and in severe weather, UAVs still face significant image challenges. Due to variations in airflow and temperature and humidity in complex terrain, uneven fog or gradient fog of varying concentrations and complex spatial distributions easily form in the atmosphere. Remote sensing images taken by UAVs in such environments inevitably suffer from severe environmental interference. The main manifestations of this image degradation are: severe shift in underlying physical colors, a significant decrease in image contrast, and loss of key edge texture information.

[0004] Uneven fog phenomena caused by complex environments can severely degrade the quality of remote sensing images, resulting in the loss of key information in the images. In practical drone mapping applications, such as mapping after landslides in mountainous areas, forest resource patrols, and mountain village planning, unprocessed foggy remote sensing images severely restrict the subsequent use of the acquired images.

[0005] Existing image dehazing methods are mostly based on global atmospheric scattering models or single degradation estimates, which cannot effectively solve the problem of non-uniform and complex environments with non-uniform fogging. They are prone to problems such as past fog, under-dehazing, color shift, and block effect. Although some local dehazing methods can adapt to fog concentration differences to a certain extent, they have shortcomings such as insufficient texture enhancement, high computational complexity, and poor global consistency. They are difficult to meet the comprehensive requirements of UAV remote sensing images for real-time performance, texture fidelity, color restoration, and global uniformity. In special scenarios such as geological disaster emergency mapping and mountain aerial photography, image quality problems caused by non-uniform fog can directly increase the rate of unusable remote sensing images and delay emergency surveys and disaster assessments.

[0006] Therefore, developing a method that can accurately adapt to the uneven fog distribution characteristics of UAV remote sensing in complex environments, and achieve adaptive fog defogging, dual fidelity of physical color and texture, and global image enhancement, is of great practical significance for improving the quality of UAV remote sensing images, expanding the application scenarios of UAV remote sensing technology, and ensuring the accuracy of remote sensing interpretation and analysis. Summary of the Invention

[0007] To address the technical challenge of mitigating uneven fog distribution in complex environments using UAV remote sensing, this application provides a UAV remote sensing defogging method based on two-dimensional prior extraction and four-level adaptive collaborative fusion.

[0008] This application provides a UAV remote sensing defogging method based on two-dimensional prior extraction and four-level adaptive collaborative fusion, which adopts the following technical solution: A UAV remote sensing dehazing method based on two-dimensional prior extraction and four-level adaptive collaborative fusion includes the following steps: S1. Obtain non-uniform fog remote sensing images collected by UAVs, perform normalization preprocessing on them, and extract spatial fog concentration masks as spatial priors through semantic segmentation networks. At the same time, extract regional atmospheric light and scattering coefficients as physical priors through feature regression networks. S2. Based on the spatial prior and physical prior, construct a multi-path physical perception image signal processing operator to perform parallel processing on the input image and obtain a multi-dimensional enhanced feature map. S3. Input the multidimensional enhanced feature map into the feature adaptive enhancement fusion network, and realize the dynamic fusion of multi-path features through spatial feature modulation, receptive field adaptive allocation and attention weighting mechanism; S4. Input the fused features into the dehazing main network based on the encoder-decoder structure for reconstruction, and output a clear remote sensing image.

[0009] By adopting the above technical solutions, the problem of non-uniform fog distribution in UAV remote sensing images under complex environments can be addressed by achieving precise regional-level perception and adaptive processing of fog concentration. By fusing spatial and physical prior information, the targeting and physical consistency of the defogging process are improved. Through multi-path physical perception operators and feature adaptive fusion mechanisms, comprehensive enhancement of image brightness, color, and texture is achieved. At the same time, high-quality image reconstruction is completed by combining an encoder-decoder structure, thereby effectively improving the defogging effect, detail fidelity, and overall visual consistency, meeting the application requirements of remote sensing image processing in complex environments.

[0010] Optionally, the spatial prior is extracted using a DeepLabv3+ network, and multi-scale contextual information is used to generate probability distribution masks for different fog concentration levels.

[0011] By adopting the above technical solution, it is possible to accurately identify and segment fog concentrations in different regions using multi-scale contextual information, improve spatial prior representation capabilities, provide a reliable basis for subsequent adaptive defogging, and thus improve defogging accuracy and overall image processing effect.

[0012] Optionally, the physical prior is extracted using an improved ResNet network, which removes the classification layer and regresses the atmospheric light and scattering coefficients corresponding to different fog concentration regions using global average pooling and a multi-branch fully connected structure.

[0013] By adopting the above technical solution, the atmospheric light and scattering coefficients in regions with different fog concentrations can be accurately estimated, enhancing the physical prior expression capability and providing reliable parameter support for the defogging process, thereby improving the accuracy of defogging and the image restoration effect.

[0014] Optionally, the multi-channel physical sensing image signal processing operator includes a dehazing operator, a noise reduction operator, a white balance operator, and a sharpening operator, and the parameters of each operator are dynamically adjusted based on physical priors through nonlinear mapping.

[0015] By adopting the above technical solution, the parameters of each processing operator can be adaptively adjusted according to different fog concentration areas, so as to achieve synergistic optimization of defogging, noise reduction and color correction, and improve image details, contrast and color reproduction.

[0016] Optionally, the dynamic adjustment involves generating parameter offsets by inputting the physical parameter vectors into a nonlinear function, and then correcting the basic parameters of each operator to achieve adaptive processing for different fog concentration regions.

[0017] By adopting the above technical solution, the operator parameters can be adaptively corrected based on physical parameters, enabling targeted processing of different fog concentration areas, improving defogging accuracy and processing flexibility, and enhancing the overall image optimization effect.

[0018] Optionally, the feature adaptive enhancement fusion network includes a spatial feature transformation module, a receptive field adaptation module, and an attention reweighting module, wherein: The spatial feature transformation module performs scaling and bias modulation on the features through affine transformation; The receptive field adaptive module dynamically allocates convolutional kernels of different scales based on the fog concentration mask; The attention reweighting module performs spatial and channel-dimensional weighted optimization on the fused features.

[0019] By adopting the above technical solutions, physical priors can be effectively embedded into the feature space to achieve dynamic modulation and accurate expression of features; through the receptive field adaptive allocation mechanism, the feature extraction capability of different fog concentration regions can be matched with appropriate scales; at the same time, by combining spatial and channel attention reweighting, key features are strengthened and redundant information is suppressed, thereby improving the feature fusion quality and defogging effect, and enhancing the image detail and overall consistency.

[0020] Optionally, the feature adaptive enhancement fusion network further includes a four-level collaborative fusion structure composed of SFT, RFB, CBAM and VMamba, used to achieve multi-scale feature enhancement and global consistency optimization.

[0021] By adopting the above technical solutions, we can achieve collaborative enhancement of multi-scale features and global information modeling, improve the expression of local details and overall consistency, reduce the problem of discontinuity at regional boundaries, and thus optimize the dehazing effect and visual quality.

[0022] Optionally, the dehazing main network adopts a U-Net encoder-decoder structure and fuses high-resolution feature information through skip connections during the encoding and decoding process.

[0023] By adopting the above technical solutions, high-resolution detail information can be preserved during feature extraction and reconstruction, information loss can be reduced, and image reconstruction quality can be improved, thereby enhancing the dehazing effect and detail restoration capability.

[0024] Optionally, the defogging main network introduces a VMamba module in the bottleneck layer, which performs multi-directional serialization scanning of features through a state space model to achieve global long-range dependency modeling and regional boundary smoothing.

[0025] By adopting the above technical solutions, global long-range dependency modeling of features and cross-regional information interaction are achieved, enhancing the continuous expression between different fog areas, smoothing regional boundaries, and thus improving the overall consistency and visual naturalness of dehazed images.

[0026] Optionally, the decoding stage uses a PixelShuffle-based upsampling method to restore the spatial resolution step by step, and outputs the final dehazed image; During model training, a joint loss function is constructed, including basic reconstruction loss, visual perception loss, contrastive learning loss, and physical consistency loss. The physical consistency loss is achieved by substituting the predicted clear image and physical prior parameters into the atmospheric scattering model to generate a simulated fog image, and then calculating the error between the simulated fog image and the input fog image to constrain the network to conform to the physical degradation law. It also includes the step of building a training dataset based on a physical scattering model, generating fogged images by introducing spatial non-uniform scattering coefficients and depth information, and generating fog concentration labels based on transmittance thresholds; The transmittance is calculated based on depth information and scattering coefficient, and is divided into multiple fog concentration levels by a preset physical threshold, which is used to supervise the training of the spatial prior extraction network.

[0027] By adopting the above technical solutions, high-resolution spatial information can be recovered step by step during the decoding stage using the PixelShuffle upsampling method, improving the clarity and detail restoration capability of the reconstructed image. Simultaneously, by constructing a joint loss function that integrates basic reconstruction, visual perception, contrastive learning, and physical consistency, the network training process is constrained from multiple dimensions—pixel level, perception level, and feature level—enhancing the realism and stability of the dehazing results. The physical consistency loss, by substituting the predicted clear image and the regressed physical prior parameters into the atmospheric scattering model for reverse rendering, forms a simulated fog image and compares the error with the input fog image, achieving physical closed-loop constraints and enhancing the model's interpretability and generalization ability. Furthermore, by constructing a training dataset based on the physical scattering model, spatial non-uniform scattering coefficients and depth information are introduced to generate fogged images, and fog concentration labels are generated based on transmittance thresholds, enabling the spatial prior extraction network to obtain reliable supervision signals. Transmittance is calculated from depth and scattering coefficients and divided into multiple fog concentration levels according to preset physical thresholds, improving the network's adaptability and segmentation accuracy in complex non-uniform fog environments.

[0028] In summary, this application includes at least one of the following beneficial technical effects: 1. In the data synthesis stage, this application innovatively adopts a fixed physical transmittance threshold based on the underlying optical formula to classify light / no, medium and dense fog labels. This mechanism optimizes the "label semantic drift" problem caused by the traditional dynamic threshold to a certain extent and provides the network with a spatial prior supervision signal based on unified physical constraints. At the same time, bicubic interpolation and nonlinear Gamma correction are introduced in the depth map processing to effectively eliminate edge jaggedness and blocky fog artifacts. This high-fidelity data processing mechanism with physical consistency provides robust prior constraints for the network, enabling the model to maintain high-precision dehazing and generalization ability in complex gradient fog environments. 2. This application achieves regional segmentation of fog concentration and accurate estimation of regional atmospheric physical parameters by extracting prior information from both physical and spatial dimensions. This weighted fusion of regional and global parameters enables the network to accurately perceive the differences between local dense fog and light fog, which helps to solve the contradiction between "globally unified parameters" and "uneven local water vapor distribution" in traditional defogging algorithms, reduces the processing distortion problem that traditional methods are prone to cause, and provides a reliable basis for accurate defogging in complex environments. 3. This application constructs a multi-dimensional physical enhancement operator module pool, which comprehensively covers the optimization needs of images in all dimensions such as dehazing, noise reduction, color restoration and brightness correction. Each processing module innovatively adopts an adaptive control mechanism of "dynamic fine-tuning of underlying anchor point parameters combined with physical priors", which can achieve precise parameter tuning based on the spatial fog concentration distribution. While dehazing, it highly enhances the color restoration, contrast and high-frequency edge information of the image. The multi-dimensional complementary features output by this mechanism provide a rich representation space for subsequent deep networks, effectively overcoming the technical defects of single algorithms that are prone to image smoothing, local color distortion and loss of underlying mapping texture when processing complex non-uniform fog. 4. This application innovatively designs a four-level fusion architecture of SFT-RFB-CBAM-VMamba, which realizes the full-process collaborative processing of regional tone control, receptive field adaptation, texture precision enhancement and global smoothing. In particular, the introduction of the VMamba module in the core control network for global state scanning can effectively smooth the boundary between light fog, medium fog and dense fog areas, effectively weaken the splicing and block effect that is easily caused by regional processing, and better balance the accurate defogging of local dense fog areas with the natural visual consistency of the entire remote sensing image. 5. This application constructs a joint loss function system that integrates basic reconstruction, visual perception, contrastive learning, and physical priors. This system ensures high-fidelity restoration of both the underlying texture and high-frequency details of remote sensing images through the joint constraints of Charbonnier and perceptual loss, avoiding smoothing distortion. It also expands the feature space distance by using contrast constraints, effectively eliminating residual fog in complex environments. Furthermore, a physical consistency loss is designed, which substitutes the physical parameters predicted by the network into the atmospheric scattering model for reverse reconstruction and verification, effectively constructing a physical closed loop and improving the model's strong generalization ability and relatively reliable physical interpretability in extreme and harsh environments to a certain extent. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the dataset generation module in an embodiment of this application.

[0030] Figure 2 This is a schematic diagram of the prior information module in an embodiment of this application.

[0031] Figure 3 This is a schematic diagram of the functional module pool in an embodiment of this application.

[0032] Figure 4 A schematic diagram of the feature fusion module in an embodiment of this application.

[0033] Figure 5 This is an overall network framework diagram of an embodiment of this application.

[0034] Figure 6This is a flowchart illustrating a specific implementation method of an embodiment of this application. Detailed Implementation

[0035] The following is in conjunction with the appendix Figure 1-6 This application will be described in further detail.

[0036] This application discloses a UAV remote sensing defogging method based on two-dimensional prior extraction and four-level adaptive collaborative fusion. (Refer to...) Figure 6 It includes the following steps: S1. Obtain non-uniform fog remote sensing images collected by UAVs, perform normalization preprocessing on them, and extract spatial fog concentration masks as spatial priors through semantic segmentation networks. At the same time, extract regional atmospheric light and scattering coefficients as physical priors through feature regression networks. S2. Based on the spatial prior and physical prior, construct a multi-path physical perception image signal processing operator to perform parallel processing on the input image and obtain a multi-dimensional enhanced feature map. S3. Input the multidimensional enhanced feature map into the feature adaptive enhancement fusion network, and realize the dynamic fusion of multi-path features through spatial feature modulation, receptive field adaptive allocation and attention weighting mechanism; S4. Input the fused features into the dehazing main network based on the encoder-decoder structure for reconstruction, and output a clear remote sensing image; The process begins with acquiring and constructing an uneven fog map dataset. The second step involves building a prior information module, a functional module pool, and a feature extraction module. The third step uses the constructed dataset to train the prior information module, functional module pool, and feature extraction and fusion module. The fourth step injects prior information into the functional module pool to improve the adaptability of the rendered image. The fifth step combines the prior information and the rendered image through the feature extraction and fusion module for image restoration and optimization. A joint loss function is used to optimize the feature extraction and fusion module and improve the model's performance. Finally, the optimal weights are used to output the image restoration and enhancement results, concluding the process.

[0037] Specifically, refer to Figures 1-5 The UAV performs remote sensing mapping or environmental monitoring tasks in complex environments (such as mountainous canyons, forest areas, and over water bodies), and its onboard camera module acquires aerial remote sensing images affected by uneven fog. The observation frames acquired by the camera module under complex weather conditions are denoted as foggy images. ,in Given two-dimensional pixel space coordinates, the tensor dimension of the input image is initialized to 1. Before entering the network, the image undergoes normalization preprocessing, linearly mapping its pixel values ​​to... The interval is defined to ensure the stability of subsequent network gradients. Based on the atmospheric scattering physical integral model, a linear domain fog image is generated. From the target clear image The direct attenuation and the superposition of atmospheric ambient light scattering are approximated as follows: in, This refers to the atmospheric light value (Airlight) for the entire region or a specific area. Transmission is the transmittance of the medium. With scene depth The atmospheric scattering coefficient β satisfies a strict exponential decay physical homomorphism: For the foggy image The paired physical prior features are defined as the physical representation of the scene state obtained jointly by a semantic segmentation network and a multi-channel feature extraction network. Specifically, a three-dimensional fog density spatial mask matrix is ​​extracted using a pre-trained DeepLabv3+ network. The process first captures multi-scale contextual information through the Hollow Spatial Pyramid Pooling (ASPP) module, and then calculates the element belonging to the first pixel using the Softmax function. Classification (no fog / light fog, moderate fog, dense fog, corresponding to...) The probability distribution of ) and the output mask tensor dimension are . : Simultaneously, to extract physical parameters, this invention improves the standard multi-channel ResNet18 network by removing its fully connected classification layers while retaining the feature extraction backbone. This network, combined with the aforementioned mask... After global average pooling (GAP) and custom multi-branch fully connected layer path mapping, the physical parameter vectors of three paths—no fog / light fog, medium fog, and dense fog—are regressed. The atmospheric light and scattering coefficients of three different fog concentration regions were matched. The spatiotemporal distribution of this prior feature was correlated with foggy images. Strict alignment makes it suitable as a guide signal for dynamic control by the ISP (Image Signal Processing) module. In this embodiment, if the scene's equivalent fog concentration changes drastically, the three types of fog concentrations corresponding to... and The degradation intensity of the scene will be dynamically quantified to provide clear physical boundary conditions for subsequent networks.

[0038] Specifically, the feature-adaptive dehazing network adopts a U-Net encoder-decoder structure as the backbone network and integrates a multi-path ISP feature processing module and a feature-adaptive enhancement fusion network (including SFT, RFB, CBAM and VMamba modules) to improve the model's ability to represent and dynamically control unevenly distributed fog.

[0039] The multi-channel ISP feature extraction module is mainly used to learn a multi-dimensional prior representation of degradation perception from the input hazy image. Let the input image be... This module constructs multi-path differentiable ISP operators (dehazing, noise reduction, white balance, and sharpening) to enable the network to explicitly capture and separate degradation patterns across different dimensions. The specific steps for obtaining multi-dimensional physical prior features using the aforementioned ISP module are as follows: The multi-path physical perception and feature adaptive modulation module is mainly used to explicitly capture and separate degradation patterns of different dimensions from the input non-uniform fog image, and to achieve efficient interaction between physical priors and deep network features. This part mainly consists of the following functional units working together: Multi-channel dynamic image signal processing (ISP) unit This unit constructs four parallel differentiable ISP operators for defogging, noise reduction, white balance, and sharpening. To adapt to the complex distribution of non-uniform fog, the system extracts regional physical parameters from the branches based on physical priors and dynamically calculates the parameter offsets of each operator through a nonlinear gating mechanism. Let the... The underlying anchor point parameters of the ISP operator are The physical parameter vector is Its dynamic correction process is expressed as: in, Represents a nonlinear mapping function. These are the corrected dynamic control parameters. Subsequently, the input image, combined with a fog density mask, is fed into the four ISP operators with corrected parameters, and the enhanced image sequences for specific physical degradation dimensions are output in parallel.

[0040] Shared feature dimensionality reduction and extraction unit To control the number of network parameters and computational redundancy, the aforementioned multi-path enhanced image sequences are uniformly fed into a lightweight feature extractor with shared parameters. This extraction unit performs dimensionality reduction from the high-dimensional pixel space to the low-dimensional feature manifold through shallow convolution operations. Its feature mapping process is expressed as follows: in, No. Road enhancement image, The shallow feature extraction operator representing the shared features outputs... To comprehensively characterize the deep features of luminosity and texture degradation information in various fog concentration areas.

[0041] Prior embedding and multi-scale receptive field modulation unit This unit aims to effectively fuse the extracted spatial concentration mask and physical parameters into deep features, and it includes two sub-modules: spatial feature modulation and receptive field adaptation. 3.1 Spatial Feature Modulation: Converting physical prior paths into corresponding feature scaling coefficients. With bias coefficient The aforementioned feature array is globally modulated in the form of an affine transformation: in, This represents the Hadamard product (element-by-element multiplication).

[0042] 3.2 Adaptive Receptive Field: Using a spatial fog concentration mask as a gating signal, receptive field branches of different scales are dynamically assigned to fog regions with different concentrations. Let the first... The mask distribution corresponding to the fog-like concentration is as follows The corresponding dilated convolution operator and its dilation rate are... Its multi-scale feature aggregation process is formulated as follows: This mechanism enables the activation of large-scale receptive fields in dense fog areas to preserve macroscopic contours, and the activation of small-scale receptive fields in light fog areas to capture microscopic details.

[0043] 4. Feature reweighting and global interaction unit This unit utilizes an attention mechanism, using a two-dimensional prior as a guiding signal for feature recalibration. A comprehensive attention weight matrix is ​​derived by jointly deducing the spatial and channel dimensions. The features that have undergone multi-scale modulation are reweighted: This fusion mechanism, while ensuring the overall network computational complexity remains controllable, achieves full interaction and filtering between local physical properties and global multi-path features, outputting high-quality fused features. This will directly serve the subsequent process of reconstructing clear images.

[0044] Next, we will introduce the feature adaptive enhancement fusion and dehazing main network.

[0045] The recalibrated multi-path feature array is then fed into a feature adaptive enhancement and fusion network, where a softmax activation function is used to generate pixel-level weight assignment moments for each feature path along the channel dimension. The final aggregation and fusion characteristics It is obtained by strictly summing the multi-path features according to their weights: The multi-level dehazing main network is built based on the U-Net encoder-decoder structure. In the main network, the input hazy image first passes through the first convolutional layer to perform preliminary feature extraction, obtaining initial shallow features. This process captures basic edge, texture, and color information while applying the LeakyReLU activation function to introduce a non-linear mapping. The computation can be represented as follows: This shallow feature is combined with the aforementioned multi-path fusion feature After being concatenated along the channel dimension, the data is fed into the U-Net's layer-by-layer downsampling encoder. The encoder contains consecutive residual refinement blocks (ResBlocks), each containing two... The residual calculation process for convolutional layers and nonlinear activation can be expressed as follows: A bottleneck layer is set in the core feature fusion region of the U-Net backbone network, and VMamba (Visual State Space Model) is introduced into this bottleneck layer as a global modeling module for core features. VMamba uses continuous-time linear state-space equations (SSM) for long-range dependency smoothing modeling. Its continuous computation process is defined as follows: To adapt to the characteristics of digital images, a zero-order hold based on step size is introduced. Discretize the continuous system to obtain the discrete state transition matrix. With input projection matrix Let the step size index of the discretized sequence be... The discretized state update mechanism can be rigorously described as follows: To address the lack of natural sequence orientation in two-dimensional images, VMamba employs a global state cross-scan mechanism. Let... This is a scan set containing four basic directions (top left to bottom right, bottom right to top left, top right to bottom left, and bottom left to top right). For along direction One-dimensional serialization unrolling operation, global modeling output Defined as inverse reconstruction fusion after mapping state space in each direction: This mechanism breaks through the limitations of a fixed receptive field, automatically smoothing the boundaries between fog-free / light fog, medium fog, and dense fog areas, eliminating the sense of regional discontinuity, and achieving... It has linear complexity.

[0046] The decoding end of the dehazing backbone network employs a multi-level structure based on PixelShuffle upsampling to progressively restore the spatial resolution of feature maps and reconstruct the final sharp image. The decoding operation can be represented as: During the decoding process, high-resolution features from the encoding front end are introduced into the decoding end through skip connections to compensate for the spatial information loss during downsampling.

[0047] To fully unleash the dehazing potential and physical constraint capabilities of feature-adaptive dehazing networks in complex environments, this invention abandons the limitations of a single loss function and constructs a joint loss function that integrates basic pixel reconstruction, high-frequency visual perception, implicit spatial contrastive learning, and macroscopic physical priors. During end-to-end model optimization, this joint loss function is used... The overall calculation formula for updating network weights is defined as follows: In the formula: The basic reconstruction loss is used to measure the pixel-level fidelity between the predicted dehazed image and the real sharp image, while preserving sharp image edges while preventing gradient oscillations. For visual perception loss, it is used to extract and constrain the high-frequency differences between the predicted image and the real image in the deep perception feature space, so as to prevent the smoothing distortion of the underlying texture required for mapping. To address the contrast loss in the feature dimension, the implicit features of the predicted image are made to approximate the fog-free positive samples and move away from the original foggy negative samples, thus solving the problem of local residual fog in uneven fog scenes. As a physical consistency loss, it calculates the consistency error between the simulated fog map reconstructed in reverse and the real input fog map by substituting the atmospheric physical prior parameters (such as atmospheric light and scattering coefficients) regressed by the network and the predicted dehazed image back into the atmospheric scattering model. This forces the network feature recombination process to conform to the physical degradation law of the real world and opens up the physical closed loop. , , , These are the weight coefficients for each loss term, used to balance the gradient contribution ratio of each constraint dimension during the model training phase.

[0048] This mechanism enforces the constraint that the feature recombination of the deep learning black box conforms to the physical degradation laws of the real world, greatly enhancing the model's generalization ability in extreme, unknown, and harsh environments.

[0049] Example This embodiment provides a UAV remote sensing defogging method based on two-dimensional prior extraction and four-level adaptive collaborative fusion, referring to... Figures 1-6 Specifically, it includes the following steps: Step 1: Constructing a non-uniform fog dataset based on a physical scattering model Acquiring fog-free remote sensing orthophotos The underlying depth tensor is obtained by bicubic interpolation alignment and maximum / minimum normalization of the corresponding depth map. Then, physically based rendering and tag assignment are performed according to the following mathematical model: (1) Spatial depth nonlinear optical correction Introducing gamma hyperparameters Nonlinear compensation is applied to the normalized depth to obtain the physical depth. : (2) Calculation of non-uniform transmittance and rendering of atmospheric scattering In the basic scalar scattering coefficient Superimposed spatial low-frequency noise matrix To characterize the non-uniform distribution of fog, pixel-by-pixel scattering coefficients are obtained. Combining physical depth with extracted local atmospheric light tensors Calculate spatial transmittance And forward synthesize foggy remote sensing images : (3) Absolute physical threshold prior label assignment Extract the transmittance The inverse mapping is based on a preset absolute physical hard threshold condition function. It is precisely discretized into a spatial category index mask containing multiple specific fog levels (such as light fog, medium fog, and dense fog). : This outputs a paired training dataset, the Used to provide spatial prior supervision signals for subsequent networks.

[0050] Step 2: Online Reasoning and Two-Dimensional Space - Physics Prior Extraction Acquire the non-uniform fog remote sensing image to be processed And perform basic data normalization operations on it. The system starts the two-dimensional prior extraction branch in parallel, specifically including: (1) The spatial prior extraction branch uses a pre-trained semantic segmentation network as a spatial feature extractor. The normalized remote sensing image is input into the network, and after being processed by the feature extraction and multi-scale context capture module, the probability distribution map of each pixel corresponding to different fog concentration levels (such as no fog / light fog, medium fog, dense fog) is output through the activation function, and the spatial concentration mask matrix is ​​extracted. The mapping relationship of this process can be represented as: In the formula, This represents the forward mapping operation of the semantic segmentation network.

[0051] (2) The physical prior extraction branch uses a feature regression backbone network with the classification head removed to extract the physical state solution. The remote sensing image... With the extracted mask matrix After concatenation and fusion along the feature channel dimension, the data is input into the regression network. The feature data is sequentially compressed from two dimensions to one dimension through global spatial pooling, and then regressed through multiple sets of parallel fully connected mapping layers to obtain atmospheric physical parameter vectors matching the corresponding fog concentration region. This vector contains the atmospheric light and scalar scattering coefficients for each region, and this process is expressed as: In the formula, This indicates a feature concatenation operation. This represents the mapping and dimensionality reduction operations of the feature regression backbone network. The extracted... and Together, they provide macroscopic numerical boundary conditions and prior guidance for the feature modulation of subsequent networks. Step 3: Construction of Multi-Branch Dynamic Routing and Feature-Adaptive Defogging Network 3.1 Physics-Driven Construction of Dynamic Operator Pools and Parallel Computation The system is built with a processing pool containing multiple physical sensing operators (such as defogging, noise reduction, white balance, and sharpening). For the first... The path operator has corresponding underlying basic anchor point parameters preset by the system. Receive the physical parameter vector output from step 2. Then, the parameter offset is calculated through a nonlinear mapping network, and the anchor point parameters of each operator are dynamically corrected. The adaptive control process of its physical parameters is expressed as follows: in, Represents a nonlinear mapping function. The corrected dynamic parameters are then used to input the non-uniform fog image, combined with a spatial mask, into four parallel regions into the four-way operators after parameter correction, generating four enhanced feature maps with different physical dimensions. Further, these enhanced feature maps are fed into a shared lightweight convolutional module for dimensionality reduction and refinement, resulting in multi-way parallel feature blocks.

[0052] 3.2 Spatial Feature Transformation and Adaptive Modulation of Receptive Field By utilizing a spatial feature transformation mechanism, the physical parameter vector is mapped pathwise to a channel-wise scaling scalar. With bias scalar Multipath feature blocks are transformed in the form of affine transformation. Perform global modulation: in, This represents the Hadamard product, which is then fed into the receptive field adaptive modulation module, where the extracted spatial mask serves as a gating signal to guide the feature flow. The system dynamically assigns convolutional branches of different scales to pixels with different fog concentrations, and its feature aggregation process is formulated as follows: In the formula, For the corresponding number Spatial mask signal for fog-like concentration levels, This represents a specific scale convolutional operation that is adaptively assigned (e.g., activating a large-scale receptive field for dense fog areas and a small-scale receptive field for light fog areas). The aggregated features are further recalibrated using an attention mechanism to output multi-path refined features.

[0053] 3.3 Dynamic Weight Adaptive Fusion and Serialization Global Modeling The aforementioned multi-path refined features are concatenated along the channel dimension, and a pixel-level attention weight allocation matrix is ​​calculated using feature mapping and a normalized exponential function (Softmax). Perform weighted summation on a pixel-by-pixel basis and merge them into a unified main feature. : In the formula, The total number of characteristic branches (in this embodiment) ), For the first The road is characterized by its refined features.

[0054] The shallow features of the original image are concatenated with the fused master feature and then input into the encoding / decoding reconstruction network. To overcome the computational bottleneck of high-resolution remote sensing image processing, a visual state space module is deployed in the core fusion region of the network: this module unfolds the two-dimensional feature map into a one-dimensional sequence along multiple spatial directions. Furthermore, it employs discretized state-space equations for extremely fast global scan smooth modeling, and its discrete state transition mechanism is rigorously described as follows: In the formula, For implicit state variables, and These are the discretized state transition matrix and the input projection matrix, respectively. The scan outputs features. After scanning, a two-dimensional feature map is reconstructed through inverse unrolling. Finally, the resolution is restored by the subpixel upsampling array at the decoding end, outputting a clear, dehazed predicted remote sensing image.

[0055] Step 4: End-to-end joint optimization and physical consistency closed loop During the network training and optimization phases, a gradient-based optimization algorithm is used to update model parameters, and a joint loss function system is constructed covering basic reconstruction, visual perception, implicit contrast, and macroscopic physical constraints. The joint loss function... The overall computational framework is defined as follows: In the formula, The weighting coefficients for each constraint dimension are adjusted. The specific calculation logic for each joint sub-item is as follows: (1) Basic Reconstruction Smoothing Constraints: To ensure the feature point matching rate of the dehazed remote sensing image in subsequent 3D mapping software, basic reconstruction constraints with smoothing factors are constructed. This item is used to measure the predicted dehazed image. With real and clear images To minimize pixel-level errors and prevent gradient oscillations in the later stages of training: in, It is a very small constant smoothing factor.

[0056] (2) High-frequency visual perception constraints: To prevent excessive smoothing and distortion of the underlying land cover texture required for remote sensing mapping, a pre-trained perceptual feature extraction network is used. Calculate the difference in high-frequency distribution between the predicted image and the real image in the deep feature space: In the formula, The feature network is represented by the first Activation feature maps of the layer These correspond to the number of channels, height, and width of the feature map, respectively.

[0057] (3) Implicit contrast enhancement constraint: To address the oversimplification problem of residual haze in deep learning models under complex environments, a contrast constraint is constructed for the feature dimension. Through implicit feature encoder Zoom in on the predicted image and the positive sample (real, clear image). The spatial distance, while strongly rejecting the predicted image from negative samples (input with fog). Distance: In the formula, Represents the distance metric operator in the implicit feature space.

[0058] (4) Physical consistency closed-loop constraint: To open up the physical prior closed loop, the prior physical parameters (including atmospheric light and scattering coefficients) obtained by regression in the intermediate steps of the network are combined with the clear image of the predicted output. By substituting differentiable mappings back into the atmospheric scattering rendering formula, a simulated reconstruction of the fog map is derived in reverse. Then, its comparison with the real-world input fog map is calculated. Consistency error: This mechanism enforces that the feature reconstruction process of the deep learning black box must conform to the physical and optical degradation laws of the real world, significantly enhancing the model's generalization ability and reconstruction reliability in extreme, unknown, and harsh environments.

[0059] Through the end-to-end feature adaptive dehazing process described above, a clear remote sensing image is finally output.

[0060] The technical innovations of this application are: 1. Traditional synthesis methods often rely on the relative brightness of the image to determine the fog density, and the degradation distribution is too uniform, which makes it easy to produce "label semantic drift" under different lighting scenarios; The technical solution of this application establishes "absolute physical transmittance" as a unified hard threshold classification mechanism based on the underlying optical formula, and introduces low-frequency spatial perturbations (such as non-uniform noise) to simulate physical-level degradation affected by wind. This mechanism eliminates the semantic drift problem at the source and provides the network with objective, rigorous, and environmentally independent high-fidelity physical prior supervision signals.

[0061] 2. Existing dehazing algorithms often fall into the dilemma of "global uniform parameter failure" or "blind speculation of local pixels" when estimating degradation parameters, lacking macroscopic physical edges; The technical solution of this application proposes a priori extraction architecture that decouples spatial semantics and optical physics in two dimensions. Through a dual-path collaborative diagnostic mechanism of "discrete semantic masking for region determination and continuous physical parameter quantification", it perfectly solves the adaptation contradiction between global parameter failure and uneven local water vapor distribution in complex scenarios, and achieves high-precision degradation state assessment. 3. Traditional deep learning dehazing presents a pure black box pixel mapping, which is very easy to destroy the underlying mapping-grade physical texture of remote sensing images and even cause local color distortion. The technical solution of this application constructs a multi-dimensional differentiable ISP operator processing pool driven by physical priors to dynamically correct underlying control parameters; at the same time, it combines spatial masking to adaptively allocate multi-scale receptive fields. This solution deeply integrates data-driven efficient perception with the reliability of physical rules, maximizing the protection of high-frequency edge sharpness and physical color fidelity while accurately dehazing.

[0062] 4. Existing technologies for long-range dependency modeling of high-resolution images often face the pain point of explosive computational complexity, and pure data-driven black boxes are prone to deviating from the real physical degradation laws. The technical solution of this application deploys a visual state space model in the core area, breaks through the computing power bottleneck through ultra-fast linear global scanning, and effectively smooths the sense of separation at the boundary of different fog areas; at the same time, it creates a unique physical consistency closed-loop constraint, which forces the network to reverse reconstruct and verify the regressed atmospheric parameters and the predicted clear map, strictly locking the feature recombination boundary with real physical laws, and significantly enhancing the generalization ability in extreme environments.

[0063] The implementation principle of the UAV remote sensing dehazing method based on two-dimensional prior extraction and four-level adaptive collaborative fusion in this application is as follows: By introducing a two-dimensional collaborative mechanism of spatial and physical priors, the fog concentration distribution and corresponding atmospheric physical parameters in non-uniform fog images are first modeled at the regional level, enabling the network to have a fine perception capability for complex fog scenes. On this basis, multi-path physical perception image signal processing operators are used to decouple and enhance different degradation dimensions of the image in parallel, and the operator parameters are adaptively adjusted through nonlinear mapping to match the processing process with the actual physical degradation process. Subsequently, through mechanisms such as spatial feature modulation, adaptive allocation of receptive field, and attention reweighting, the physical prior is effectively embedded into the deep feature expression to achieve dynamic fusion of multi-scale and multi-dimensional features. Furthermore, the U-Net structure and VMamba global modeling capability are combined in the main dehazing network to achieve the unification of local detail restoration and global consistency. At the same time, by constructing a joint loss function that includes reconstruction, perception, contrast, and physical consistency, physical closed-loop constraints are established to make the network output result conform to the real atmospheric scattering law. Finally, high-quality dehazing and enhancement of non-uniform fog remote sensing images in complex environments are achieved.

[0064] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A UAV remote sensing defogging method based on two-dimensional prior extraction and four-level adaptive collaborative fusion, characterized in that, Includes the following steps: S1. Obtain non-uniform fog remote sensing images collected by UAVs, perform normalization preprocessing on them, and extract spatial fog concentration masks as spatial priors through semantic segmentation networks. At the same time, extract regional atmospheric light and scattering coefficients as physical priors through feature regression networks. S2. Based on the spatial prior and physical prior, construct a multi-path physical perception image signal processing operator to perform parallel processing on the input image and obtain a multi-dimensional enhanced feature map. S3. Input the multidimensional enhanced feature map into the feature adaptive enhancement fusion network, and realize the dynamic fusion of multi-path features through spatial feature modulation, adaptive allocation of receptive field and attention weighting mechanism; S4. Input the fused features into the dehazing main network based on the encoder-decoder structure for reconstruction, and output a clear remote sensing image.

2. The method according to claim 1, characterized in that: The spatial prior is extracted using a DeepLabv3+ network, and multi-scale contextual information is used to generate probability distribution masks for different fog concentration levels.

3. The method according to claim 1, characterized in that: The physical priors are extracted using an improved ResNet network, which removes the classification layer and regresses the atmospheric light and scattering coefficients corresponding to different fog concentration regions through global average pooling and a multi-branch fully connected structure.

4. The method according to claim 1, characterized in that: The multi-channel physical sensing image signal processing operators include a dehazing operator, a noise reduction operator, a white balance operator, and a sharpening operator. The parameters of each operator are dynamically adjusted based on physical priors through nonlinear mapping.

5. The method according to claim 4, characterized in that: The dynamic adjustment is achieved by inputting the physical parameter vector into a nonlinear function to generate parameter offsets, and then correcting the basic parameters of each operator to achieve adaptive processing for different fog concentration regions.

6. The method according to claim 1, characterized in that: The feature adaptive enhancement fusion network includes a spatial feature transformation module, a receptive field adaptive module, and an attention reweighting module. in: The spatial feature transformation module performs scaling and bias modulation on the features through affine transformation; The receptive field adaptive module dynamically allocates convolutional kernels of different scales based on the fog concentration mask; The attention reweighting module performs spatial and channel-dimensional weighted optimization on the fused features.

7. The method according to claim 6, characterized in that: The feature adaptive enhancement fusion network further includes a four-level collaborative fusion structure consisting of SFT, RFB, CBAM and VMamba, used to achieve multi-scale feature enhancement and global consistency optimization.

8. The method according to claim 1, characterized in that: The dehazing main network adopts a U-Net encoder-decoder structure and fuses high-resolution feature information through skip connections during the encoding and decoding process.

9. The method according to claim 8, characterized in that: The defogging main network introduces a VMamba module at the bottleneck layer, and performs multi-directional serialization scanning of features through a state-space model to achieve global long-range dependency modeling and regional boundary smoothing.

10. The method according to claim 1, characterized in that: The decoding stage uses a PixelShuffle-based upsampling method to restore spatial resolution step by step, and outputs the final dehazed image; During model training, a joint loss function is constructed, including basic reconstruction loss, visual perception loss, contrastive learning loss, and physical consistency loss. The physical consistency loss is achieved by substituting the predicted clear image and physical prior parameters into the atmospheric scattering model to generate a simulated fog image, and then calculating the error between the simulated fog image and the input fog image to constrain the network to conform to the physical degradation law. It also includes the step of building a training dataset based on a physical scattering model, generating fogged images by introducing spatial non-uniform scattering coefficients and depth information, and generating fog concentration labels based on transmittance thresholds; The transmittance is calculated based on depth information and scattering coefficient, and is divided into multiple fog concentration levels by a preset physical threshold, which is used to supervise the training of the spatial prior extraction network.

Images