A method and device for polarization-aware adaptive dehazing based on autonomous cues
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2025-10-20
- Publication Date
- 2026-06-02
Smart Images
Figure CN121437318B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, specifically relating to a method and apparatus for adaptive dehazing based on autonomous prompting polarization sensing. Background Technology
[0002] In recent years, image dehazing technology has made significant progress in the field of computer vision, especially with the advancement of deep learning frameworks such as DehazeNet and FFA-Net, achieving preliminary restoration of haze-degraded images. However, in practical outdoor applications, especially in complex weather conditions with low visibility such as haze, dust, rain, and fog, traditional methods face the problem of drastic performance degradation.
[0003] Particulate matter in the atmosphere causes severe Rayleigh and Mie scattering, resulting in low contrast, color shift, and loss of visibility in images. This complex degradation not only reduces image quality but also directly undermines the ability of deep learning models to recover key scenes. To address this issue, researchers have begun to combine polarization imaging, physical modeling, and image dehazing. For example, they preprocess polarization information before recovery or use multimodal data (such as depth maps) to improve dehazing robustness.
[0004] Existing image dehazing techniques can be broadly categorized into physical model-based methods and deep learning-based methods. Physical model-based methods typically rely on atmospheric scattering models, requiring explicit estimation of parameters such as transmittance, scattering coefficient, and atmospheric light intensity. While effective under ideal conditions, these methods often fail in complex real-world environments due to the invisibility of the sky, difficulty in obtaining prior knowledge, or inaccurate parameter estimations, resulting in unsatisfactory dehazing effects. Deep learning-based methods, although capable of learning haze degradation patterns from large datasets, lack physical interpretability and generally suffer from insufficient generalization and robustness when dealing with non-uniform scattering, multi-scale haze layers, and complex lighting conditions. Particularly in utilizing polarization information, existing methods typically require explicit calculation of transmitted light polarization (PT), atmospheric light polarization (PA), and global atmospheric light polarization (P). The calculation is complex and prone to error accumulation, making it difficult to adapt to the changing atmospheric scattering environment.
[0005] Therefore, how to provide a defogging method that improves generalization ability and robustness in complex environments has become an urgent problem to be solved. Summary of the Invention
[0006] To address the aforementioned problems in the prior art, this invention provides a method and apparatus for adaptive defogging based on autonomous prompting polarization sensing.
[0007] The technical problem to be solved by this invention is achieved through the following technical solution:
[0008] In a first aspect, the present invention provides a method for polarization-sensing adaptive dehazing based on autonomous prompting, comprising:
[0009] Acquire polarization input feature image and intensity input image of the same region;
[0010] Based on Fourier polarization modulation, a descriptor is obtained from the polarization input feature image;
[0011] The descriptor is projected into a latent space with a set scale to obtain a descriptor at scale s, and a reparameterized cue word is obtained based on a cue library based on polarization characteristics and the descriptor at scale s.
[0012] The intensity input image is input into a cue enhancement U-Net module based on a cross-attention mechanism to obtain a dehazed image based on the reparameterized cue words and the intensity input image.
[0013] Secondly, the present invention also provides an apparatus for adaptive defogging based on autonomous prompting polarization sensing, comprising:
[0014] The image acquisition module is used to acquire polarization input feature images and intensity input images of the same region;
[0015] A polarization module is used for Fourier-based polarization modulation to obtain a descriptor based on the polarization input feature image;
[0016] The parameterization module is used to project the descriptor into a latent space with a set scale to obtain a descriptor at scale s, and to obtain a reparameterized cue word based on a cue library based on polarization characteristics and the descriptor at scale s.
[0017] The dehazing module is used to input the intensity input image into the cue enhancement U-Net module based on the cross-attention mechanism, so as to obtain the dehazed image according to the reparameterized cue words and the intensity input image.
[0018] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0019] This invention proposes a polarization-aware adaptive dehazing method based on autonomous prompts. This method is based on Fourier polarization modulation, obtaining descriptors from the polarization input feature image and projecting these descriptors into a latent space with a defined scale, resulting in descriptors at scale *s*. Reparameterized prompts are then derived based on a prompt library based on polarization characteristics and the descriptors at scale *s*. Finally, the intensity input image is input into a prompt enhancement U-Net module based on a cross-attention mechanism to obtain the dehazed image based on the reparameterized prompts and the intensity input image. Therefore, this invention combines Fourier polarization perception with a multi-scale prompt injection mechanism to simulate the light scattering and polarization process in haze. Through dynamic global descriptor selection and reparameterized prompts, it achieves adaptive modeling for different haze conditions. By injecting polarization prompts through cross-attention, the network not only relies on local convolutional features during reconstruction but also possesses global physical prior perception capabilities. This improves scene recoverability and dehazing accuracy, constructing a highly robust image processing method for extreme atmospheric environments and enhancing generalization ability and robustness in complex environments.
[0020] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description
[0021] Figure 1 This is a flowchart illustrating a polarization-sensing adaptive defogging method based on autonomous prompting, provided in an embodiment of the present invention.
[0022] Figure 2 This is a schematic diagram of the overall architecture of a polarization-sensing adaptive defogging method based on autonomous prompting provided in an embodiment of the present invention;
[0023] Figure 3 This is a schematic diagram of an architecture of a Fourier-based polarization sensor (FPP) provided in an embodiment of the present invention;
[0024] Figure 4 This is a schematic diagram of an architecture of a cue enhancement U-Net based on multi-scale cross-attention provided by an embodiment of the present invention;
[0025] Figure 5 This is a schematic diagram illustrating a visual effect provided by an embodiment of the present invention;
[0026] Figure 6 This is a schematic diagram illustrating the polarization image calculation effect provided in an embodiment of the present invention;
[0027] Figure 7 This is a comparison diagram of visual effects under different fog conditions provided by an embodiment of the present invention;
[0028] Figure 8This is a schematic diagram of a polarization-sensing adaptive defogging device based on autonomous prompting, provided in an embodiment of the present invention. Detailed Implementation
[0029] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.
[0030] Example 1
[0031] Currently, the more advanced technical approaches include traditional polarization dehazing methods, learning-based polarization dehazing networks, and methods that integrate prior knowledge or data-driven approaches.
[0032] (1) Traditional polarization defogging method.
[0033] A typical approach is the polarization separation method based on Malus's law, which treats transmitted light as a physical quantity without polarization information and estimates transmitted light and atmospheric light by utilizing the polarization difference between air light and scene radiation. While this type of method improves physical interpretability, its applicability in complex real-world scenes is limited due to its reliance on idealized assumptions (such as known sky regions or global atmospheric light directions) and the need for manual priors (such as labeling reference objects or depth maps). Furthermore, it is susceptible to non-uniform scattering, which can amplify parameter estimation errors.
[0034] (2) Learning-based polarization dehazing network.
[0035] Previous studies have attempted to incorporate deep learning structures into dehazing models to enhance polarization information processing, using several U-Net sub-networks to explicitly estimate transmitted light polarization (PT), atmospheric light polarization (PA), and global atmospheric light polarization (P). This method, however, only optimizes from the perspective of parameter regression and fails to implicitly model the frequency-dependent attenuation caused by haze. Therefore, it remains unstable under conditions of extremely low visibility or mixed material surfaces.
[0036] 2.2.3 Integrating Prior or Data-Driven Approaches
[0037] Some methods attempt to use GANs (such as CycleGAN) or Transformers (such as Dehazeformer) to guide dehazing, but due to cost and physical adaptability limitations, they are not yet widely deployable in polarization scenarios.
[0038] First, in natural outdoor environments, haze and smog severely obscure scene details in images due to their high-intensity atmospheric scattering and particle reflection. Traditional dehazing methods based on dark channels or GANs struggle to extract effective polarization information, leading to unstable model outputs and high color distortion rates. Second, scenes in haze exhibit significant contrast reduction and boundary blurring, especially on non-uniform fog layers or surfaces of mixed materials, where visibility drops sharply due to scattering, significantly degrading the recovery range and accuracy of existing dehazing algorithms. Furthermore, due to the frequency-dependent attenuation and spatial distribution uncertainty caused by haze and smog, existing models often employ fixed physical assumptions or static parameters, lacking adaptive polarization modeling mechanisms, resulting in weak perception of different scattering intensities. Finally, most existing dehazing models lack implicit encoding of the physical models of real atmospheric degradation; their feature processing relies entirely on data fitting and deep learning, lacking clear physical interpretation and control mechanisms. This leads to poor model controllability, high deployment risks, and difficulty in widespread application in scenarios with high reliability requirements, such as intelligent monitoring and autonomous driving.
[0039] Therefore, please see Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating a polarization-sensing adaptive dehazing method based on autonomous prompting, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall architecture of a polarization-sensing adaptive dehazing method based on autonomous prompting, provided by an embodiment of the present invention. The present invention provides a polarization-sensing adaptive dehazing method based on autonomous prompting, which includes:
[0040] Step 1: Obtain the polarization input feature image and intensity input image of the same region.
[0041] Specifically, the intensity input image is the RGB image acquired by the camera, and the polarization input feature image is the image obtained through a polarization camera, that is, the polarization image obtained after polarization processing of the RGB image. Polarization input feature image , For the set of real numbers, The number of channels when inputting. For the number of channels, For height, For width.
[0042] Step 2: Based on Fourier polarization modulation, obtain descriptors from the polarization input feature image.
[0043] In an optional embodiment, please refer to Figure 3 Step 2 may include:
[0044] Step 2.1: Process the polarization input feature image through two convolutional layers to obtain the convolutional features.
[0045] Specifically, the polarization input feature image is processed sequentially by two convolutional layers in the spatial domain to extract intermediate features, and the features output by the second convolutional layer are the convolutional features.
[0046] Step 2.2: Perform Fourier transform (FFT) on the convolutional features to obtain the frequency domain modulation features of all channels.
[0047] Specifically, a Fourier transform is performed on the convolutional features, and the convolutional features are converted to the frequency domain for target modulation to obtain all frequency domain modulated features.
[0048] Here, the characteristics of frequency domain modulation are expressed as:
[0049]
[0050] in, This is a characteristic of frequency domain modulation. This is the index of the channel converted from the channel. For frequency domain variables, For Fourier transform, For the first The entire two-dimensional feature map of each channel.
[0051] Step 2.3: Obtain the characteristics of the modulation spectrum based on the characteristics of frequency domain modulation.
[0052] Specifically, modulation is achieved through a learnable spectral filter that dynamically adjusts frequency components to simulate anisotropic scattering attenuation effects. This filter employs a complex-valued weighted approximation of the MTF (Mean Transmission Factor). Its parameters are adaptively adjusted to reflect the band-limited scattering effect between different channels.
[0053] Here, the characteristics of the modulation spectrum are represented as follows:
[0054]
[0055] in, The characteristics of the modulation spectrum, For learnable complex weights, approximately... .
[0056] Step 2.4: Perform inverse Fourier transform (iFFT) on the features of the modulation spectrum to obtain the reconstructed features.
[0057]
[0058] in, Features for reconstruction.
[0059] Step 2.5: Obtain the global features based on the set of frequency domain modulation features of all channels and the reconstructed features.
[0060] Here, the global features are represented as:
[0061]
[0062] in, This is a global feature, which implicitly embeds global scattering information. It is the set of frequency domain modulation characteristics for all channels. As a learnable mixing factor, , Scaling factor and Both methods optimize the balance between the original features and the modulated features through adaptive adjustment.
[0063] Step 2.6: Process the global features sequentially through global pooling and multilayer perceptron (MLP) to generate descriptors.
[0064] Here, the descriptor is represented as:
[0065]
[0066] in, For descriptors, For spatial dimensions, That is .
[0067] In this embodiment, the descriptor As transmitted polarization (PT), atmospheric polarization (PA), and background light ( The latent embedding of physical parameters such as ) can eliminate the need to explicitly estimate these parameters.
[0068] Step 3: Project the descriptor into the latent space with the scale set to obtain the descriptor at scale s, and obtain the reparameterized cue words based on the cue library based on polarization characteristics and the descriptor at scale s.
[0069] In an optional embodiment, step 3 may include:
[0070] Step 3.1: Establish a cue library based on polarization characteristics based on multi-scale polarization cue words, where multi-scale polarization cue words include atmospheric light polarization, transmitted polarization, and background light.
[0071] Specifically, in atmospheric scattering, polarization characteristics are essentially determined by parameters such as atmospheric light polarization, transmitted polarization, and background light. Traditional dehazing methods attempt to explicitly estimate these parameters, but this approach is often error-prone and struggles to maintain robustness in uncontrolled environments. To avoid the explicit regression problem while preserving physical interpretability, this embodiment designs a hint library based on polarization characteristics, including PA, PT, and... The underlying abstract encoding is a learnable tag. Specifically, for each scale, the cue library is represented as:
[0072]
[0073] in, For the prompt library, The total number of physical prior abstractions. To indicate the length, For scale Feature dimensions, For the first k A learnable label of physical prior abstraction. The illumination prior represented by atmospheric light polarization. The transmission scattering prior is represented by transmission polarization. The background light represents the prior environment.
[0074] Step 3.2: Project the descriptor onto the latent space with a set scale using a mapping function to obtain the descriptor at scale s. The mapping function is implemented using a lightweight two-layer multilayer perceptron and ReLU is used as the activation function.
[0075] Here, the descriptor at scale s is represented as:
[0076]
[0077] in, For a descriptor at scale s, the mapping function is... Implemented as a lightweight two-layer multilayer perceptron (MLP), it employs the ReLU activation function and sets parameters independently for each scale. This design ensures that the frequency-aware global descriptor can be adaptively projected into the latent space at a specific scale without parameter sharing, thus enabling each scale to capture unique scattering behavior features.
[0078] Step 3.3: Use the softmax function to process the descriptors and weights at scale s to obtain adaptive selection weights.
[0079] Specifically, the global frequency cues are decomposed into local representations at multiple feature scales, which are then used to calculate adaptive selection weights, expressed as follows:
[0080]
[0081] in, for k An adaptive selection weight based on a physical prior abstraction. for k The weight of a physical prior abstraction.
[0082] Step 3.4: Based on the cue library, adaptively select descriptors under weight and scale s to obtain reparameterized cue words.
[0083] Here, the reparameterized prompt words are represented as:
[0084]
[0085] in, The prompt words are for reparameterization.
[0086] This framework allows each input to activate a unique set of physics-based cues, essentially acting as PA, PT, and Implicit substitution. Unlike traditional methods that require explicit parameter inversion, this embodiment treats these variables as learnable latent carriers, carrying polarization prior knowledge. This mechanism provides a robust, differentiable, and interpretable guiding framework for multi-scale representation learning, reducing reliance on heuristic priors while seamlessly adapting to various scattering conditions.
[0087] Step 3.5: Construct the mutual information between the reparameterized cue words and the multiscale polarization cue words, and maximize the mutual information so that the maximized mutual information is greater than or equal to the variational lower bound.
[0088] Specifically, to further ensure physical interpretability, this embodiment analyzes the learned reparameterized cue words. With underlying physical variables Mutual information between them. Specifically, maximizing mutual information:
[0089]
[0090] in, For mutual information, To Variational inference, specifically the lower bound of evidence ELBO, q It is a variational approximation. By enforcing this auxiliary objective during training, we ensure that the cue words and the underlying polarization cues maintain mutual information, thereby avoiding mode collapse and ensuring that cross attention injects semantically and physically consistent guidance into the feature map.
[0091] Step 4: Input the intensity input image into the cue enhancement U-Net module based on the cross-attention mechanism to obtain a dehazed image based on the reparameterized cue words and the intensity input image.
[0092] Optionally, the enhanced U-Net module includes an encoder, a bottleneck layer, and a decoder.
[0093] In an optional embodiment, please refer to Figure 4 Step 4 may include:
[0094] Step 4.1: Based on the cross-attention mechanism, cross-attention features are obtained from the label sequence of features obtained in the encoder of the intensity input image and the reparameterized prompt words. A feature map is obtained based on the cross-attention features and the label sequence.
[0095] Optionally, the encoder includes a first encoding unit, a second encoding unit, a third encoding unit, and a fourth encoding unit.
[0096] Step 4.11: Input the intensity input image into the first coding unit of the encoder and perform convolution and downsampling processing in sequence to obtain the first coding feature. Flatten the first coding feature into a label sequence to obtain the first label sequence.
[0097] Specifically, the first coding unit sequentially performs convolution and downsampling on the intensity input image to obtain the first coding feature, and then uses the torch.flatten() function to flatten the first coding feature into the first label sequence.
[0098] Here, the convolution of the intensity input image is represented as: Downsampling is represented as: , This is the first coding feature. This is the first convolutional layer. For downsampling, As the activation function, the first labeled sequence obtained after flattening is represented as: , .
[0099] Step 4.12: Based on the cross-attention mechanism, obtain the first cross-attention feature according to the reparameterized cue words, the cue words complementary to the reparameterized cue words, and the first tag sequence.
[0100] Here, the first cross-attention feature is represented as:
[0101]
[0102] in, This represents the first cross-attention feature at scale s. Multihead indicates multi-head cross-attention, with Q, K, and V as the basic operation elements. The first labeled sequence, To and Complementary cue words, These represent two prompt words carrying different information. and Together they form the entire hint library.
[0103] Step 4.13: Obtain the first enhanced feature based on the first cross-attention feature and the first label sequence, and reshape the first enhanced feature into the first feature map.
[0104] Here, the first enhanced feature is represented as:
[0105]
[0106] Here, the first feature map is represented as:
[0107]
[0108] in, This is the first enhanced feature at scale s. This is the first feature map at scale s. The function is a reshaping function.
[0109] Step 4.14: Input the first coding feature into the second coding unit and perform convolution and downsampling processing in sequence to obtain the second coding feature, and flatten the second coding feature into a label sequence to obtain the second label sequence.
[0110] Specifically, the second coding unit sequentially performs convolution and downsampling on the first coding feature to obtain the second coding feature, and then uses the torch.flatten() function to flatten the second coding feature into the second label sequence.
[0111] Here, the convolution representation of the first encoded feature is as follows: Downsampling is represented as: , This is the second coding feature. This is the second convolutional layer.
[0112] Step 4.15: Based on the cross-attention mechanism, obtain the second cross-attention feature according to the reparameterized cue words, the cue words complementary to the reparameterized cue words, and the second tag sequence.
[0113] Here, the second cross-attention feature is represented as:
[0114]
[0115] in, This is the second cross-attention feature. This is the second marker sequence.
[0116] Step 4.16: Obtain the second enhanced feature based on the second cross-attention feature and the second label sequence, and reshape the second enhanced feature into the second feature map.
[0117] Here, the second enhancement feature is represented as:
[0118]
[0119] Here, the second feature map is represented as:
[0120]
[0121] in, This is the second enhanced feature at scale s. This is the second feature map at scale s.
[0122] Step 4.17: Input the second coding feature into the third coding unit and perform convolution and downsampling processing in sequence to obtain the third coding feature. Flatten the third coding feature into a label sequence to obtain the third label sequence.
[0123] Specifically, the third coding unit sequentially performs convolution and downsampling on the second coding features to obtain the third coding features, and then uses the torch.flatten() function to flatten the third coding features into the third label sequence.
[0124] Here, the convolution representation of the second encoded feature is as follows: Downsampling is represented as: , This is the third coding feature. This is the third convolutional layer.
[0125] Step 4.18: Based on the cross-attention mechanism, obtain the third cross-attention feature according to the reparameterized cue words, the cue words complementary to the reparameterized cue words, and the third tag sequence.
[0126] Here, the third cross-attention feature is represented as:
[0127]
[0128] in, This is the third cross-attention feature. This is the third marker sequence.
[0129] Step 4.19: Obtain the third enhanced feature based on the third cross-attention feature and the third label sequence, and reshape the third enhanced feature into the third feature map.
[0130] Here, the third enhancement feature is represented as:
[0131]
[0132] Here, the third feature map is represented as:
[0133]
[0134] in, This is the third enhancement feature at scale s. This is the third feature map at scale s.
[0135] Step 4.110: Input the third coding feature into the fourth coding unit and perform convolution and downsampling processing in sequence to obtain the fourth coding feature. Flatten the fourth coding feature into a label sequence to obtain the fourth label sequence.
[0136] Specifically, the fourth coding unit sequentially performs convolution and downsampling on the third coding feature to obtain the fourth coding feature, and then uses the torch.flatten() function to flatten the fourth coding feature into the fourth label sequence.
[0137] Here, the convolution representation of the third encoded feature is as follows: Downsampling is represented as: , This is the fourth coding feature. This is the fourth convolutional layer.
[0138] Step 4.111: Based on the cross-attention mechanism, obtain the fourth cross-attention feature according to the reparameterized cue words, the cue words complementary to the reparameterized cue words, and the fourth label sequence.
[0139] Here, the fourth cross-attention feature is represented as:
[0140]
[0141] in, This is the fourth cross-attention feature. This is the fourth marker sequence.
[0142] Step 4.112: Obtain the fourth enhanced feature based on the fourth cross-attention feature and the fourth label sequence, and reshape the fourth enhanced feature into the fourth feature map.
[0143] Here, the fourth enhancement feature is represented as:
[0144]
[0145] Here, the fourth feature map is represented as:
[0146]
[0147] in, This is the fourth enhancement feature at scale s. This is the fourth feature map at scale s.
[0148] Step 4.2: The encoded features output by the encoder are transmitted to the decoder after passing through the bottleneck layer, so as to obtain the dehazed image based on the feature map obtained in the encoder and the encoded features transmitted to the decoder through the bottleneck layer.
[0149] Optionally, the decoder includes a first decoding unit, a second decoding unit, and a third decoding unit.
[0150] Step 4.21: Perform convolution on the fourth coding feature through the bottleneck layer to obtain the fifth coding feature.
[0151] Optionally, the bottleneck layer can be a single convolutional layer.
[0152] Here, the fifth coding feature is represented as:
[0153]
[0154] in, This is the fifth coding feature. This is set as the bottleneck layer for the convolutional layer.
[0155] Step 4.22: Input the fifth encoded feature and the second feature map into the first decoding unit, perform upsampling processing on the fifth encoded feature, and concatenate the upsampled fifth encoded feature and the second feature map to obtain the first concatenated feature.
[0156] Step 4.23: Perform a convolution operation on the first concatenated feature to obtain the first decoded feature.
[0157] Here, the first decoded feature is represented as:
[0158]
[0159] in, This is the first decoding feature. For upsampling operation, For splicing operations, This is a convolution operation.
[0160] Step 4.24: Input the first decoded feature and the third feature map into the second decoding unit, perform upsampling processing on the first decoded feature, and then concatenate the upsampled first decoded feature and the third feature map to obtain the second concatenated feature.
[0161] Step 4.25: Perform a convolution operation on the second concatenated feature to obtain the second decoded feature.
[0162] Here, the second decoding feature is represented as:
[0163]
[0164] in, This is the second decoding feature.
[0165] Step 4.26: Input the second decoded feature and the fourth feature map into the third decoding unit, perform upsampling processing on the second decoded feature, and then concatenate the upsampled second decoded feature and the fourth feature map to obtain the third concatenated feature.
[0166] Step 4.27: Perform a convolution operation on the third concatenated feature to obtain the third decoded feature.
[0167] Here, the third decoding feature is represented as:
[0168]
[0169] in, This is the third decoding feature.
[0170] Step 4.28: Perform a convolution operation on the third decoded feature to obtain the dehazed image.
[0171] Here, the dehazed image is represented as:
[0172]
[0173] in, This is the image after dehazing.
[0174] During the evaluation process, this embodiment uses an auxiliary regression task to reverse-parse the decoding prompts into approximate PT, PA, and The experimental results for parameters such as [parameter name] showed a high correlation with the actual values, confirming their reliability as a substitute index for physical parameters.
[0175] Dataset preparation and experimental results:
[0176] 1) Preparation of experimental data and setting of experimental parameters.
[0177] To achieve controllability and realism assessment of the polarization-aware dehazing model, this embodiment constructs a large-scale synthetic dataset based on the Cityscapes benchmark dataset. This dataset ensures the reliability of supervised learning and achieves diverse degradation effects by adding physically plausible fog effects and polarization cues.
[0178] To achieve high-precision depth-sensing fog effect synthesis, this embodiment employs the robust monocular depth estimation model Next-ViT-384, pre-trained on the MiDaS dataset. This model performs excellently in generating transferable depth maps across datasets, achieving physically consistent fog simulation even in real-world scenarios. Figure 5 It shows the changes in data visibility. Figure 5 The images from left to right represent scenes of light fog, medium fog, and dense fog, respectively.
[0179] Figure 6 This demonstrates the results of polarization image calculation. The final result was 3,475 realistic foggy images, along with corresponding polarization maps at 0°, 45°, and 90° angles.
[0180] Training cycles: 150, initial learning rate: set to The model was trained on a high-performance computing server equipped with four NVIDIA RTX 4090 graphics cards, using a batch size of 32 and an image resolution of [missing information]. .
[0181] 2) Experimental results.
[0182] To verify the effectiveness of PromptPolar (the overall framework of this invention) under different haze conditions, this embodiment performs quantitative and qualitative comparisons with several state-of-the-art dehazing methods, including Dark Channel Prior (DCP), CycleGAN (domain transformation), MSBDN, FFA-Net, and Dehazeformer. All methods are evaluated on the proposed polarization haze dataset, with SSIM and PSNR as the primary evaluation metrics. Comparative tests under light, moderate, and heavy fog scenarios verify the model's generalization ability under different scattering conditions.
[0183] 3) Experimental Results and Analysis
[0184] Table 1 presents the overall quantitative analysis results, in which the PromptPolar method continues to maintain its leading advantage, with an average SSIM value of 0.9959 and a PSNR value of 40.83 dB. Its advantages are particularly pronounced in dense fog scenes—traditional methods, methods based on Generative Adversarial Networks (GANs), methods based on Convolutional Neural Networks (CNNs), methods based on Transformers, and methods employing polarization prior learning all suffer from problems such as excessive texture smoothing and color artifacts.
[0185] Table 1. Comparison of Experimental Results
[0186]
[0187] Figure 7 It demonstrates a visual contrast effect. Figure 7 The images from top to bottom represent the test results for light fog, medium fog, and dense fog scenes, respectively. The PromptPolar model performs excellently in reconstructing object boundaries such as license plates and traffic signs, improving the structural sharpness of road markings, and restoring fine textures such as leaves and distant pedestrians. Particularly noteworthy is that, in extreme foggy environments, the model of this invention can capture visual cues that are almost invisible in real-world data, fully demonstrating the advantages of physical guidance and cue modulation technology.
[0188] This invention provides a novel polarization-aware dehazing network, PromptPolar, whose core architecture consists of three interconnected parts: (1) a Fourier transform-based polarization perceptron (FPP), which introduces frequency domain scattering modeling for the first time in dehazing tasks. It uses Fourier transform combined with a learnable spectral filter to extract global scattering features as the basis for prompt selection. By extracting the global frequency-aware descriptor of the polarization input, it captures the scattering characteristics derived from radiative transfer theory; (2) a multi-scale polarization cue library, which constructs a library consisting of transmitted polarization (PT), atmospheric light polarization (PA), and background light polarization. The implicit representation consists of a cue library, and the cue is dynamically selected and reparameterized through global descriptors to achieve adaptive modeling for different fog conditions. These descriptors are used to dynamically select and reparameterize physically interpretable cue markers as potential alternative indicators for polarization parameters; (3) Enhanced U-Net model, in the multi-layer features of the U-Net encoder, polarization cue is injected through cross-attention, so that the network not only relies on local convolutional features during reconstruction, but also has global physical prior perception ability. It injects these cue into the hierarchical features through a multi-scale cross-attention mechanism, so that adaptive repair can be completed without direct regression.
[0189] The cue-driven mechanism provided by this invention enables the network to dynamically adjust the semantic focus of each layer based on physically interpretable prior knowledge, without the need for explicit parameter estimation. This invention abandons the traditional approach of using polarization information as auxiliary input, directly embedding the latent structure into the feature space. This design allows the model to infer image degradation caused by scattering through an adaptive, context-aware guidance mechanism. Compared to the traditional U-Net architecture that relies solely on local convolutions, the cross-attention design of this invention obtains context-rich feature maps through global polarization information, thereby achieving more accurate image reconstruction in complex scenes such as non-uniform haze, mixed-material surfaces, and scenes lacking prior knowledge of sky or atmospheric light.
[0190] This invention proposes a polarization-aware adaptive dehazing method based on autonomous prompts. This method transforms polarization physical priors into learnable prompts and utilizes frequency domain scattering features to achieve dynamic prompt selection, thus avoiding the cumbersome process of physical parameter estimation. This method not only automatically captures optical scattering patterns but also enhances multi-scale features through prompt injection into deep networks, thereby improving generalization ability and robustness in complex environments. Therefore, this invention eliminates reliance on manual priors (such as sky regions and depth maps), adaptively perceives environmental variables, and achieves intelligent reasoning in the dehazing algorithm. This invention uses a prompt mechanism instead of physical parameter regression, resulting in a less complex network structure and physical interpretability. This invention utilizes Fourier domain features to enhance scattering modeling, improving adaptability to non-uniform fog layers and complex backgrounds.
[0191] This invention proposes a novel polarization-aware dehazing network, PromptPolar, which overcomes the dependence of traditional methods on explicit physical parameters by introducing physically guided cue modulation technology. This innovative paradigm transforms dehazing from a static inverse problem into a dynamic, perception-driven process, enabling the algorithm to make intelligent decisions based on polarization cues.
[0192] Example 2
[0193] Please see Figure 8 Based on Embodiment 1, this invention also provides a device for adaptive defogging based on autonomous prompting polarization sensing, comprising:
[0194] The image acquisition module is used to acquire polarization input feature images and intensity input images of the same region;
[0195] The polarization module is used for Fourier-based polarization modulation, which obtains descriptors based on the polarization input feature image.
[0196] The parameterization module is used to project descriptors into a latent space with a set scale to obtain descriptors at scale s, and to obtain reparameterized cue words based on a cue library based on polarization characteristics and descriptors at scale s.
[0197] The dehazing module takes the intensity input image and feeds it into the U-Net cue enhancement module based on the cross-attention mechanism to obtain a dehazed image based on the reparameterized cue words and the intensity input image.
[0198] It should be noted that the device for adaptive defogging based on autonomous prompting polarization sensing provided in this embodiment of the invention is similar in principle and effect to the method for adaptive defogging based on autonomous prompting polarization sensing provided in Embodiment 1, and will not be described again here.
[0199] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.
Claims
1. A method for adaptive dehazing based on autonomous prompting polarization sensing, characterized in that, include: Acquire polarization input feature image and intensity input image of the same region; Based on Fourier polarization modulation, a descriptor is obtained from the polarization input feature image; The descriptor is projected into a latent space with a set scale to obtain a descriptor at scale s, and a reparameterized cue word is obtained based on a cue library based on polarization characteristics and the descriptor at scale s. The intensity input image is input into the cue enhancement U-Net module based on the cross-attention mechanism to obtain a dehazed image based on the reparameterized cue words and the intensity input image; Specifically, the descriptor is projected into a latent space with a defined scale to obtain a descriptor at scale s, and reparameterized cue words are obtained based on a cue library based on polarization characteristics and the descriptor at scale s, including: A cue library based on polarization characteristics is established based on multi-scale polarization cue words, wherein the multi-scale polarization cue words include atmospheric light polarization, transmitted polarization, and background light; The descriptor is projected into a latent space with a set scale using a mapping function to obtain the descriptor at scale s. The mapping function is implemented by a lightweight two-layer multilayer perceptron and uses ReLU as the activation function. The softmax function is used to process the descriptors and weights at scale s to obtain adaptive selection weights; Based on the suggestion library, the adaptive selection weights, and the descriptors under the scale s, reparameterized suggestion words are obtained; The suggestion library is represented as follows: in, For the prompt library, The total number of physical prior abstractions. To indicate the length, Let be the set of real numbers. For scale Feature dimensions, For the first k A learnable label of physical prior abstraction. The illumination prior represented by atmospheric light polarization. The transmission scattering prior is represented by transmission polarization. The background light represents the prior environment; The adaptive selection weights are expressed as follows: in, for k An adaptive selection weight based on a physical prior abstraction. for k Weights of physical prior abstractions For scale s, descriptor; The reparameterized prompt word is represented as follows: in, The prompt words are reparameterized; After obtaining the reparameterized cue words based on the cue library, the adaptive selection weights, and the descriptor under the scale s, the process further includes: Construct the mutual information between the reparameterized cue words and the multiscale polarization cue words, and maximize the mutual information such that the maximized mutual information is greater than or equal to the variational lower bound; Maximizing the mutual information is represented as: in, For mutual information, For reparameterized prompt words, Atmospheric light polarization, For transmission polarization, As background light, To Variational inference, q This is a variational approximation.
2. The method for adaptive dehazing based on autonomous prompting polarization sensing according to claim 1, characterized in that, Based on Fourier polarization modulation, a descriptor is obtained from the polarization input feature image, including: The polarization input feature image is processed by two convolutional layers to obtain the convolutional features; The convolutional features are subjected to Fourier transform to obtain the frequency domain modulation features of all channels; The characteristics of the modulation spectrum are obtained based on the characteristics of the frequency domain modulation. The reconstructed features are obtained by performing an inverse Fourier transform on the features of the modulation spectrum; The global features are obtained based on the set of frequency domain modulation features of all channels and the reconstructed features; The global features are processed sequentially through global pooling and a multilayer perceptron to generate the descriptor.
3. The method for adaptive dehazing based on autonomous prompting polarization sensing according to claim 2, characterized in that, The expression for the global feature is: in, As a global feature, For the set of all frequency domain modulations, The modulated features, As a learnable mixing factor, This is the scaling factor.
4. The method for adaptive dehazing based on autonomous prompting polarization sensing according to claim 1, characterized in that, The enhanced U-Net module includes an encoder, a bottleneck layer, and a decoder; The process involves inputting the intensity input image into a cue enhancement U-Net module based on a cross-attention mechanism to obtain a dehazed image based on the reparameterized cue words and the intensity input image, including: Based on the cross-attention mechanism, cross-attention features are obtained from the label sequence of features obtained in the encoder based on the intensity input image and the reparameterized cue words, and a feature map is obtained based on the cross-attention features and the label sequence; The encoded features output by the encoder are transmitted to the decoder after passing through the bottleneck layer, so that the dehazed image can be obtained based on the feature map obtained by the encoder and the encoded features transmitted to the decoder through the bottleneck layer.
5. The method for adaptive dehazing based on autonomous prompting polarization sensing according to claim 4, characterized in that, The encoder includes a first encoding unit, a second encoding unit, a third encoding unit, and a fourth encoding unit; Specifically, based on the cross-attention mechanism, cross-attention features are obtained from the labeled sequence of features obtained in the encoder from the intensity input image and the reparameterized cue words. A feature map is then obtained based on the cross-attention features and the labeled sequence, including: The intensity input image is input into the first encoding unit of the encoder and subjected to convolution and downsampling processes in sequence to obtain the first encoding feature. The first encoding feature is then flattened into a label sequence to obtain the first label sequence. Based on the cross-attention mechanism, a first cross-attention feature is obtained according to the reparameterized cue words, the cue words complementary to the reparameterized cue words, and the first tag sequence; A first enhanced feature is obtained based on the first cross-attention feature and the first label sequence, and the first enhanced feature is reshaped into a first feature map; The first encoded feature is input into the second encoding unit and subjected to convolution and downsampling processes in sequence to obtain the second encoded feature. The second encoded feature is then flattened into a label sequence to obtain the second label sequence. Based on the cross-attention mechanism, a second cross-attention feature is obtained according to the reparameterized cue words, the cue words complementary to the reparameterized cue words, and the second tag sequence; The second enhanced feature is obtained based on the second cross-attention feature and the second label sequence, and the second enhanced feature is reshaped into a second feature map; The second coding feature is input into the third coding unit and subjected to convolution and downsampling processes in sequence to obtain the third coding feature. The third coding feature is then flattened into a label sequence to obtain the third label sequence. Based on the cross-attention mechanism, a third cross-attention feature is obtained according to the reparameterized cue words, the cue words complementary to the reparameterized cue words, and the third tag sequence; The third enhanced feature is obtained based on the third cross-attention feature and the third label sequence, and the third enhanced feature is reshaped into a third feature map; The third coding feature is input into the fourth coding unit and subjected to convolution and downsampling processes in sequence to obtain the fourth coding feature. The fourth coding feature is then flattened into a label sequence to obtain the fourth label sequence. Based on the cross-attention mechanism, a fourth cross-attention feature is obtained according to the reparameterized cue words, the cue words complementary to the reparameterized cue words, and the fourth tag sequence; The fourth enhanced feature is obtained based on the fourth cross-attention feature and the fourth label sequence, and the fourth enhanced feature is reshaped into a fourth feature map.
6. The method for adaptive dehazing based on autonomous prompting polarization sensing according to claim 5, characterized in that, The decoder includes a first decoding unit, a second decoding unit, and a third decoding unit; The encoder's final output encoded features are transmitted to the decoder after passing through the bottleneck layer. Based on the feature map obtained from the encoder and the encoded features transmitted to the decoder via the bottleneck layer, the dehazed image is obtained, including: The fourth encoded feature is convolved through the bottleneck layer to obtain the fifth encoded feature; The fifth encoded feature and the second feature map are input into the first decoding unit, the fifth encoded feature is upsampled, and the upsampled fifth encoded feature and the second feature map are concatenated to obtain the first concatenated feature; Perform a convolution operation on the first concatenated feature to obtain the first decoded feature; The first decoding feature and the third feature map are input into the second decoding unit. The first decoding feature is upsampled, and the upsampled first decoding feature and the third feature map are concatenated to obtain the second concatenated feature. Perform a convolution operation on the second concatenated feature to obtain the second decoded feature; The second decoding feature and the fourth feature map are input into the third decoding unit. The second decoding feature is upsampled, and the upsampled second decoding feature and the fourth feature map are concatenated to obtain the third concatenated feature. Perform a convolution operation on the third concatenated feature to obtain the third decoded feature; The third decoded feature is convolved to obtain the dehazed image.
7. A device for adaptive defogging based on autonomous prompting polarization sensing, characterized in that, The apparatus is used for performing the method of polarization-sensing adaptive defogging based on autonomous prompting as described in any one of claims 1 to 6, the apparatus comprising: The image acquisition module is used to acquire polarization input feature images and intensity input images of the same region; A polarization module is used for Fourier-based polarization modulation to obtain a descriptor based on the polarization input feature image; The parameterization module is used to project the descriptor into a latent space with a set scale to obtain a descriptor at scale s, and to obtain a reparameterized cue word based on a cue library based on polarization characteristics and the descriptor at scale s. The dehazing module is used to input the intensity input image into the cue enhancement U-Net module based on the cross-attention mechanism, so as to obtain the dehazed image according to the reparameterized cue words and the intensity input image.