Adaptive method for removing fog from remote sensing image based on pseudo-fog synthesis

Abstract

This invention provides an adaptive dehazing method for remote sensing images based on pseudo-haze synthesis. It utilizes readily available original RGB images from multiple scenes, preprocessing them to construct a haze-free reference image. Then, it combines a cloud / fog probability map with a fog distribution weight map derived from a pseudo-spectral vegetation index to construct a fog density map that closely matches the characteristics of real remote sensing fog. Finally, it synthesizes a foggy image with a remote sensing style using an atmospheric scattering model. This constructs supervised training pairs that do not require real haze-free remote sensing samples, solving the problems of difficulty in obtaining and high annotation costs of real haze-free remote sensing samples. Subsequently, the initial dehazing network acquires basic dehazing capabilities through training pairs. Furthermore, addressing the complexities and significant cross-domain differences in real remote sensing imaging environments, CLIP semantic constraints and Fourier transform spectral constraints are introduced to fine-tune the network unsupervised, allowing the dehazing network to adapt to the imaging patterns and feature distributions of real remote sensing, achieving high-quality dehazing results without relying on real haze-free remote sensing samples.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and computer vision technology, and in particular to an adaptive method for dehazing remote sensing images based on pseudo-haze synthesis. Background Technology

[0002] Remote sensing images are often affected by various factors during actual acquisition, such as atmospheric composition, lighting conditions, imaging angle, and sensor performance. Among these, image degradation caused by haze, thin clouds, and atmospheric scattering is the most common. This type of degradation not only causes overall brightness shift and reduced contrast, but also leads to blurred edges of ground features, loss of texture details, and distortion of color information, further affecting downstream intelligent remote sensing applications such as ground feature classification, target recognition, and change detection.

[0003] Traditional remote sensing dehazing methods mostly rely on physical atmospheric models, such as dark chromatic priors (DCP) and atmospheric scattering models (ASM), to recover clear images by estimating transmittance and atmospheric light. However, these methods have significant limitations in remote sensing scenarios: the complexity of remotely sensed ground features, uneven spectral distribution, and unclear shadow structures make it difficult to directly apply statistical priors from natural images. Furthermore, the large spatial scale of remote sensing images and the significant differences in fog patterns across different regions make it difficult for a uniform physical model to cover all situations.

[0004] In recent years, deep learning-based dehazing methods have become mainstream. However, remote sensing images lack paired "foggy-clear" data, and real satellite images often cannot obtain strictly aligned, clear versions, making it difficult to train supervised dehazing networks. While unsupervised or no-reference methods avoid dependence on ground truth, they lack stable and clear optimization objectives, often resulting in problems such as color cast, over-enhancement, and unstable semantic structure, making it difficult to meet engineering application standards in remote sensing scenarios.

[0005] Meanwhile, real remote sensing haze has distinct spatial distribution characteristics, such as density variations with terrain, structural occlusion from cloud shadows, and spectral compression due to low saturation and high haze, features that do not exist in natural images. Dehazing networks trained directly on natural datasets struggle to learn the characteristics and spectral patterns of fog in remote sensing scenes, resulting in decreased cross-domain performance. Summary of the Invention

[0006] This invention provides an adaptive method for dehazing synthetic remote sensing images based on pseudo-haze, which solves the problems in the prior art such as difficulty in obtaining real remote sensing fog-free samples, high annotation costs, complex imaging environments, and significant cross-domain differences.

[0007] On one hand, the present invention provides an adaptive method for dehazing remote sensing images based on pseudo-haze synthesis, comprising:

[0008] Acquire raw RGB images of multiple scenes captured by the first device, and preprocess the raw RGB images to obtain the first image;

[0009] The first image is processed with low saturation and low dynamic range to obtain a fog-free reference image;

[0010] Generate a cloud and fog probability map of the fog-free reference image, and calculate a pseudo-spectral vegetation index based on the red and green channels of the fog-free reference image to generate a fog distribution weight map.

[0011] The fog density map is obtained by fusing the cloud probability map and the fog distribution weight map, and the fog-free reference image is fogged and synthesized based on the atmospheric scattering model and the fog density map to generate a synthesized foggy image.

[0012] The synthesized foggy image and the fog-free reference image are used to form a training sample pair. An initial fog network is trained based on the training sample pair. The L1 loss function is used to calculate the loss between the defogging result of the synthesized foggy image and the fog-free reference image to obtain the defogging network.

[0013] The remote sensing image with fog generated by the second device is input into the defogging network. The defogging network is then fine-tuned unsupervised using semantic loss based on the CLIP model and spectral loss based on Fourier transform, and the defogging result image is output.

[0014] Optionally, the first image is subjected to low saturation and low dynamic range processing to obtain a fog-free reference image, including:

[0015] The first image is subjected to color space conversion and saturation reduction processing to obtain the first intermediate image;

[0016] The first intermediate image is subjected to brightness distribution compression and contrast adjustment processing to obtain the second intermediate image;

[0017] The second intermediate image is subjected to texture sparsification simulation processing to obtain a fog-free reference image.

[0018] Optionally, a cloud and fog probability map of the fog-free reference image is generated, and a pseudo-spectral vegetation index is calculated based on the red and green channels of the fog-free reference image to generate a fog distribution weight map, including:

[0019] Based on the fog-free reference image, an initial cloud and fog probability map is generated using a lightweight convolutional network;

[0020] The initial cloud probability map is subjected to multi-scale feature fusion, edge feathering, and morphological processing to obtain a cloud probability map.

[0021] Extract the red channel R and green channel G from the haze-free reference image to construct a pseudo-spectral vegetation index;

[0022] The formula for calculating the pseudo-spectral vegetation index is as follows:

[0023] ;

[0024] The pseudospectral vegetation index is mapped to spatial weights to generate a fog distribution weight map.

[0025] Optionally, the fog density map is obtained by fusing the cloud probability map and the fog distribution weight map, including:

[0026] Perform pixel-level multiplication on the cloud probability map and the fog distribution weight map to obtain the calculation result;

[0027] The numerical range of the calculation results is clipped to obtain a fog density map.

[0028] Optionally, based on the atmospheric scattering model and the fog density map, the fog-free reference image is fogged and synthesized to generate a synthesized foggy image, including:

[0029] The spatial transmittance distribution is obtained by calculating the pixel-level spatial transmittance based on the fog density map and depth prior.

[0030] By employing the imaging equation of the atmospheric scattering model and combining the spatial transmittance distribution with the atmospheric light vector, fog effect injection is performed on the fog-free reference image to obtain a synthesized foggy image.

[0031] Optionally, the spatial transmittance distribution is obtained by calculating the pixel-level spatial transmittance based on the fog density map and depth prior, including:

[0032] Combine the fog density map depth prior information;

[0033] Based on the atmospheric scattering attenuation model, the spatial transmittance at each pixel location is calculated, forming a spatial transmittance distribution.

[0034] Optionally, the synthesized foggy image and the fog-free reference image are used to form a training sample pair. An initial dehazing network is trained based on the training sample pair, and the L1 loss function is used to calculate the loss between the dehazing result of the synthesized foggy image and the fog-free reference image to obtain the dehazing network, including:

[0035] The synthetic hazy image from the training sample pair is input into the initial dehazing network for dehazing processing to obtain the initial dehazing result;

[0036] The L1 loss function is used to calculate the difference between the initial dehazing result and the haze-free reference image in the training sample pair, thus obtaining the L1 loss value; wherein the formula for calculating the L1 loss value is:

[0037] ;

[0038] in, To achieve the desired defogging result, Image pixels of the haze-free reference image;

[0039] Based on the L1 loss value, the parameters of the initial dehazing network are updated through backpropagation. This process is repeated until the initial dehazing network converges, resulting in a dehazing network with dehazing capabilities.

[0040] Optionally, the remotely sensed foggy image is input into the dehazing network, and the network is fine-tuned unsupervised using semantic loss based on the CLIP model and spectral loss based on Fourier transform. The dehazing result image is then output, including:

[0041] The remotely sensed foggy image is input into the dehazing network to obtain an initial dehazed image;

[0042] The semantic loss value of the initial dehazed map is calculated based on the CLIP model;

[0043] The spectral loss between the initial dehazed image and the unpaired real fog-free remote sensing image is calculated based on Fourier transform.

[0044] The semantic loss value and the spectral loss value are weighted and fused to obtain the total loss value;

[0045] The total loss value is propagated to the defogging network through end-to-end backpropagation to update the network parameters of the defogging network.

[0046] The dehazing result image is output through the finely tuned dehazing network.

[0047] Optionally, the semantic loss value of the initial dehazed map is calculated based on the CLIP model, including...

[0048] Construct a set of remote sensing fog-free semantic prompts;

[0049] The remote sensing fog-free semantic cue set is input into the CLIP text encoder to obtain text vectors;

[0050] The initial dehazed image is input into the CLIP image encoder to obtain the image vector;

[0051] Calculate the cosine similarity between the text vector and the image vector to obtain the cosine similarity value;

[0052] The semantic loss is constructed based on the cosine similarity value, and the semantic loss value is obtained.

[0053] Optionally, the spectral loss is calculated using Fourier transform, including:

[0054] Perform a two-dimensional Fourier transform on the initial dehazed image to obtain the first spectral amplitude;

[0055] A two-dimensional Fourier transform is performed on the unpaired fog-free remote sensing image to obtain the second spectral amplitude;

[0056] The first and second spectral amplitudes are divided into low-frequency, mid-frequency and high-frequency regions, respectively. The loss corresponding to each frequency band is calculated to obtain the low-frequency loss value, mid-frequency loss value and high-frequency loss value.

[0057] The low-frequency loss value, mid-frequency loss value, and high-frequency loss value are assigned weights and summed to obtain the spectrum loss value;

[0058] The formula for calculating the spectral loss value is as follows:

[0059] ;

[0060] in, The low-frequency loss value, The intermediate frequency loss value is... For high-frequency loss values, , These are the weights corresponding to the low-frequency loss value, the mid-frequency loss value, and the high-frequency loss value, respectively. On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the adaptive dehazing method for remote sensing images based on pseudo-haze synthesis as described above.

[0061] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the adaptive dehazing method for synthetic remote sensing images based on pseudo-fog as described above.

[0062] On the other hand, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the adaptive method for dehazing of remote sensing images based on pseudo-fog synthesis as described above.

[0063] This invention provides an adaptive dehazing method for remote sensing images based on pseudo-haze synthesis. It utilizes readily available original RGB images from multiple scenes, preprocessing and performing remote sensing stylization transformation to construct a haze-free reference image. Then, it combines a cloud / fog probability map with a fog distribution weight map derived from a pseudo-spectral vegetation index to construct a fog density map that closely matches the characteristics of real remote sensing fog. Finally, it synthesizes a foggy image with a remote sensing style using an atmospheric scattering model. This constructs supervised training pairs that do not require real remote sensing haze-free samples, solving the problems of difficulty in obtaining and high annotation costs associated with real remote sensing haze-free samples. Subsequently, the initial dehazing network acquires basic dehazing capabilities through training pairs. Furthermore, addressing the complexity and significant cross-domain differences in real remote sensing imaging environments, CLIP semantic constraints and Fourier transform spectral constraints are introduced to fine-tune the network unsupervised, allowing it to adapt to the imaging patterns and feature distributions of real remote sensing. Ultimately, this achieves high-quality dehazing results without relying on real remote sensing haze-free samples, reducing dependence on labeled data and improving the method's adaptability and generalization ability in real remote sensing scenarios. This effectively overcomes the problems of poor dehazing effects and semantic distortion caused by cross-domain differences in traditional methods. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0065] Figure 1 This is a schematic diagram of the adaptive method for dehazing remote sensing images based on pseudo-fog synthesis provided in an embodiment of the present invention;

[0066] Figure 2 This is a technical roadmap for the adaptive method for dehazing remote sensing images based on pseudo-fog synthesis provided in this embodiment of the invention;

[0067] Figure 3 This is a schematic diagram of the cloud and fog probability map generation process provided in an embodiment of the present invention.

[0068] Figure 4 This is a schematic diagram illustrating the calculation and characteristics of the pseudo-spectral vegetation index provided in an embodiment of the present invention;

[0069] Figure 5 This is a schematic diagram of fog effect injection and transmittance calculation of the atmospheric scattering model provided in the embodiment of the present invention;

[0070] Figure 6 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0072] Figure 1 This is a flowchart illustrating the adaptive method for dehazing of remote sensing images based on pseudo-fog synthesis provided in an embodiment of the present invention.

[0073] like Figure 1 As shown in the embodiments of the present invention, the adaptive method for dehazing of remote sensing images based on pseudo-haze synthesis mainly includes the following steps:

[0074] 101. Acquire original RGB images of multiple scenes captured by the first device, and preprocess the original RGB images to obtain the first image.

[0075] The original RGB images consist of a set of original RGB images with different scenes, textures, and categories. Each original RGB image in the set is subjected to size unification processing to ensure consistent image size. Then, brightness normalization processing is performed to bring the brightness of the original RGB images within a uniform standard range. Finally, color space unification processing is performed to unify the color space of all original RGB images. After the above preprocessing operations, the first image used for subsequent remote sensing style conversion and false fog synthesis is obtained.

[0076] Because real remote sensing images are easily affected by various factors such as atmospheric composition, lighting conditions, imaging angle, and sensor performance during actual acquisition, it is impossible to obtain a perfectly aligned and clear version, making it difficult to train supervised dehazing networks. Therefore, this invention processes raw RGB images captured by a first device to obtain image data suitable for training. The first device is a common device capable of capturing RGB images, such as a digital camera or smartphone. This facilitates the acquisition of raw RGB images from various scenes, offering broad applicability and convenience.

[0077] 102. Perform low saturation and low dynamic range processing on the first image to obtain a fog-free reference image.

[0078] Among these techniques, low saturation and low dynamic range processing can make the first image more visually consistent with a fog-free scene. Specifically, by reducing color saturation, the colors in the image become softer and more natural, reducing the visual interference of overly vibrant colors and making the first image closer to the color representation of a natural scene in a fog-free state. Low dynamic range processing is mainly used to balance the contrast between bright and dark areas in the image, avoiding overly bright or dark areas and making the overall brightness distribution of the first image more uniform.

[0079] Specifically, the first image is processed with low saturation and low dynamic range to obtain a haze-free reference image, including:

[0080] The first image is subjected to color space conversion and saturation reduction to obtain the first intermediate image;

[0081] The first intermediate image is subjected to brightness distribution compression and contrast adjustment to obtain the second intermediate image;

[0082] The second intermediate image is subjected to texture sparsification simulation processing to obtain a fog-free reference image.

[0083] In order to obtain a fog-free reference image by performing low saturation and low dynamic range processing on the first image, the first image needs to be converted from the original RGB color space to the HSV color parameter color space. By reducing the value of the saturation channel in the original RGB color space, the color vividness of the first image is weakened, the high saturation characteristics of the photographic image are removed, and the first intermediate image is obtained.

[0084] Next, a dynamic range compression algorithm is applied to the first intermediate image to limit the range of brightness values ​​in the first intermediate image in order to narrow the overall brightness distribution range. At the same time, the contrast parameter of the first intermediate image is adjusted to reduce the difference between light and dark, so that the dynamic range of the first intermediate image conforms to the narrow dynamic range characteristics of the remote sensing image, thus obtaining the second intermediate image.

[0085] Finally, multi-scale texture filtering or texture simplification algorithms are applied to the second intermediate image to weaken the dense detail textures and preserve the textures of ground features, simulating the sparse multi-scale textures of remote sensing images, and obtaining a fog-free reference image with the appearance features of remote sensing images.

[0086] 103. Generate a cloud and fog probability map of the fog-free reference image, and calculate the pseudo-spectral vegetation index based on the red and green channels of the fog-free reference image to generate a fog distribution weight map.

[0087] Among them, such as Figure 3 As shown, the fog-free probability map shows the probability distribution of each pixel belonging to the fog-free region in the fog-free reference image.

[0088] The pseudospectral vegetation index is calculated by using the pixel values ​​of the red and green channels of a haze-free reference image and performing mathematical operations on the pixel values ​​of the red and green channels.

[0089] The fog distribution weight map is generated based on the cloud and fog probability map of the fog-free reference image and the pseudo-spectral vegetation index.

[0090] Specifically, a cloud and fog probability map is generated from a fog-free reference image, and a pseudo-spectral vegetation index is calculated based on the red and green channels of the fog-free reference image to generate a fog distribution weight map, including:

[0091] An initial cloud and fog probability map is generated based on a fog-free reference image using a lightweight convolutional network.

[0092] Multi-scale feature fusion, edge feathering, and morphological processing are performed on the initial cloud probability map to obtain the cloud probability map.

[0093] Extract the red channel R and green channel G from the haze-free reference image to construct a pseudo-spectral vegetation index;

[0094] The pseudospectral vegetation index is mapped to spatial weights to generate a fog distribution weight map.

[0095] Among them, such as Figure 4 As shown, a lightweight convolutional network is constructed based on a fog-free reference image. The fog-free reference image is then input into the lightweight convolutional network. The fog distribution features of the fog-free reference image are extracted through the lightweight convolutional network, and the cloud and fog probability value corresponding to each pixel is directly output to obtain the initial cloud and fog probability map.

[0096] After obtaining the cloud / fog probability map, edge feathering processing is required. First, the boundary contours between the cloud / fog region and the background region in the cloud / fog probability map can be extracted using the Canny edge detection algorithm to obtain a binarized edge map. Then, a two-dimensional Gaussian kernel with a standard deviation of 1-3 is constructed, and the size of the two-dimensional Gaussian kernel is set to 5×5 or 7×7. Convolution operation is performed on the binarized edge map to obtain a gradient-smooth edge transition map.

[0097] The edge transition map and the cloud / fog probability map are fused at the pixel level with a weight of 0.3-0.5. This allows the pixel values ​​of the cloud / fog region and the background region to gradually transition from high probability to low probability, thus weakening the harsh boundary.

[0098] Next, morphological processing is performed. 3×3 or 5×5 circular structural elements are selected, and morphological opening operation is performed on the probability map after edge feathering to remove isolated small noise points in the cloud and fog probability map and avoid messy fog field distribution. Then, morphological closing operation is performed to fill the small holes inside the cloud / fog area to make the fog field morphology more complete and coherent.

[0099] After multi-scale feature fusion, edge feathering, and morphological processing, a cloud and fog probability map with soft boundaries, random morphology, and accurate guidance for the spatial distribution of fog fields is obtained.

[0100] Pixel values ​​of the red channel (R) and green channel (G) of the haze-free reference image are extracted. A pseudo-spectral vegetation index is introduced, and the index is calculated after normalizing the pixel values ​​of the two channels. The formula for calculating the pseudo-spectral vegetation index is as follows:

[0101] ;

[0102] in, To prevent extremely small constants with a denominator of zero, a pseudo-spectral vegetation index is used to simulate the spatial structure of the remote sensing normalized vegetation index, reflecting the vegetation cover correlation characteristics of different land cover types. Based on the numerical differences in the pseudo-spectral vegetation index, a mapping formula is adopted:

[0103] ;

[0104] in, This represents the minimum value of the pseudo-spectral vegetation index. This represents the maximum value of the pseudo-spectral vegetation index. The value is a very small constant. The pseudospectral vegetation index is mapped to spatial weights, so that areas with high pseudospectral vegetation index correspond to low fog weights, and areas with low pseudospectral vegetation index correspond to high fog weights. Finally, a fog distribution weight map that can reflect the differentiated coverage patterns of fog on different land cover is generated.

[0105] 104. The fog density map is obtained by fusing the cloud probability map and the fog distribution weight map, and the fog-free reference image is fogged and synthesized based on the atmospheric scattering model and the fog density map to generate a synthesized foggy image.

[0106] Among them, the fog density map is a comprehensive representation of the probability of fog and the weight of fog distribution, which can better reflect the true distribution and density differences of fog in the target area. The atmospheric scattering model, as a physical model widely used to describe the propagation of light in the atmosphere, considers the scattering and absorption of light by gas molecules and aerosols in the atmosphere. By using the atmospheric scattering model in combination with the fog density map to synthesize a fog-free reference image, the propagation path and intensity of light will change when light passes through the foggy atmosphere, resulting in a blurred and dim image.

[0107] By analyzing the fog density values ​​in different regions of the fog density map, the degree to which each pixel is affected by fog can be determined. For regions with higher fog density, a denser fog effect is simulated during image synthesis, making the pixel colors in that region paler and the contrast lower; while for regions with lower fog density, a lighter fog effect is simulated accordingly, to realistically reproduce the image features under different fog conditions as much as possible.

[0108] Specifically, the fog density map is obtained by fusing the cloud probability map and the fog distribution weight map, including:

[0109] Pixel-level multiplication is performed on the cloud probability map and the fog distribution weight map to obtain the calculation result;

[0110] The calculation results are cropped to obtain a fog density map.

[0111] The process involves acquiring a cloud and fog probability map characterizing the spatial distribution trend of the fog field and a fog distribution weight map reflecting the differences in fog coverage intensity among different ground features. Pixel-level multiplication is performed on the cloud and fog probability map and the fog distribution weight map. Specifically, for pixels at the same coordinate (x, y) in both images, the pixel value of ({cloud and fog probability map (x, y)}) is directly multiplied by the corresponding pixel value of ({fog distribution weight map (x, y)}), resulting in a pixel-wise coupled computational result. The computational result simultaneously integrates the spatial distribution characteristics of the fog field and the differentiated fog coverage characteristics of ground features.

[0112] Then, the clip function is used to clip the numerical range of the calculation result. The clipping interval is set to [0,1]. If the pixel value in the calculation result is less than 0, it is forcibly corrected to 0. If it is greater than 1, it is forcibly corrected to 1. If it is in the 0-1 interval, the original value is kept unchanged. The clipping ensures that the calculation result conforms to the physical meaning of fog density.

[0113] After pixel-level coupling and numerical cropping, a fog density map that combines spatial distribution rationality and ground feature adaptability is obtained.

[0114] Furthermore, based on the atmospheric scattering model and fog density map, fog-free reference images are synthesized to generate composite foggy images, including:

[0115] The spatial transmittance distribution is obtained by calculating the pixel-level spatial transmittance based on the fog density map and depth prior.

[0116] Specifically, the spatial transmittance is calculated at the pixel level based on the fog density map and depth prior, resulting in the spatial transmittance distribution, including:

[0117] Combine fog density maps with depth prior information;

[0118] Based on the atmospheric scattering attenuation model, the spatial transmittance at each pixel location is calculated, forming a spatial transmittance distribution.

[0119] Specifically, fog density maps, depth priors, and fog-free reference images are acquired. The depth prior is determined using methods including directly using depth information, replacing it with a low-frequency guiding map, replacing it with a smoothed range field, or using a constant approximation. Scattering coefficients and scaling factors are set based on the attenuation formula of the atmospheric scattering model.

[0120] ;

[0121] in, The scattering coefficient is... This represents a deep prior or a deep prior equivalent. This is the scaling factor; when reliable depth information is unavailable, It can be approximated by a low-frequency guide map, a smoothed distance field, or a constant. (Regarding fog density maps...) With depth prior Perform pixel-by-pixel calculations to determine the transmittance value for each pixel, and then summarize the results for all pixels to obtain the spatial transmittance distribution. Spatial transmittance distribution This is used to quantify the degree to which fog attenuates light; the lower the value, the stronger the fog attenuation.

[0122] By employing the imaging equation of the atmospheric scattering model and combining spatial transmittance distribution with atmospheric light vectors, fog effects are injected into a fog-free reference image to obtain a synthetic foggy image.

[0123] like Figure 5 As shown, atmospheric light vectors are obtained. The atmospheric light vector is obtained by extracting the mean RGB brightness of pixels in the sky region of the haze-free reference image. The imaging equation using the atmospheric scattering model is as follows:

[0124] ;

[0125] Among them, for the fog-free reference image Perform pixel-by-pixel fog injection; during computation, use a fog-free reference image. The pixel value and the transmittance at the corresponding position Multiply the signals to obtain the image signal after fog attenuation, and simultaneously calculate the atmospheric light vector. and The product of the two signals yields the fog contribution signal. The image signal and the fog contribution signal are then superimposed to obtain the pixel-by-pixel fogging result.

[0126] By summing the fogging results of all pixels, a synthetic fogged image with remote sensing-style fog effects is obtained. Synthesized foggy images It also reflects the spatial distribution pattern of fog, the differentiated fog coverage characteristics of ground objects, and the optical attenuation effect of real remote sensing fog.

[0127] 105. Combine the synthesized foggy image with the fog-free reference image to form a training sample pair. Train the initial fog network based on the training sample pair and use the L1 loss function to calculate the loss between the defogging result of the synthesized foggy image and the fog-free reference image to obtain the defogging network.

[0128] Specifically, a training sample pair is constructed using a synthesized hazy image and a hazy-free reference image. An initial dehazing network is trained based on this pair, and the L1 loss function is used to calculate the loss between the dehazing result of the synthesized hazy image and the hazy-free reference image, resulting in the dehazing network, which includes:

[0129] The synthetic hazy image from the training sample pair is input into the initial dehazing network for dehazing processing to obtain the initial dehazing result.

[0130] The L1 loss function is used to calculate the difference between the initial dehazing result and the haze-free reference image in the training sample pair, and the L1 loss value is obtained.

[0131] The formula for calculating the L1 loss value is as follows:

[0132] ;

[0133] in, For the dehazed image pixels, Image pixels of the fog-free reference image.

[0134] Based on the L1 loss value, the parameters of the initial dehazing network are updated through backpropagation. This process is repeated until the initial dehazing network converges, resulting in a dehazing network with dehazing capabilities.

[0135] Specifically, for a training sample pair consisting of a synthesized foggy image and a fog-free reference image, if the fog-free reference image is... The dehazing result output by the initial dehazing network is Both are digital images of the same size, existing in the form of a two-dimensional pixel matrix, where the position of each pixel can be determined by row coordinates. Column coordinates The pixel values ​​at the corresponding positions are uniquely determined and denoted as follows: and For color images, an additional channel dimension c needs to be added, with the corresponding pixel value denoted as c. and At the pixel level, the L1 loss for a single pixel is the absolute difference between the dehazed pixel value and the pixel value of the hazy reference image at that location. The global L1 loss value used to guide network parameter updates... This is obtained by summing or averaging the absolute differences of all pixels in the image.

[0136] 106. Input the remote sensing foggy image generated by the second device into the defogging network, perform unsupervised fine-tuning of the defogging network using semantic loss based on CLIP model and spectral loss based on Fourier transform, and output the defogging result image.

[0137] The dehazed image output, based on the CLIP model's semantic loss, constrains the dehazing process semantically, ensuring the dehazed result retains semantic features while better reflecting real-world scene characteristics. Meanwhile, the Fourier transform-based spectral loss focuses on the image's frequency domain information. By performing a Fourier transform on the image, converting it from the spatial domain to the frequency domain, information at different frequencies can be captured. During dehazing, the spectral loss helps adjust the dehazing network, making the dehazed image more closely resemble the spectral characteristics of the hazy-free image in the frequency domain. This helps remove blur and noise from the image, improving image clarity and detail.

[0138] In addition, the remote sensing foggy images generated by the second device are real satellite images.

[0139] During unsupervised fine-tuning, the parameters of the dehazing network are continuously adjusted to optimize the semantic loss based on the CLIP model and the spectral loss based on Fourier transform. As fine-tuning progresses, the dehazing network becomes better adapted to the characteristics of hazy remote sensing images, thereby outputting high-quality dehazed images.

[0140] Specifically, the remotely sensed foggy image is input into the dehazing network. Unsupervised fine-tuning of the network is performed using semantic loss based on the CLIP model and spectral loss based on Fourier transform. The dehazing result image is then output, including:

[0141] The remotely sensed foggy image is input into the dehazing network to obtain the initial dehazed image.

[0142] The dehazing network adopts a lightweight architecture based on MobileNetV2, consisting of a multi-scale encoder-decoder, spatial-channel dual attention enhancement, and residual connections. The input layer receives a normalized remote sensing foggy image tensor with dimensions H×W×3. The encoder extracts low-frequency basic features, mesoscale fog density and ground object edge features, and high-frequency detail features through four downsampling stages containing convolutional blocks, BN layers, activation functions, and residual connections, and fuses them into a global semantic feature map. The attention enhancement module generates focused features and suppresses redundant information through channel weight learning and spatial attention maps. The decoder fuses the features from each stage of the encoder with transposed convolution upsampling and skip connections, gradually restoring the image resolution and optimizing it through detail restoration blocks. Finally, the output layer obtains a feature map with pixel values ​​in the range [0,1] of H×W×3 by mapping with a 1×1 convolution and a Tanh activation function.

[0143] When generating the initial dehazed image, a dehazed network with stable initial parameters is loaded through supervised pre-training. The network receives the pre-processed remote sensing hazy image and extracts multi-scale features through the encoder forward propagation, optimizes features through the attention module, and reconstructs details through the decoder. Finally, the output layer outputs an initial dehazed image with the same size as the input image, which initially eliminates the effects of fog attenuation but has not undergone semantic and spectral loss fine-tuning.

[0144] The semantic loss value of the initial dehazed map is calculated based on the CLIP model.

[0145] Specifically, the semantic loss value of the initial dehazed map is calculated based on the CLIP model, including:

[0146] Construct a set of remote sensing fog-free semantic prompts.

[0147] Input the remote sensing fog-free semantic cue set into the CLIP text encoder to obtain text vectors.

[0148] The initial dehazed image is input into the CLIP image encoder to obtain the image vector.

[0149] Calculate the cosine similarity between the text vector and the image vector to obtain the cosine similarity value.

[0150] The semantic loss is constructed based on the cosine similarity value, and the semantic loss value is obtained.

[0151] When constructing the remote sensing fog-free semantic prompt set, the remote sensing fog-free semantic prompt set includes text descriptions that fit the real remote sensing scene, such as clear satellite remote sensing images, satellite images of fog-free farmland and roads, remote sensing images with zero cloud cover and clear texture, and aerial photos of cities and rivers without haze.

[0152] Subsequently, the remote sensing fog-free semantic cue set is input into the CLIP text encoder. The encoder extracts and maps features from the text description to obtain a text vector with semantic representation capabilities.

[0153] Simultaneously, the initial dehazed image is preprocessed, including size normalization and pixel value standardization, and then input into the CLIP image encoder. The encoder extracts the global semantic features of the image and converts them into image vectors with the same dimensions as the text vectors. Then, the cosine similarity calculation formula is used to calculate the similarity between the text vectors and the image vectors, and the cosine similarity value, which reflects the degree of semantic matching between the two, is obtained. The closer the cosine similarity value is to 1, the higher the consistency between the image and the text in the semantic space. The smaller the value, the greater the semantic difference between the two.

[0154] Finally, a semantic loss is constructed based on the cosine similarity value, using the following formula:

[0155] ;

[0156] in, For CLIP image encoder, For CLIP text encoder, For the initial dehazed image, For a predefined set of semantic prompt texts, For the corresponding global text semantic representation, This is the corresponding semantic representation of the image.

[0157] By calculating the semantic loss, a semantic loss value is obtained to constrain the semantic consistency of the initial dehazed image. The larger the semantic loss value, the more significant the semantic difference between the initial dehazed image and the clear remote sensing image, and vice versa.

[0158] The spectral loss values ​​of the initial dehazed image and the unpaired true fog-free remote sensing image are calculated based on Fourier transform.

[0159] Specifically, the spectral loss is calculated using Fourier transform, including:

[0160] A two-dimensional Fourier transform is performed on the initial dehazed image to obtain the first spectral amplitude.

[0161] A two-dimensional Fourier transform is performed on the unpaired fog-free remote sensing image to obtain the second spectral amplitude.

[0162] The first and second spectral amplitudes are divided into low-frequency, mid-frequency, and high-frequency regions, respectively. The loss corresponding to each frequency band is calculated to obtain the low-frequency loss value, mid-frequency loss value, and high-frequency loss value.

[0163] In this process, the initial dehazed image and the unpaired true haze-free remote sensing image are preprocessed to achieve uniform size. Then, two-dimensional fast Fourier transforms are performed on the preprocessed initial dehazed image and the unpaired true haze-free remote sensing image, respectively. Low-frequency components are shifted to the center of the image through spectral centering. Finally, the absolute value of the transform result is taken to obtain the first spectral amplitude representing the frequency domain energy distribution of the initial dehazed image. The second spectral amplitude, which characterizes the frequency domain features of a true fog-free remote sensing image. The calculation formula is as follows:

[0164] ;

[0165] in, To perform a two-dimensional fast Fourier transform on the input image, For the initial dehazed image, For unregistered real fog-free remote sensing images, Initial dehazed image Spectral amplitude in the frequency domain For real fog-free remote sensing images The spectral amplitude.

[0166] Weights are assigned to the low-frequency loss value, mid-frequency loss value, and high-frequency loss value, and then summed to obtain the spectral loss value.

[0167] Using the center of the spectrum-centered image as the origin, frequency bands are divided according to the frequency radius ratio. For example, the low-frequency region corresponds to a radius of 0-1 / 4 of the image diagonal, the mid-frequency region corresponds to a radius of 1 / 4-3 / 4 of the diagonal, and the high-frequency region corresponds to a radius of 3 / 4-1 / 4 of the diagonal. The first and second spectral amplitudes are cropped into sub-images corresponding to the frequency bands, and the difference between the two is calculated band by band using the L1 loss function, thus obtaining the low-frequency loss value that reflects the consistency between overall brightness and fog distribution. Mid-frequency loss value reflecting the transition characteristics of ground feature edges and high-frequency loss values ​​representing the recovery effect of texture details (roads, building boundaries, etc.) .

[0168] Weights are assigned to the three loss values ​​based on the importance of the frequency domain features of the remote sensing image, for example... , , Through formula By performing a weighted summation, the final result is a spectral loss value that reflects the difference in frequency domain energy distribution between the initial dehazed image and the actual fog-free remote sensing image. The larger the spectral loss value, the more inconsistent the spectral structures of the two images are, and vice versa.

[0169] The semantic loss value and the spectral loss value are weighted and fused to obtain the total loss value.

[0170] The total loss value is propagated back to the defogging network through end-to-end backpropagation to update the network parameters of the defogging network.

[0171] The dehazing result image is output through the finely tuned dehazing network.

[0172] Specifically, based on the optimization requirements for dehazing remote sensing images, the semantic loss value is... With spectral loss value Assign appropriate fusion weights. For example, set semantic consistency weights. Frequency domain matching weights It emphasizes the consistency between frequency domain texture and real remote sensing images, or dynamically adjusts the weight ratio according to the actual scene, through formulas. Weighted summation is performed to obtain the total loss value that takes into account both semantic consistency and frequency domain feature matching. The total loss value comprehensively quantifies the overall differences between the initial dehazed image and the clear remote sensing image in terms of semantics and frequency domain structure.

[0173] The total loss value is then passed to the dehazing network through an end-to-end backpropagation algorithm. Based on the chain rule, the gradient of the total loss with respect to the parameters of each layer of the network is calculated layer by layer from the output layer to the input layer. Then, stochastic gradient descent or adaptive moment estimation optimizer is used to iteratively update the network parameters according to the calculated gradient, gradually reducing the total loss value, so that the network learns the semantic features and frequency domain distribution patterns of the real remote sensing scene.

[0174] After multiple rounds of iterative optimization until the network converges, the fine-tuned dehazing network performs complete feature extraction, attention optimization, and image reconstruction on the input remote sensing hazy image. The final output is a dehazing result image with sufficient fog removal, faithful semantic representation of ground features, clear texture details, and frequency domain features that closely match the real fog-free remote sensing image.

[0175] In addition, by Figure 2 As shown, Figure 2 This invention provides a technical roadmap for an adaptive method for dehazing synthetic remote sensing images based on pseudo-fog. The adaptive method for dehazing synthetic remote sensing images based on pseudo-fog provided in this application is divided into two parts: Stage I (supervised pre-training) on ​​the left and Stage II (unsupervised fine-tuning) on ​​the right.

[0176] Stage I includes the input part, cloud mask, pseudo DNVI, atmospheric scattering model, dehazing network, loss function L1, and haze-free reference image; among them, cloud mask corresponds to fog probability map, and pseudo DNVI corresponds to pseudo normalized vegetation index.

[0177] Stage II includes input of real remote sensing images with fog, and a defogging network. Semantic loss value Spectral loss and final output. Backpropagation in Stage I and Stage II represents parameter updates during training. Both stages share the same dehazing network.

[0178] Figure 6 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention.

[0179] like Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions from the memory 630 to execute an adaptive dehazing method based on pseudo-haze synthetic remote sensing images.

[0180] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0181] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the adaptive dehazing method for remote sensing images based on pseudo-fog synthesis provided by the above methods.

[0182] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the adaptive dehazing method for pseudo-fog synthetic remote sensing images provided by the methods described above.

[0183] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0184] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive method for dehazing remote sensing images based on pseudo-haze synthesis, characterized in that, include: Acquire raw RGB images of multiple scenes captured by the first device, and preprocess the raw RGB images to obtain the first image; The first image is processed with low saturation and low dynamic range to obtain a fog-free reference image; Generate a cloud and fog probability map of the fog-free reference image, calculate a pseudo-spectral vegetation index based on the red and green channels of the fog-free reference image, map the pseudo-spectral vegetation index to spatial weights, and generate a fog distribution weight map. The fog density map is obtained by fusing the cloud probability map and the fog distribution weight map, and the fog-free reference image is fogged and synthesized based on the atmospheric scattering model and the fog density map to generate a synthesized foggy image. The synthesized foggy image and the fog-free reference image are used to form a training sample pair. An initial fog network is trained based on the training sample pair. The L1 loss function is used to calculate the loss between the defogging result of the synthesized foggy image and the fog-free reference image to obtain the defogging network. The remote sensing hazy image generated by the second device is input into the dehazing network to obtain an initial dehazing image; The semantic loss value of the initial dehazed map is calculated based on the CLIP model; The spectral loss between the initial dehazed image and the unpaired real fog-free remote sensing image is calculated based on Fourier transform. The semantic loss value and the spectral loss value are weighted and fused to obtain the total loss value; The total loss value is propagated to the defogging network through end-to-end backpropagation to update the network parameters of the defogging network. The dehazing result image is output through the finely tuned dehazing network.

2. The adaptive dehazing method for remote sensing images based on pseudo-haze synthesis according to claim 1, characterized in that, The first image is processed with low saturation and low dynamic range to obtain a fog-free reference image, including: The first image is subjected to color space conversion and saturation reduction processing to obtain the first intermediate image; The first intermediate image is subjected to brightness distribution compression and contrast adjustment processing to obtain the second intermediate image; The second intermediate image is subjected to texture sparsification simulation processing to obtain a fog-free reference image.

3. The adaptive dehazing method for remote sensing images based on pseudo-haze synthesis according to claim 1, characterized in that, Generate a cloud and fog probability map of the fog-free reference image, and calculate a pseudo-spectral vegetation index based on the red and green channels of the fog-free reference image to generate a fog distribution weight map, including: Based on the fog-free reference image, an initial cloud and fog probability map is generated using a lightweight convolutional network; The initial cloud probability map is subjected to multi-scale feature fusion, edge feathering, and morphological processing to obtain a cloud probability map. Extract the red channel R and green channel G from the haze-free reference image to construct a pseudo-spectral vegetation index; The formula for calculating the pseudo-spectral vegetation index is as follows: ； in, This refers to the pixel value of the green channel G. This represents the pixel value of the red channel R. To prevent extremely small constants with a denominator of zero; The pseudospectral vegetation index is mapped to spatial weights to generate a fog distribution weight map.

4. The adaptive dehazing method for remote sensing images based on pseudo-haze synthesis according to claim 1, characterized in that, The fog density map is obtained by fusing the cloud probability map and the fog distribution weight map, including: Perform pixel-level multiplication on the cloud probability map and the fog distribution weight map to obtain the calculation result; The numerical range of the calculation results is clipped to obtain a fog density map.

5. The adaptive method for dehazing remote sensing images based on pseudo-haze synthesis according to claim 1, characterized in that, Based on the atmospheric scattering model and the fog density map, the fog-free reference image is fogged and synthesized to generate a synthesized foggy image, including: The spatial transmittance distribution is obtained by calculating the pixel-level spatial transmittance based on the fog density map and depth prior. By employing the imaging equation of the atmospheric scattering model and combining the spatial transmittance distribution with the atmospheric light vector, fog effect injection is performed on the fog-free reference image to obtain a synthesized foggy image.

6. The adaptive method for dehazing remote sensing images based on pseudo-haze synthesis according to claim 5, characterized in that, Based on the fog density map and depth prior, pixel-level spatial transmittance is calculated to obtain the spatial transmittance distribution, including: Combine the fog density map depth prior information; Based on the atmospheric scattering attenuation model, the spatial transmittance at each pixel location is calculated, forming a spatial transmittance distribution.

7. The adaptive method for dehazing remote sensing images based on pseudo-haze synthesis according to claim 1, characterized in that, The synthesized hazy image and the hazy-free reference image form a training sample pair. An initial dehazing network is trained based on the training sample pair. The L1 loss function is used to calculate the loss between the dehazing result of the synthesized hazy image and the hazy-free reference image to obtain the dehazing network, which includes: The synthetic hazy image from the training sample pair is input into the initial dehazing network for dehazing processing to obtain the initial dehazing result; The L1 loss function is used to calculate the difference between the initial dehazing result and the haze-free reference image in the training sample pair, thus obtaining the L1 loss value; wherein the formula for calculating the L1 loss value is: ； in, To achieve the desired defogging result, Image pixels of the haze-free reference image; Based on the L1 loss value, the parameters of the initial dehazing network are updated through backpropagation. This process is repeated until the initial dehazing network converges, resulting in a dehazing network with dehazing capabilities.

8. The adaptive method for dehazing remote sensing images based on pseudo-haze synthesis according to claim 1, characterized in that, The semantic loss value of the initial dehazed map is calculated based on the CLIP model, including... Construct a set of remote sensing fog-free semantic prompts; The remote sensing fog-free semantic cue set is input into the CLIP text encoder to obtain text vectors; The initial dehazed image is input into the CLIP image encoder to obtain the image vector; Calculate the cosine similarity between the text vector and the image vector to obtain the cosine similarity value; The semantic loss is constructed based on the cosine similarity value, and the semantic loss value is obtained.

9. The adaptive method for dehazing remote sensing images based on pseudo-haze synthesis according to claim 1, characterized in that, The spectral loss is calculated using Fourier transform, including: Perform a two-dimensional Fourier transform on the initial dehazed image to obtain the first spectral amplitude; A two-dimensional Fourier transform is performed on the unpaired fog-free remote sensing image to obtain the second spectral amplitude; The first and second spectral amplitudes are divided into low-frequency, mid-frequency and high-frequency regions, respectively. The loss corresponding to each frequency band is calculated to obtain the low-frequency loss value, mid-frequency loss value and high-frequency loss value. The low-frequency loss value, mid-frequency loss value, and high-frequency loss value are assigned weights and summed to obtain the spectrum loss value; The formula for calculating the spectral loss value is as follows: ； in, The low-frequency loss value is... The intermediate frequency loss value is... For high-frequency loss values, , These are the weights corresponding to the low-frequency loss value, the mid-frequency loss value, and the high-frequency loss value, respectively.

Images