A multi-task cooperative iterative defogging method for remote sensing images

Abstract

The application discloses a multi-task cooperative iterative defogging method for remote sensing images, and belongs to the image restoration technical field based on computer vision. The method comprises the following steps: 1, acquiring a remote sensing image dataset; 2, inputting remote sensing foggy images of a training set into a defogging network; 3, inputting the remote sensing foggy images of the training set, a blue gradient prior feature map and a depth prior feature map into a cloud detection network; 4, calculating a cloud perception reconstruction loss by using remote sensing defogging images and a cloud probability map; 5, combining the defogging network and the cloud detection network obtained through iterative training to obtain a defogging network framework; and 6, selecting remote sensing foggy images in a test set in step 1 as inputs of the defogging network framework, evaluating and outputting test results. Experimental results show that the application is superior to existing similar methods in objective indexes and subjective visual evaluation.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence algorithms and remote sensing image dehazing technology, specifically to a multi-task collaborative iterative dehazing method for remote sensing images. Background Technology

[0002] In recent years, with the rapid upgrading of applications such as high-resolution remote sensing and disaster monitoring, optical satellite imagery has become an irreplaceable data source for acquiring surface information. However, remote sensing imaging is often affected by thin clouds and haze at high altitudes, leading to decreased image contrast and severely weakening the accuracy and reliability of subsequent tasks such as change detection and target recognition. Unlike traditional low-altitude photographs, remote sensing images are characterized by non-uniform distribution of haze and its coupling with cloud layers, making it difficult to directly apply general dehazing algorithms. Existing methods are mostly based on atmospheric scattering models, assuming that haze is uniform and exists only near the ground. However, at imaging altitudes of tens of thousands of meters, cloud and haze layers overlap, and the spectral response is complex, often resulting in color distortion and loss of surface details after dehazing. Therefore, efficient remote sensing image dehazing and cloud detection technologies have become a key step in remote sensing data preprocessing.

[0003] To overcome the limitations of the uniform haze assumption, recent research has begun to introduce deep learning into the field of remote sensing dehazing. Convolutional neural networks are used to learn the mapping relationship between hazy and clear images, and attempts are made to embed physical constraints to improve interpretability. However, purely data-driven methods still face significant bottlenecks: First, most methods are still primarily single-task dehazing, lacking simultaneous modeling of the coupled degradation of clouds and haze. This makes it difficult to achieve adaptive restoration of regions when non-uniform thick fog coexists with clouds, often resulting in insufficient restoration of thick fog areas and over-enhancement of clear areas. Second, traditional purely data-driven schemes lack prior cloud-fog interaction, making cloud edges easily misjudged as dense fog, leading to spectral artifacts and loss of texture details. Therefore, simultaneously modeling the degradation of both "clouds" and "fog" in remote sensing dehazing tasks and constructing a mechanism for information complementarity between the two has become a key direction for improving dehazing quality and the reliability of downstream applications.

[0004] The existing technical problems can be summarized as follows:

[0005] 1. Clouds and fog layers have similar brightness and texture in remote sensing images and often exhibit spatial overlapping distribution. Existing single-task dehazing methods lack cloud information constraints, making it difficult to effectively distinguish between clouds and fog. They are prone to mistaking cloud bodies and their edges as haze and suppressing them, resulting in cloud structure distortion and loss of details.

[0006] 2. Non-uniform haze in high-altitude remote sensing scenarios exhibits significant spatial and spectral variations depending on terrain and meteorological conditions, making the traditional uniform scattering assumption untenable. While existing deep learning-based dehazing methods can capture this variability to some extent, they still lack prior guidance based on physical features and fail to effectively utilize spectral priors such as the blue channel to distinguish differences between clouds and haze, leading to over-enhancement or under-enhancement in cloud-haze mixed regions. Furthermore, traditional multi-prior combination methods often achieve this through simple splicing or weighting, which can easily cause some irrelevant priors to interfere with feature extraction, increasing the uncertainty of the reconstruction process.

[0007] 3. Existing remote sensing dehazing methods often only consider the unidirectional fogging or dehazing process in frequency domain modeling, lacking complementary constraints on the spectral allocation of both. This easily leads to overlap or energy unevenness between low-frequency and high-frequency components, resulting in structural blurring and high-frequency artifacts. Especially in complex and non-uniform haze areas, when the frequency domain mask repeatedly responds at excessively high frequencies, the dehazing results show inconsistencies in brightness and color, thereby reducing the overall restoration quality and stability. Summary of the Invention

[0008] To overcome the aforementioned technical problems, this application provides a multi-task collaborative iterative dehazing method for remote sensing images. This method solves the problem in existing remote sensing image dehazing methods where dehazing and cloud detection are independent and information cannot be effectively shared. Through dual-task iterative collaboration and information interaction, dehazing and cloud detection mutually promote each other, improving the overall clarity and structural consistency of remote sensing images.

[0009] A multi-task collaborative iterative dehazing method for remote sensing images includes the following steps:

[0010] Step 1: Obtain the remote sensing image dataset, including the test set and the training set;

[0011] Step 2: Input the remote sensing foggy images from the training set into the iGLD-Net dehazing network to obtain the remote sensing dehazed images and depth prior feature maps. ;

[0012] Step 3: Combine the remote sensing foggy images and depth prior feature maps from the training set. The inputs are fed into the cloud detection network CD-Net; then modulated through a four-layer prior gating module MGFF, and finally the cloud probability map predicted by the cloud detection network CD-Net is obtained.

[0013] Step 4: Calculate the cloud perception reconstruction loss by combining the dehazed remote sensing image obtained in Step 2 and the cloud probability map obtained in Step 3. Then, the learnable frequency domain fogging module LFHM and the learnable frequency domain defogging module LFDM in the iGLD-Net defogging network are used to calculate the frequency domain reciprocal consistency loss. ;

[0014] Step 5: Repeat steps 2, 3, and 4 in sequence. Each cycle is considered an iteration. Iterate until the preset number of iterations is reached. Then, merge the dehazing network iGLD-Net and the cloud detection network CD-Net obtained through iterative training to obtain the dehazing network framework ICDF. The cloud detection network CD-Net only updates its network weights in the first iteration and keeps its parameters frozen for prediction only in subsequent iterations. The dehazing network iGLD-Net continuously updates its parameters in each iteration.

[0015] Step 6: Select the remote sensing foggy images from the test set in Step 1 as input to the defogging network framework ICDF, evaluate it, and output the test results.

[0016] Furthermore, step 1 specifically includes the following:

[0017] Step 11: Perform cloud detection on the remote sensing foggy images in the training set using the traditional thresholding method. After coarsely labeling the cloud areas, perform precise manual adjustments to construct cloud mask training samples corresponding to the remote sensing foggy images. In the cloud mask training samples, the pixels corresponding to the cloud areas are marked as 1, and the pixels corresponding to the other cloudless areas are marked as 0.

[0018] Step 12: The test set is divided into two categories: synthetic image test set and real image test set.

[0019] Furthermore, in step 2, the iGLD-Net dehazing network adopts a hybrid domain architecture and includes two lightweight encoder-decoder network branches; the hybrid domain architecture is implemented by a learnable frequency domain dehazing module LFDM and a learnable frequency domain fogging module LFHM; the two lightweight encoder-decoder network branches are M-Net and G-Net, respectively; the specific process of the iGLD-Net dehazing network includes the following:

[0020] Step A: During iterative training, the dehazing network iGLD-Net will use the data from the previous training iteration... The dehazed remote sensing image obtained in the second iteration The data is fed into M-Net and G-Net in parallel, and outputs the first... The haze distribution mask obtained in the next iteration With the The gamma correction mask obtained in the next iteration Subsequently, pixel-by-pixel fusion is performed using formula (1), and the LFDM frequency domain dehazing module can be learned using formula (2) to obtain the first pixel in this training round. The dehazed remote sensing image obtained in the second iteration Formulas (1) and (2) are shown below:

[0021] (1);

[0022] (2);

[0023] in, Represents pixel coordinates. and Representing the first The haze distribution mask and gamma correction mask obtained in the next iteration This represents the intermediate fusion result obtained through formula (1). and Representing the first Second and third The dehazed image obtained from the next iteration; This represents the operation of the learnable frequency domain dehazing module;

[0024] Step B: In the frequency domain of the iGLD-Net dehazing network, masks are constructed using a learnable frequency-domain fogging module (LFHM) and a learnable frequency-domain dehazing module (LFDM) to control the low-frequency and high-frequency components respectively. The learnable frequency-domain fogging module (LFHM) modifies the intermediate depth feature map of the M-Net in the frequency domain. Low-frequency enhancement is performed to form a deep prior feature map. .

[0025] Furthermore, the processing flow of the cloud detection network CD-Net in step 3 includes the following steps:

[0026] Step 31: Extract blue gradient prior feature maps from the remotely sensed foggy images in the training set. We obtain it through formula (3), which is shown below:

[0027] (3);

[0028] in and These are the pixel coordinates in the image. The gradient values ​​of the blue channel and the gradient values ​​of the red channel;

[0029] Step 32: Use the blue gradient prior feature map obtained in Step 31. With depth prior feature map The features are concatenated to obtain the initial mixed prior feature map. ; This is obtained through formula (4), which is shown below:

[0030] (4);

[0031] In the formula, For channel connection operations, The blue gradient prior feature map represents the area under the gradient. The blue gradient prior feature map represents... With depth prior feature map Stitching along the channel dimension;

[0032] Step 33: Combine the remote-sensed foggy images from the training set with the initial mixed prior feature map. The input is fed into the first-level prior gating module MGFF in the cloud detection network CD-Net;

[0033] Step 34: The prior gating module MGFF, located in the first stage, receives the initial image feature map through its main branch. Its prior branch processes the initial mixed prior feature map Initial Mixed Prior Feature Map Pointwise convolution and grouped depthwise convolution are performed using formula (5) to generate group gate weights; formula (5) is shown below:

[0034] (5);

[0035] in Represents grouped depthwise convolution. Represents the GeLU activation function; Representative group gating weights;

[0036] Step 35: The main branch of the prior gating module MGFF in the first level uses formula (6) to process the initial image feature map. Pixel-depth convolution and non-linear activation are performed; then, element-wise product modulation is performed using the group gating weights provided by the prior branches to obtain the first-level modulated feature map output by the main branch. Formula (6) is shown below:

[0037] (6);

[0038] in Represents grouping along the channel dimension. This represents the feature map after the first-level modulation.

[0039] Step 36: The prior branches of the first-level prior gating module MGFF utilize the initial mixed prior feature map. Group gating weights The updated first-level hybrid prior feature map is output through concatenation and convolution using formula (7). Formula (7) is shown below:

[0040] (7);

[0041] Step 37: Modulate the first-level feature map Mixed prior feature maps with first-level Input is sent to the prior gating module MGFF located in the second level;

[0042] Step 38: The prior gating module MGFF, located in the second stage, uses the modulated feature map from the first stage. The first-level mixed prior feature map serves as the input feature map for its main branch. As input to its prior branch, the processing and modulation operations of the main branch and prior branch are repeatedly executed, outputting the second-level modulated feature map and the second-level hybrid prior feature map; the prior gating module MGFF in the third level and the prior gating module MGFF in the fourth level repeatedly execute the processing and modulation operations of the main branch and prior branch, and the prior gating module MGFF in the fourth level finally outputs the cloud probability map predicted by the cloud detection network CD-Net.

[0043] Furthermore, the cloud-aware reconstruction loss in step 4 The cloud region of the remote sensing dehazed image output by the iGLD-Net is weighted using the cloud probability map predicted by the cloud detection network CD-Net, and then processed by formula (8), which is shown below:

[0044] (8);

[0045] in and These represent the height and width of the cloud detection map, respectively. Represents the total number of iterations. Represents the L1 norm. CD-Net, representing the cloud detection network, is at the... Cloud probability map predicted by round, Representing the The dehazed remote sensing image output from the next iteration This represents a clear, fog-free remote sensing image.

[0046] Furthermore, step 4 involves frequency domain reciprocal consistency loss. Fog amplitude scaling factor output by the learnable frequency domain fog module LFHM And fog curvature parameters The learnable frequency domain defogging module LFDM output defogging scaling factor and defogging curvature parameters Constraints are applied to the learnable frequency domain fogging module LFHM and the learnable frequency domain defogging module LFDM, which are obtained through formula (9), as shown below:

[0047] (9);

[0048] in To preset a fixed balance coefficient, Represents the total frequency constant 1. Represents the total number of iterations; where, the fog amplitude scaling factor is... And fog curvature parameters The intermediate depth feature map is obtained by the learnable frequency domain fogging module LFHM. The dehazing scaling factor is obtained by performing multi-layer convolution and Fourier transform. and defogging curvature parameters The intermediate fusion result of formula (1) is obtained by performing multi-layer convolution and Fourier transform on the learnable frequency domain dehazing module LFDM.

[0049] Furthermore, the evaluation process for the experimental results in step 6 is as follows: Peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and learned perceptual similarity (LPIPS) are selected as evaluation metrics for the synthetic image test set; for the real image test set, the Natural Image Quality Evaluator without Reference Metrics (NIQE), Multi-Level Perceptual Image Quality Assessment (MANIQA), and Multi-Scale Image Quality Assessment (MUSIQ) are used to measure the dehazing effect.

[0050] The beneficial effects of this invention are:

[0051] 1. This application constructs a collaborative iterative dehazing framework, enabling the cloud detection network to provide cloud distribution guidance information to the dehazing network through the cloud detection map at each iteration stage. At the same time, it uses the features extracted during the dehazing process to inversely enhance the cloud detection results, solving the problem in existing remote sensing image dehazing methods where dehazing and cloud detection are independent and information cannot be effectively shared. It achieves mutual promotion between dehazing and cloud detection, improving the overall clarity and structural consistency of remote sensing images.

[0052] 2. This application introduces gamma correction and fog distribution mask in the defogging network iGLD-Net, and gradually corrects the distribution of fog intensity in multiple iterations to achieve phased removal of non-uniform fog areas. This solves the problem that traditional prior methods and single-inference depth models are difficult to accurately model non-uniform fog distribution, and avoids the phenomenon of over-enhancing local dense fog areas or under-removing sparse fog areas.

[0053] 3. This application proposes a fusion of blue gradient physical prior and prior gating module, which adaptively modulates the feature map by gating weights generated by grouped convolution; it solves the problem in existing methods that simply superimpose different priors and find it difficult to adaptively adjust the prior contribution according to local and global context.

[0054] 4. This application proposes a frequency domain reciprocal consistency loss to constrain the complementary allocation of the learnable fogging module LFHM and the learnable defogging module LFDM in the frequency domain mask, so as to avoid energy conflict or overlay between the two in the same frequency band, better preserve the texture and edge details of ground features, and make the frequency domain operation physically interpretable. Attached Figure Description

[0055] Figure 1 This is a diagram of the overall network architecture of this application.

[0056] Figure 2 This is an architecture diagram of the iGLD-Net defogging network of this application.

[0057] Figure 3 This is an architecture diagram of the CD-Net cloud detection network in this application.

[0058] Figure 4 This is an architecture diagram of the learnable frequency domain fogging module LFHM and the learnable frequency domain defogging module LFDM in the iGLD-Net defogging network proposed in this application.

[0059] Figure 5 This is an architecture diagram of the prior gating module MGFF in the CD-Net cloud detection network of this application.

[0060] Figure 6 Comparison of the dehazing effects of this application with other algorithms on different datasets. Detailed Implementation

[0061] The present application will be further explained below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the present application. It should be noted that the terms "front", "rear", "left", "right", "up" and "down" used in the following description refer to the directions in the accompanying drawings, and the terms "inner" and "outer" refer to the directions toward or away from the geometric center of a specific component, respectively.

[0062] like Figure 1 As shown, this embodiment provides a multi-task collaborative iterative dehazing method for remote sensing images, which specifically includes the following steps:

[0063] Step 1: Obtain the remote sensing image dataset, including the test set and the training set;

[0064] Step 1 specifically includes the following:

[0065] Step 11: Perform cloud detection on the remote sensing foggy images in the training set using the traditional thresholding method. After coarsely labeling the cloud areas, perform precise manual adjustments to construct cloud mask training samples corresponding to the remote sensing foggy images. In the cloud mask training samples, the pixels corresponding to the cloud areas are marked as 1, and the pixels corresponding to the other cloudless areas are marked as 0.

[0066] Step 12: The test set is divided into two categories: synthetic image test set and real image test set.

[0067] Step 2: Input the remote sensing foggy images from the training set into the iGLD-Net dehazing network to obtain the remote sensing dehazed images and depth prior feature maps. The network architecture of the fog network iGLD-Net is as follows: Figure 2 As shown;

[0068] In step 2, the iGLD-Net dehazing network adopts a hybrid domain architecture and includes two lightweight encoder-decoder network branches. The hybrid domain architecture is implemented by a learnable frequency domain dehazing module LFDM and a learnable frequency domain fogging module LFHM. The two lightweight encoder-decoder network branches are M-Net and G-Net, respectively. The specific process of the iGLD-Net dehazing network includes the following:

[0069] Step A: During iterative training, the dehazing network iGLD-Net will use the data from the previous training iteration... The dehazed remote sensing image obtained in the second iteration The data is fed into M-Net and G-Net in parallel, and outputs the first... The haze distribution mask obtained in the next iteration With the The gamma correction mask obtained in the next iteration Subsequently, pixel-by-pixel fusion is performed using formula (1), and the LFDM frequency domain dehazing module can be learned using formula (2) to obtain the first pixel in this training round. The dehazed remote sensing image obtained in the second iteration Formulas (1) and (2) are shown below:

[0070] (1);

[0071] (2);

[0072] in, Represents pixel coordinates. and Representing the first The haze distribution mask and gamma correction mask obtained in the next iteration This represents the intermediate fusion result obtained through formula (1). and Representing the first Second and third The dehazed image obtained from the next iteration; This represents the operation of the learnable frequency domain dehazing module;

[0073] Step B: In the frequency domain of the iGLD-Net dehazing network, masks are constructed using a learnable frequency-domain fogging module (LFHM) and a learnable frequency-domain dehazing module (LFDM) to control the low-frequency and high-frequency components respectively. The learnable frequency-domain fogging module (LFHM) modifies the intermediate depth feature map of the M-Net in the frequency domain. Low-frequency enhancement is performed to form a deep prior feature map. .

[0074] Among them, such as Figure 4 As shown, the learnable frequency domain fogging module LFHM and the learnable defogging module LFDM share a similar architecture. The learnable frequency domain fogging module LFHM first receives the intermediate depth feature map. Intermediate depth feature maps were extracted using PConv and DConv. Multiscale feature maps Then, a two-dimensional Fourier transform is used to transform the multi-scale feature map. and intermediate depth feature map Transform to the frequency domain to obtain multi-scale feature maps. Corresponding multiscale amplitude spectrum and intermediate depth feature map The corresponding original amplitude spectrum Subsequently, PConv and fully connected layers were applied to multi-scale amplitude spectra. Extracting the fog amplitude scaling factor And fog curvature parameters And a frequency-domain fogging mask is constructed based on the Gaussian function. The specific formula is as follows:

[0075]

[0076] In the formula Represents the coordinates in the frequency domain plot. and These represent the width and height of the frequency domain plot, respectively. Represent the center coordinates of the spectrum; add a fog mask to the constructed frequency domain. Compared with the original amplitude spectrum The depth prior feature map is obtained by element-wise multiplication and inverse Fourier transform. The specific formula is as follows:

[0077] ;in Represents the inverse Fourier transform; the learnable frequency domain dehazing module LFDM has a similar architecture to the frequency domain fogging module, extracting the corresponding dehazing amplitude scaling factor. and defogging curvature parameters .

[0078] like Figure 3 and Figure 4 As shown, step 3: Combine the remote sensing foggy images and depth prior feature maps from the training set. The data are input into the cloud detection network CD-Net; then modulated through a four-layer prior gating module MGFF, finally obtaining the cloud probability map predicted by the cloud detection network CD-Net; the specific processing flow includes the following steps:

[0079] Step 31: Extract blue gradient prior feature maps from the remotely sensed foggy images in the training set. We obtain it through formula (3), which is shown below:

[0080] (3);

[0081] in and These are the pixel coordinates in the image. The gradient values ​​of the blue channel and the gradient values ​​of the red channel;

[0082] Step 32: Use the blue gradient prior feature map obtained in Step 31. With depth prior feature map The features are concatenated to obtain the initial mixed prior feature map. ; This is obtained through formula (4), which is shown below:

[0083] (4);

[0084] In the formula, For channel connection operations, The blue gradient prior feature map represents the area under the gradient. This represents the blue gradient prior feature map. With depth prior feature map Stitching along the channel dimension;

[0085] Step 33: Combine the remote-sensed foggy images from the training set with the initial mixed prior feature map. The input is fed into the first-level prior gating module MGFF in the cloud detection network CD-Net;

[0086] Step 34: The prior gating module MGFF, located in the first stage, receives the initial image feature map through its main branch. Its prior branch processes the initial mixed prior feature map Initial Mixed Prior Feature Map Pointwise convolution and grouped depthwise convolution are performed using formula (5) to generate group gate weights; formula (5) is shown below:

[0087] (5);

[0088] in Represents grouped depthwise convolution. Represents the GeLU activation function; Representative group gating weights; the framework of the prior gating module MGFF is as follows: Figure 5 As shown;

[0089] Step 35: The main branch of the prior gating module MGFF in the first level uses formula (6) to process the initial image feature map. Pixel-depth convolution and non-linear activation are performed; then, element-wise product modulation is performed using the group gating weights provided by the prior branches to obtain the first-level modulated feature map output by the main branch. Formula (6) is shown below:

[0090] (6);

[0091] in Represents grouping along the channel dimension. This represents the feature map after the first-level modulation.

[0092] Step 36: The prior branches of the first-level prior gating module MGFF utilize the initial mixed prior feature map. Group gating weights The updated first-level hybrid prior feature map is output through concatenation and convolution using formula (7). Formula (7) is shown below:

[0093] (7);

[0094] Step 37: Modulate the first-level feature map Mixed prior feature maps with first-level Input is sent to the prior gating module MGFF located in the second level;

[0095] Step 38: The prior gating module MGFF, located in the second stage, uses the modulated feature map from the first stage. The first-level mixed prior feature map serves as the input feature map for its main branch. As input to its prior branch, the processing and modulation operations of the main branch and prior branch are repeatedly executed, outputting the second-level modulated feature map and the second-level hybrid prior feature map; the prior gating module MGFF in the third level and the prior gating module MGFF in the fourth level repeatedly execute the processing and modulation operations of the main branch and prior branch, and the prior gating module MGFF in the fourth level finally outputs the cloud probability map predicted by the cloud detection network CD-Net.

[0096] Step 4: Calculate the cloud perception reconstruction loss by combining the dehazed remote sensing image obtained in Step 2 and the cloud probability map obtained in Step 3. Then, the learnable frequency domain fogging module LFHM and the learnable frequency domain defogging module LFDM in the iGLD-Net defogging network are used to calculate the frequency domain reciprocal consistency loss. Among them, cloud perception reconstruction loss The cloud region of the remote sensing dehazed image output by the iGLD-Net is weighted using the cloud probability map predicted by the cloud detection network CD-Net and processed by formula (8), which is shown below:

[0097] (8);

[0098] in and These represent the height and width of the cloud detection map, respectively. Represents the total number of iterations. Represents the L1 norm. CD-Net, representing the cloud detection network, is at the... Cloud probability map predicted by round, Representing the The dehazed remote sensing image output from the next iteration This represents a clear, fog-free remote sensing image.

[0099] Among them, frequency domain reversible consistency loss Fog amplitude scaling factor output by the learnable frequency domain fog module LFHM And fog curvature parameters The learnable frequency domain defogging module LFDM output defogging scaling factor and defogging curvature parameters Constraints are applied to the learnable frequency domain fogging module LFHM and the learnable frequency domain defogging module LFDM, as shown in formula (9);

[0100] (9); among which To preset a fixed balance coefficient, Represents the total frequency constant 1. This represents the total number of iterations.

[0101] Step 5: Repeat steps 2, 3, and 4 sequentially, with each cycle considered an iteration. Iterate until the preset number of iterations is reached. Then, merge the iGLD-Net and CD-Net obtained through iterative training to obtain the dehazing network framework ICDF, where the ICDF framework is as follows: Figure 1 As shown.

[0102] Step 6: Select the remote sensing foggy images from the test set in Step 1 as input to the ICDF dehazing network framework, evaluate it, and output the test results. The evaluation process in Step 6 is as follows:

[0103] Peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and learned perceptual similarity (LPIPS) were selected as evaluation metrics for the synthetic image test set. For the real image test set, the Natural Image Quality Evaluator without Reference Metrics (NIQE), Multi-Level Perceptual Image Quality Assessment (MANIQA), and Multi-Scale Image Quality Assessment (MUSIQ) were used to measure the dehazing effect.

[0104] Table 1 compares the objective evaluation metrics of this application with those of mainstream algorithms in recent years on synthetic datasets, covering traditional physical model methods, general deep learning algorithms, and remote sensing-specific methods.

[0105] Table 1. Experimental results of different algorithms on synthetic datasets.

[0106]

[0107] As can be seen from Table 1, compared with other methods in the table, the method of this application basically achieves the best performance on the three datasets, which proves the effectiveness of the method of this application.

[0108] Table 2 shows the experimental results of different algorithms on real datasets, further demonstrating the comparison of this application with other algorithms on real image test sets without reference metrics;

[0109]

[0110] As shown in Table 2, this application demonstrates superior performance compared to other algorithms on real datasets. Figure 6 The actual simulation results of this application are shown, and the subjective visual results are compared with those of other algorithms. Figure 6 As shown, the dehazing effect of the proposed method is significantly better than that of other compared algorithms. Whether measured by objective evaluation metrics on a synthetic image test set or by no-reference metrics on a real image test set, the proposed method outperforms other mainstream methods used for comparison. The dehazing results generated by this method are closer to clear images, with less color distortion and artifacts.

[0111] The technical means disclosed in this application are not limited to the technical means disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features.

Claims

1. A multi-task collaborative iterative dehazing method for remote sensing images, characterized in that, Specifically, the steps include the following: Step 1: Obtain the remote sensing image dataset, including the test set and the training set; Step 2: Input the remote sensing foggy images from the training set into the iGLD-Net dehazing network to obtain the remote sensing dehazed images and depth prior feature maps. ; Step 3: Combine the remote sensing foggy images and depth prior feature maps from the training set. The inputs are fed into the CD-Net cloud detection network; then modulated through a four-layer prior gating module MGFF, which consists of four cascaded prior gating modules MGFF; the first-layer prior gating module MGFF receives the initial image feature map through its main branch. and the initial image feature map Pixel-depth convolution and non-linear activation are performed, and the prior branches process the initial mixed prior feature maps. Pointwise convolution and grouped depthwise convolution are performed to generate group-gated weights; element-wise modulation is then performed using the group-gated weights to obtain the first-level modulated feature map of the main branch output. Initial Mixed Prior Feature Map Group gating weights The first-level hybrid prior feature map is updated after concatenation and convolution. The prior gating module MGFF, located in the second stage, uses the modulated feature map from the first stage. The first-level mixed prior feature map serves as the input feature map for its main branch. As the input of its prior branch, the processing and modulation operations of the main branch and the prior branch are repeatedly executed. Finally, the prior gating module MGFF in the fourth level outputs the cloud probability map predicted by the cloud detection network CD-Net. Step 4: Calculate the cloud perception reconstruction loss by combining the dehazed remote sensing image obtained in Step 2 and the cloud probability map obtained in Step 3. Then, the learnable frequency domain fogging module LFHM and the learnable frequency domain defogging module LFDM in the iGLD-Net defogging network are used to calculate the frequency domain reciprocal consistency loss. ; Step 5: Repeat steps 2, 3 and 4 in sequence. Each cycle is considered an iteration process. Iterate until the preset number of iterations are reached. Then, merge the dehazing network iGLD-Net and the cloud detection network CD-Net obtained through iterative training to obtain the dehazing network framework ICDF. Step 6: Select the remote sensing foggy images from the test set in Step 1 as input to the defogging network framework ICDF, evaluate it, and output the test results.

2. The multi-task collaborative iterative dehazing method for remote sensing images according to claim 1, characterized in that, Step 1 specifically includes the following: Step 11: Perform cloud detection on the remote sensing foggy images in the training set using the traditional thresholding method. After coarsely labeling the cloud areas, perform precise manual adjustments to construct cloud mask training samples corresponding to the remote sensing foggy images. In the cloud mask training samples, the pixels corresponding to the cloud areas are marked as 1, and the pixels corresponding to the other cloudless areas are marked as 0. Step 12: The test set is divided into two categories: synthetic image test set and real image test set.

3. The multi-task collaborative iterative dehazing method for remote sensing images according to claim 2, characterized in that, In step 2, the iGLD-Net dehazing network adopts a hybrid domain architecture and includes two lightweight encoder-decoder network branches. The hybrid domain architecture is implemented by a learnable frequency domain dehazing module LFDM and a learnable frequency domain fogging module LFHM. The two lightweight encoder-decoder network branches are M-Net and G-Net, respectively. The specific process of the iGLD-Net dehazing network includes the following: Step A: During iterative training, the dehazing network iGLD-Net will use the data from the previous training iteration... The dehazed remote sensing image obtained in the second iteration The data is fed into M-Net and G-Net in parallel, and outputs the first... The haze distribution mask obtained in the next iteration With the The gamma correction mask obtained in the next iteration Subsequently, pixel-by-pixel fusion is performed using formula (1), and the LFDM frequency domain dehazing module can be learned using formula (2) to obtain the first pixel in this training round. The dehazed remote sensing image obtained in the second iteration Formulas (1) and (2) are shown below: （1）； （2）； in, Represents pixel coordinates. and Representing the first The haze distribution mask and gamma correction mask obtained in the next iteration This represents the intermediate fusion result obtained through formula (1). and Representing the first Second and third The dehazed image obtained from the next iteration; This represents the operation of the learnable frequency domain defogging module LFDM; Step B: In the frequency domain of the iGLD-Net dehazing network, masks are constructed using a learnable frequency-domain fogging module (LFHM) and a learnable frequency-domain dehazing module (LFDM) to control the low-frequency and high-frequency components respectively. The learnable frequency-domain fogging module (LFHM) modifies the intermediate depth feature map of the M-Net in the frequency domain. Low-frequency enhancement is performed to form a deep prior feature map. .

4. The multi-task collaborative iterative dehazing method for remote sensing images according to claim 3, characterized in that, Step 3 of the CD-Net cloud detection network processing flow includes the following steps: Step 31: Extract blue gradient prior feature maps from the remotely sensed foggy images in the training set. We obtain it through formula (3), which is shown below: （3）； in and These are the pixel coordinates in the image. The gradient values ​​of the blue channel and the gradient values ​​of the red channel; Step 32: Use the blue gradient prior feature map obtained in Step 31. With depth prior feature map The features are concatenated to obtain the initial mixed prior feature map. ; This is obtained through formula (4), which is shown below: （4）； In the formula, For channel connection operations, The blue gradient prior feature map represents the area under the gradient. The blue gradient prior feature map represents... With depth prior feature map Stitching along the channel dimension; Step 33: Combine the remote-sensed foggy images from the training set with the initial mixed prior feature map. The input is fed into the first-level prior gating module MGFF in the cloud detection network CD-Net; Step 34: The main branch of the prior gating module MGFF in the first level uses formula (6) to process the initial image feature map. Pixel-depth convolution and non-linear activation are performed; then, element-wise product modulation is performed using the group gating weights provided by the prior branches to obtain the first-level modulated feature map output by the main branch. Formula (6) is shown below: （6）； in Represents grouping along the channel dimension. This represents the feature map after the first-level modulation. Step 36: The prior branches of the first-level prior gating module MGFF utilize the initial mixed prior feature map. Group gating weights The updated first-level hybrid prior feature map is output through concatenation and convolution using formula (7). Formula (7) is shown below: （7）； Step 37: The prior gating module MGFF, located in the second stage, uses the feature map modulated in the first stage. The first-level mixed prior feature map serves as the input feature map for its main branch. As input to its prior branch, the processing and modulation operations of the main branch and prior branch are repeatedly executed, outputting the second-level modulated feature map and the second-level mixed prior feature map; then the prior gating module MGFF in the third level and the prior gating module MGFF in the fourth level repeatedly execute the processing and modulation operations of the main branch and prior branch, and the prior gating module MGFF in the fourth level finally outputs the cloud probability map predicted by the cloud detection network CD-Net.

5. A multi-task collaborative iterative dehazing method for remote sensing images according to claim 4, characterized in that, Cloud-aware reconstruction loss in step 4 The cloud region of the remote sensing dehazed image output by the iGLD-Net is weighted using the cloud probability map predicted by the cloud detection network CD-Net and processed by formula (8), which is shown below: （8）； in and These represent the height and width of the cloud detection map, respectively. Represents the total number of iterations; Represents the L1 norm; CD-Net, representing the cloud detection network, is at the... Cloud probability map predicted by round, Representing the The dehazed image obtained from the next iteration This represents a clear, fog-free remote sensing image.

6. The multi-task collaborative iterative dehazing method for remote sensing images according to claim 5, characterized in that, Step 4 involves frequency domain reciprocal consistency loss. Fog amplitude scaling factor output by the learnable frequency domain fog module LFHM And fog curvature parameters The learnable frequency domain defogging module LFDM output defogging scaling factor and defogging curvature parameters Constraints are applied to the learnable frequency domain fogging module LFHM and the learnable frequency domain defogging module LFDM, which are obtained through formula (9), as shown below: （9）； in To preset a fixed balance coefficient, Represents the total frequency constant 1. This represents the total number of iterations.

7. A multi-task collaborative iterative dehazing method for remote sensing images according to claim 6, characterized in that, The evaluation process for the experimental results in step 6 is as follows: Peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and learned perceptual similarity (LPIPS) are selected as evaluation metrics for the synthetic image test set; for the real image test set, the Natural Image Quality Evaluator without Reference Metrics (NIQE), Multi-Level Perceptual Image Quality Assessment (MANIQA), and Multi-Scale Image Quality Assessment (MUSIQ) are used to measure the dehazing effect.

Images