A remote sensing turbulent flow image restoration method for soil and water composite pollution monitoring

Abstract

The application provides a remote sensing turbulent flow image restoration method for water and soil composite pollution monitoring, collects water area images, carries out standardized pretreatment on the collected water area images, constructs a pyramid attention generative adversarial network based on a Swin-U-Net backbone network to restore images, constructs a hybrid loss function containing L1 loss, perception loss, adversarial loss and spectral loss, carries out segmented training on the restored images, ensures that the overall water area image is not distorted, further refines the key details of the water-land junction line and water surface ripples, and finally adapts the water area restored image to a water and soil pollution intelligent evaluation platform, meets the evaluation of key pollution details of the water area and forms a closed-loop monitoring and evaluation.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, specifically to a method for restoring remote sensing turbulent images for monitoring water and soil pollution. Background Technology

[0002] Urban river and riparian zone water and soil pollution is characterized by spatial fragmentation, strong concealment, and complex water-land interaction. By investigating surface water pollution, tracing the source of water-land pollution spread, and monitoring pollution at the water-soil boundary, environmental governance can be shifted from passive response to proactive early warning. This, in turn, allows for the precise implementation of ecological restoration projects through reliable data. Currently, remote sensing image restoration for water and soil pollution mainly utilizes traditional digital image deblurring techniques, including signal processing algorithms such as Wiener filtering, blind deconvolution, and Gaussian filtering, as well as basic deep learning image restoration techniques, including native UNet networks, ordinary generative adversarial networks (GANs), and DeblurGAN deblurring networks. However, traditional digital image techniques do not rely on deep learning computing power and have a fast running speed, making them suitable for global blur restoration of conventional aerial images. However, traditional digital image techniques cannot solve nonlinear image blur problems. Basic deep learning image restoration techniques, compared to traditional digital image techniques, are better at solving nonlinear image blur problems and are therefore widely used in various image deblurring scenarios such as surface remote sensing, traffic aerial photography, and geological remote sensing, achieving an overall improvement in the clarity of aerial images.

[0003] For example, Chinese patent application number 202311725750.0, publication date February 27, 2024, and classification number G06T5 / 00 discloses an atmospheric turbulence image restoration method and system based on multi-scale generative adversarial networks, relating to the field of atmospheric turbulence degradation image restoration technology. The method includes the following steps: establishing an atmospheric turbulence degradation image dataset; training an atmospheric turbulence image restoration network model using the dataset as the base network model, which includes a generator and a discriminator; introducing a cascaded dilated convolution module and an attention mechanism module into the generator's network structure for optimization; using an adversarial loss term and a content loss term, applying the loss function as a regularization constraint for image deblurring on the restored image output by the generator, to repair the image edge distortion phenomenon of the restored image output by the generator; and verifying the atmospheric turbulence image restoration model.

[0004] The aforementioned literature addresses the dependence on multiple frames and complex data to improve the visual quality of image restoration by using multi-scale generative adversarial networks (GANs) in atmospheric turbulence image restoration, combining cascaded dilated convolutions and attention mechanisms, and training the DeblurGAN network model. It introduces a Markov discriminator and adversarial loss function, optimizing the generator and discriminator structures. However, its image processing only verifies the overall authenticity of the image through the GAN model, failing to specifically repair the water body's unique color features and surface micro-textures. The restored water body exhibits color distortion, missing surface ripple textures, and incongruous shoreline textures. It cannot distinguish between water pollution and surface noise based on abnormal water color differences, the outlines of floating pollutants, and the water turbidity gradient, thus hindering reliable image restoration and leading to distortion. Furthermore, the integration of multimodal large-scale language models with distorted images significantly reduces the accuracy of water and soil pollution level assessment, preventing the formation of an integrated monitoring and assessment closed loop. Summary of the Invention

[0005] The purpose of this invention is to provide a remote sensing turbulence image restoration method for monitoring water and soil composite pollution, which can accurately repair interference in remote sensing images, thereby effectively improving the accuracy of pollution feature extraction and ensuring the accuracy of image restoration.

[0006] To achieve the above objectives, this invention provides a method for restoring remote sensing turbulence images for monitoring combined water and soil pollution, comprising the following steps: S1. Collect water area images, and then construct a water system pairing dataset from the disturbed water area images with turbulence blur, motion blur, and water surface reflection degradation and the undisturbed water area images. S2. Standardize and preprocess the collected water area images, and divide the water system pairing dataset into training set and test set according to the preset ratio; S3. Construct a pyramid attention generative adversarial network based on the Swin-U-Net backbone network. Extract multi-scale features from the image of the water area with interference through the encoder. Then, generate feature maps of different scales through feature pyramid generation. The multi-head attention module determines the comprehensive attention weight of each scale feature map by combining window multi-head attention and spatial attention. After fusing the feature maps of different scales with the comprehensive attention weight, we get attention-enhanced feature maps of different scales. The decoder decodes the attention-enhanced feature maps of different scales and outputs feature maps. Then, the multi-scale discriminator performs three-fold discrimination on the output feature maps: global, color, and texture. S4. Construct a hybrid loss function that includes L1 loss, perceptual loss, adversarial loss and spectral loss, and change the weight coefficients corresponding to the adversarial loss and the weight coefficients corresponding to the adversarial loss and perceptual loss in turn to import the training set into the pyramid attention generative adversarial network for two-stage training. After training, the water image restoration model is obtained. S5. Re-input the acquired water area images into the water area remote sensing image restoration model to obtain a distortion-free water area restoration image.

[0007] The above method, after extracting water area images, uses a pyramid attention generative adversarial network to generate feature maps of different scales from a pyramid structure. Then, a multi-head attention module combines window-based multi-head attention and spatial attention to form a comprehensive attention weight. This allows different scale feature maps to be layered with local image details and spatial correlations both on the window and spatially, resulting in attention-enhanced feature maps at different scales. The multi-head attention module adaptively focuses on key pollution areas in the water area, accurately handling the unique non-uniform atmospheric turbulence noise near the water surface. This adapts to the complex interference environment of close-to-water aerial photography, avoiding the limitation of being unable to repair local turbulence blur. Furthermore, through global, chromatic, and texture triple discrimination, it can separately identify the pollution in the water area image. The model preserves the outline, color, and micro-texture of the water bodies, thus retaining the water area divisions, color differences, and turbidity gradient pollution features from aerial photography. This ensures that the restored image perfectly matches the actual aerial view and can distinguish whether the color difference is caused by water body color differences or surface noise. By performing segmented training on the restored image, adjusting the adversarial loss separately and combining adversarial loss and perceptual loss in two stages, the model first ensures that the overall water area image is distortion-free, and then refines key details such as the water-land boundary and water surface ripples, avoiding excessive smoothing of the restored image edges. This ensures the reliability of the entire water area image restoration model, allowing subsequent models to obtain the restored image of Wushi Town's water area simply by inputting the collected water area map into the model, further improving the accuracy of pollution feature extraction.

[0008] Furthermore, step S5 also includes: inputting the water area restoration image into the intelligent assessment system for water and soil composite pollution, extracting the water area segmentation and pollution characteristics, recording the spatial location of water pollution and the pollution diffusion range into the water and soil pollution knowledge graph, determining the spatial entity relationship, then inputting the water area restoration image into the knowledge-enhanced big data language model, combining soil physicochemical test data and historical water quality data to complete the dynamic pollution assessment, automatic spatial profiling, pollution source tracing analysis, and finally outputting the zoned governance decision.

[0009] The above settings input the restored water area image into the intelligent water and soil pollution assessment system, extract the water area and pollution characteristics, and correlate them with the water and soil pollution situation. This allows for the reliable determination of the location and situation of water and soil pollution. Furthermore, inputting the restored water area image into the enhanced large language model can output corresponding treatment plans after inputting relevant parameters. Adapting the restored water area image to the intelligent water and soil pollution assessment platform can meet the assessment of key pollution details in water areas and form a closed-loop monitoring and assessment.

[0010] Furthermore, step S3 includes: steps S3.1-S3.4. S3.1. Multi-scale features of water area images affected by turbulence are extracted by an encoder. The encoder includes one or more Swin Block modules based on the Swin Transformer architecture. The water area image is used as input feature. After passing through the Swin Block module, the first feature map is obtained. Then, it is pooled by the max pooling layer connected to the output of the Swin Block module and the second feature map is output. The Swin Block module includes two window attention layers, two shift window attention layers, one normalization layer and one ReLU activation function. The encoder processing process is shown in the following formulas (1) and (2): (1), (2), F swin This is the first feature map of the Swing Block module, where ReLU is the activation function. For normalization layer, For shift window attention layer, For window attention layer, F in This is the input feature map for the Swing Block module. θ ω For window attention layer parameters, θ sω For the shift window attention layer parameters, F pool This is the second feature map after pooling. This is a pooling layer, where stride is the layer number.

[0011] The above settings can effectively capture multi-scale features of images affected by turbulence by extracting the first feature map from the window, the window and the activation function, and then extracting the second feature through pooling.

[0012] Furthermore, in step S3.1, the encoder includes four Swing Block modules, and the processing via the Swing Block modules further includes: Water area image as input feature image I t After passing through the first Swin Block module, the output feature map has 64 channels and a size of 256×256. F 1. Extract shallow texture features through window attention; after passing through the second Swin Block module, output a feature map with 128 channels and a size of 128×128. F 2. Extract mid-layer features; after passing through the third SwinBlock module, the output feature map has 256 channels and a size of 64×64. F 3. Extract deep semantic features; after the fourth Swin Block module, the output feature map has 512 channels and a size of 32×32. F 4. Global features are captured through long-distance attention. The feature extraction process is shown in the following formula (3): (3), when hour, F 0= I t , θ ω,i , θ sω,i The first i Window attention parameters and shift window attention parameters for each Swin Block module.

[0013] The above settings enable the encoder to expand the image region by adjusting the window and shift window attention mechanism, without having to use stacked convolutions. This reduces computational cost while improving the ability to extract long-range features from contaminated areas in the image, thus avoiding the limitations of feature extraction.

[0014] Furthermore, step S3.2 includes: constructing a pyramid multi-head attention module (PAM), which comprises a feature pyramid, a multi-head attention module, and a fusion module. The PAM processing procedure includes: S3.2.1, Apply convolutional kernels of three different scales to the feature map from step S3.1. F 4. Perform parallel convolution operations to generate three feature maps of different scales, corresponding to small-scale, medium-scale, and large-scale feature maps respectively, to form a feature pyramid, as shown in the following formula (5): (5), θ s / m / l These are the weight parameters for three preset convolutional kernels with different scales. Fs , F m , F l These correspond to small-scale feature maps, medium-scale feature maps, and large-scale feature maps, respectively. S3.2.2. Adaptive average pooling is performed on feature maps at different scales. In the multi-head attention of the window in the Swin-U-Net backbone network, the global spatial features of the feature maps are compressed into channel features, and then connected to the network through two fully connected layers. Sigmoid The activation function calculates the channel attention weights, and then, in the spatial attention of the Swin-U-Net backbone network, 1×1 convolutions are performed on the feature maps of different scales for dimensionality reduction. Sigmoid The activation function calculates the spatial attention weights, and the channel attention weights are multiplied by the spatial attention weights to obtain the comprehensive attention weights corresponding to each feature map at different scales, as shown in the following formula (6): (6), The comprehensive attention weights are for feature maps of different scales. These are the channel attention weights corresponding to feature maps of different scales. Spatial attention weights corresponding to feature maps at different scales are determined by... Sigmoid The activation function is normalized to the [0,1] interval; S3.2.3 Multiply each feature map at different scales with its corresponding comprehensive attention weight to obtain enhanced feature maps at different scales. Then, concatenate the enhanced feature maps at different scales and reduce their dimensionality through 1×1 convolution to obtain the fused attention-enhanced feature map, as shown in the following formula (7): (7), Concat(·) is a channel concatenation operation. These are the weight parameters for the fusion convolution.

[0015] The above settings form a multi-scale feature pyramid through multi-scale convolutional kernels to cover key detail features at different scales in water pollution areas of remote sensing images of water bodies. By combining window multi-head attention and spatial attention in the Swin-U-Net backbone network and adaptively allocating comprehensive attention weights, local detail features in the feature map and global spatial dependencies of the image can be extracted separately. This effectively reduces the impact of cloud scattering, loss of detail, and reduced contrast on remote sensing images of water bodies caused by factors such as cloud scattering and sea surface turbulence. At the same time, it enhances the correlation between the global pollution range features of the sea surface and coast, and between water bodies and suspended matter in remote sensing images of water bodies, thereby achieving the effect of suppressing interference from irrelevant background features.

[0016] Furthermore, step S3.3 includes: The attention-enhanced feature map output by the Pyramid Multi-Head Attention Module (PAM) is processed by the decoder. F PAM The spatial resolution of the image is gradually restored by upsampling through one or more deconvolution blocks, and the output is the same as the input feature image. I t Restored images of the same size I c ; In step S3.3, the decoder processing includes: the decoder adopts a structure symmetrical to the encoder, including 4 deconvolution blocks and 4 Swin Transformer blocks; each deconvolution block includes a 2×2 deconvolution layer, a Swin Transformer submodule, a normalization layer and a ReLU activation function, as shown in the following formula (8): (8), This represents the output feature map of the deconvolution block, where ReLU is the activation function. For normalization layer, These are the weight parameters of the deconvolution layer. For the shift window attention parameters of the Swing Transformer submodule, It is a deconvolutional layer; Input feature map to the decoder; This is a submodule of Swin Transformer; After upsampling, the feature map output by the encoder is channel-adjusted, and then concatenated with the feature map output by the corresponding deconvolution block of the decoder. The shallow texture features extracted by the encoder are fused with the deep semantic features by skip connections, as shown in the following formula (9): (9), For the decoder i The output feature map of each deconvolution block This is the feature map of the corresponding layer of the encoder. Adjust the weight parameters of the convolution for each channel. This refers to the output features after skip connection fusion.

[0017] The above settings restore the spatial resolution of the feature map after passing through the Pyramid Multi-Head Attention (PAM) module by upsampling the feature map.

[0018] Through the synergistic effect of skip connection feature fusion and Swing Transformer sub-blocks, the decoder can perform skip connection fusion of shallow texture features and deep semantic features extracted by the encoder, making up for the loss of detail features during image upsampling, while further refining contaminated detail features and improving the clarity and detail representation of the restored image.

[0019] Furthermore, step S3.4 includes a multi-scale discriminator performing multiple discriminations on the feature map output by the decoder. The multi-scale discriminator includes a global discriminator, a chromaticity discriminator, and a texture discriminator set up in parallel branches. The overall discrimination function of the multi-scale discriminator is shown in the following formula (10): (10) x For input to the multi-scale discriminator, For multi-scale discriminator parameters, D g (·), D c (·), D t (·) represent the outputs of the global discriminator, chroma discriminator, and texture discriminator, respectively. , , These are the loss weights corresponding to the global discriminator, chroma discriminator, and texture discriminator, respectively.

[0020] The above settings are for easier approval later. LeakyReLU The activation function determines the authenticity of an image, calculates the discrimination loss, and provides feedback in reverse, thereby achieving collaborative optimization between the generator and the discriminator, which improves the restoration quality and visual realism of water pollution images.

[0021] Furthermore, step S3.4 also includes: The global discriminator consists of 5 convolutional blocks and 1 output layer. Each convolutional block includes a 4×4 convolution, a normalization layer, and... Sigmoid The activation function and the global discriminant function are shown in the following formula (11): (11), For global branch convolution sequence operations, The parameters are preset global discriminator parameters, and σ(•) is... Sigmoid Activation function, global discriminator output ; The chromaticity discriminator consists of two convolutional blocks, and the chromaticity discriminant function is shown in the following formula (12): (12), RGB2CbCr (·)for RGB Space to CbCr Space transformation functions, ConvSeq c (·) represents the chroma branch convolution sequence operation, and σ(•) is... Sigmoid Activation function Preset the parameters for the colorimetric discriminator; The texture discriminator consists of two convolutional blocks, and the texture discrimination function is shown in the following formula (13): (13) DoG (·) represents the Gaussian difference filtering operation. ConvSeq t (·) represents the convolution sequence operation for texture branches, and σ(·) is... Sigmoid Activation function These are the preset texture discriminator parameters.

[0022] The above settings, through the global discriminator, can identify global structural differences in water pollution images and suppress global geometric distortion in the restored image; through the chromaticity discriminator, it can determine whether the chromaticity of the input image is the same, reducing chromaticity shift caused by turbulence distortion, thereby ensuring the spectral fidelity of the restored image; through the texture discriminator, it can improve the authenticity of the detailed texture features of water pollution, prompting the generator to optimize image detail features, improve the image restoration effect, and reduce image false sharpness.

[0023] Furthermore, the hybrid loss function in step S4 is shown in formula (14) below: (14) For a mixed loss function, For L1 loss, In order to perceive loss, To combat the losses, For spectral loss, These are the weighting coefficients corresponding to L1 loss, perceptual loss, adversarial loss, and spectral loss, respectively.

[0024] The above settings, through weighted fusion of different loss terms, can achieve synergistic optimization of objective accuracy and subjective visual effect, while ensuring the spectral consistency of the restored image.

[0025] Furthermore, step S4 also includes: Keep the weight coefficients corresponding to L1 loss, perceptual loss and spectral loss in the hybrid loss function unchanged, reduce the weight coefficients corresponding to adversarial loss, and import the training set into the pyramid attention generative adversarial network for the first stage of training. Keeping the weight coefficients corresponding to L1 loss and spectral loss in the hybrid loss function unchanged, while increasing the weight coefficients corresponding to adversarial loss and perceptual loss, the training set is imported into the pyramid attention generative adversarial network for the second stage of training.

[0026] The above settings, through the first stage of training, can prioritize the repair of overall image blur and large-scale turbulence distortion, ensuring the clarity and accuracy of the global outline of the water area in the image; through the second stage of training, it can optimize the microscopic texture of the water surface and the detailed features of the water-land boundary edge in the image, avoiding excessive smoothing of the image. Attached Figure Description

[0027] Figure 1 This is a flowchart of the process of the present invention.

[0028] Figure 2 A schematic diagram of a pyramid attention-based adversarial network.

[0029] Figure 3 This is a schematic diagram of the pyramid multi-head attention module.

[0030] Figure 4 A comparison chart showing the effects of turbulence suppression and restoration on various models in a eutrophication pollution scenario. Detailed Implementation

[0031] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0032] like Figure 1-4 As shown, a remote sensing turbulence image restoration method for monitoring combined water and soil pollution includes the following specific steps: S1. Collect water area images, including disturbed water area images and undisturbed water area images. Construct a water system remote sensing paired dataset by combining the disturbed water area remote sensing images with turbulence blur, motion blur, and water surface reflection degradation with the undisturbed water area remote sensing images.

[0033] S2. Standardize the collected water area images. Specifically, crop the water area images to 512×512 pixels, normalize the pixel values ​​to the range of [-1,1], and then divide the water area pairing dataset into training and test sets according to a preset ratio of 8:2.

[0034] S3. Construct a pyramid attention generative adversarial network based on the Swin-U-Net backbone network. The encoder extracts multi-scale features from the interfering water area image, and then the feature pyramid generates feature maps of different scales. The multi-head attention module combines windowed multi-head attention and spatial attention to determine the comprehensive attention weight for each scale feature map. The feature maps of different scales are fused with the comprehensive attention weights to obtain attention-enhanced feature maps of different scales. The decoder then upsamples and restores the features of these attention-enhanced feature maps at different scales. Finally, a multi-scale discriminator performs global, chromaticity, and texture discrimination on the feature maps output by the decoder to restore the image. Specifically, this includes steps S3.1 to S3.4: S3.1 Extracting multi-scale features of water area images affected by turbulence through an encoder. The encoder includes one or more Swin Block modules based on the Swin Transformer architecture. The water area remote sensing image is used as input feature. After passing through the Swin Block module, the first feature map is obtained. Then, it is pooled by the max pooling layer connected to the output of the Swin Block module and the second feature map is output. The Swin Block module includes two window attention layers, two shift window attention layers, one normalization layer and one ReLU activation function, as shown in the following formulas (1) and (2): (1), (2), in, F swin This is the output feature map of the Swing Block module, i.e., the first feature map, where ReLU is the activation function. For normalization layer, For shift window attention layer, For window attention layer, F in This is the input feature map for the Swing Block module. θ ω For window attention layer parameters, θ sω For the shift window attention layer parameters, F pool This is the output feature map after pooling, i.e., the second feature map. This is a pooling layer, where stride is the layer number.

[0035] In this embodiment, the encoder has four Swing Block modules, and the water area image is used as the input feature image. I tAfter passing through the first Swin Block module, the output feature map has 64 channels and a size of 256×256. F 1. Extract shallow texture features through window attention; after passing through the second Swin Block module, output a feature map with 128 channels and a size of 128×128. F 2. Extract mid-layer features; after passing through the third Swin Block module, the output feature map has 256 channels and a size of 64×64. F 3. Extract deep semantic features; after the fourth Swin Block module, the output feature map has 512 channels and a size of 32×32. F 4. Global features are captured through long-distance attention. The feature extraction process is shown in the following formula (3): (3), Among them, when hour, F 0= I t , θ ω,i , θ sω,i The first i The window attention parameters and shift window attention parameters of each Swing Block module enable the encoder to expand the image region by adjusting the window and shift window attention mechanisms without the need for stacked convolutions. This reduces computational cost while improving the ability to extract long-range features from contaminated areas in the image, avoiding the limitations of feature extraction.

[0036] S3.2 Construct the Pyramid Multi-Head Attention Module (PAM). The Pyramid Multi-Head Attention Module (PAM) consists of three parts: feature pyramid, multi-head attention weight module, and feature fusion. The PAM processing procedure is shown in the following formula (4): (4), in F PAM This is the attention-enhanced feature map output by the Pyramid Multi-Head Attention Module (PAM). θ PAM The parameters are for the Pyramid Multi-Head Attention Module (PAM); the PAM processing procedure specifically includes steps S3.2.1 to S3.2.3. S3.2.1. Apply convolutional kernels of three different scales (1×1, 3×3, and 6×6) to the feature map from step S3.1. F4. Perform parallel convolution operations to generate three feature maps of different scales, corresponding to small-scale, medium-scale, and large-scale feature maps respectively, to form a feature pyramid, as shown in the following formula (5): (5), in, θ s / m / l These are the weight parameters for three different scale convolution kernels. F s , F m , F l Corresponding to small-scale feature maps, medium-scale feature maps, and large-scale feature maps respectively, 1×1 convolution is used to extract small-scale features, 3×3 convolution is used to extract medium-scale features, and 6×6 convolution is used to extract large-scale features. In this way, a multi-scale feature pyramid is formed by multi-scale convolution kernels, which can cover key detail features of different scales in water pollution areas in water remote sensing images. S3.2.2. Adaptive average pooling is performed on feature maps at different scales. In the multi-head attention of the window in the Swin-U-Net backbone network, the global spatial features of the feature maps are compressed into channel features, and then connected to the network through two fully connected layers. Sigmoid The activation function calculates the channel attention weights, and then, in the spatial attention of the Swin-U-Net backbone network, 1×1 convolutions are performed on the feature maps of different scales for dimensionality reduction. Sigmoid The activation function calculates the spatial attention weights, and the channel attention weights are multiplied by the spatial attention weights to obtain the comprehensive attention weights corresponding to each feature map at different scales, as shown in the following formula (6): (6), in, The comprehensive attention weights are for feature maps of different scales. These are the channel attention weights corresponding to feature maps of different scales. Spatial attention weights corresponding to feature maps at different scales are determined by... Sigmoid The activation function is normalized to the [0,1] interval; By combining window multi-head attention and spatial attention in the Swin-U-Net backbone network and adaptively allocating the comprehensive attention weight, it is possible to extract local detail features in the feature map and construct the global spatial dependency relationship of the image. Furthermore, window multi-head attention can extract local detail features such as debris fragments in the image by calculating attention in separate windows, thereby effectively reducing the impact of cloud and fog scattering, sea surface turbulence and other factors on the water remote sensing image, which can lead to blurring, loss of detail and reduced contrast. S3.2.3 Multiply each feature map at different scales with its corresponding comprehensive attention weight to obtain enhanced feature maps at different scales. Then, concatenate the enhanced feature maps at different scales and reduce the dimensionality through 1×1 convolution to adjust the number of channels to 512 to obtain the fused attention-enhanced feature map, as shown in the following formula (7): (7), Concat(·) is a channel concatenation operation. To fuse the weight parameters of the convolution, the attention-enhanced feature effect in the fused attention-enhanced feature map satisfies:

[0037] in, The original multi-scale features output by the encoder. Features after attention enhancement To comprehensively consider attention weights, pollute key areas. The value is close to 1, while the value in the background area is close to 0.

[0038] This can enhance the correlation between the global pollution range characteristics of the sea surface and coast, and between water bodies and suspended matter in remote sensing images of water areas, thereby achieving the effect of suppressing interference from irrelevant background features.

[0039] S3.3, The decoder outputs the attention-enhanced feature map from the Pyramid Multi-Head Attention Module (PAM). F PAM The spatial resolution of the image is gradually restored by upsampling through one or more deconvolution blocks, and the output is the same as the input feature image. I t Restored images of the same size I c In this embodiment, the decoder has four deconvolution blocks. Each deconvolution block includes a 2×2 deconvolution layer, a Swin Transformer submodule, a normalization layer, and a ReLU activation function, as shown in the following formula (8): (8), in, This represents the output feature map of the deconvolution block, where ReLU is the activation function. For normalization layer, These are the weight parameters of the deconvolution layer. For the shift window attention parameters of the Swing Transformer submodule, It is a deconvolution layer. Input feature map to the decoder; This is a submodule of SwinTransformer; After upsampling, the feature map output by the encoder is channel-adjusted, and then concatenated with the feature map output by the corresponding deconvolution block of the decoder. The shallow texture features extracted by the encoder are fused with the deep semantic features by skip connections, as shown in the following formula (9): (9), in, For the decoder i The output feature map of each deconvolution block This is the feature map of the corresponding layer of the encoder. Adjust the weight parameters of the convolution for each channel. The output features are those obtained after skip connection fusion. In this embodiment, the last deconvolution block of the decoder outputs 3 channels, corresponding to the RGB three channels, and outputs the restored image through the Tanh activation function. I c ,

[0040] in, This is the output feature map of the fourth deconvolution block of the decoder; By combining skip connection feature fusion with the synergistic effect of the Swing Transformer sub-blocks, the decoder can perform skip connection fusion of the shallow texture features and deep semantic features extracted by the encoder. This compensates for the loss of detail features during image upsampling and further refines contaminated detail features, thereby improving the clarity and detail representation of the restored image.

[0041] S3.4. The multi-scale discriminator performs multiple discriminations on the feature map output by the decoder. The multi-scale discriminator includes a global discriminator, a chromaticity discriminator, and a texture discriminator set in parallel branches. The overall discrimination function of the multi-scale discriminator is shown in the following formula (10): (10) in, x This is the input to the multi-scale discriminator, i.e., the input feature map. I c , For multi-scale discriminator parameters, D g (·), D c (·), D t (·) represent the outputs of the global discriminator, chroma discriminator, and texture discriminator, respectively. These are the loss weights corresponding to the global discriminator, chroma discriminator, and texture discriminator, respectively, and were determined through experimental debugging. The values ​​are 1.0, 0.5, and 0.7, respectively, to balance the discrimination contributions of different discriminators; The global discriminator consists of 5 convolutional blocks and 1 output layer. Each convolutional block includes a 4×4 convolution (with a stride of 2), a normalization layer, and... Sigmoid The activation function (slope 0.2) and the global discriminant function are shown in formula (11) below: (11), in, For global branch convolution sequence operations, The parameters are preset global discriminator parameters, and σ(•) is... Sigmoid Activation function, global discriminator output ∈[0, 1], representing the global realism of the input image; the global discriminator can identify the global structural differences in the water pollution image, suppress global geometric distortion in the restored image, and verify the integrity of the overall water area outline of the river. The chromaticity discriminator consists of two convolutional blocks, with the input being the CbCr chromaticity space features of the image. The chromaticity discriminant function is shown in the following formula (12): (12), in, RGB2CbCr (·)for RGB Space to CbCr Space transformation functions, ConvSeq c (·) represents the chroma branch convolution sequence operation, and σ(•) is... Sigmoid Activation function Preset the parameters for the colorimetric discriminator; the colorimetric discriminator can determine whether the colorimetric values ​​of the input images are the same, monitor the changes in the blue-green color of the water, correct the color difference distortion of polluted water, reduce the color shift caused by turbulence distortion, and thus ensure the spectral fidelity of the restored image. The texture discriminator consists of two convolutional blocks. The input is the high-frequency texture features of the image extracted by the difference in Gaussian filtering (DoG). The texture discrimination function is shown in the following formula (13): (13) in, DoG (·) represents the Gaussian difference filtering operation. ConvSeq t (·) represents the convolution sequence operation for texture branches, and σ(·) is... Sigmoid Activation function Preset texture discriminator parameters; The texture discriminator can repair the micro-textures of water surface ripples and shoreline boundaries, improve the authenticity of the texture features of water pollution details, drive the generator to optimize image detail features, improve image restoration effect, and reduce image false clarity.

[0042] S4. Construct a hybrid loss function that includes L1 loss, perceptual loss, adversarial loss, and spectral loss. The hybrid loss function is shown in the following formula (14): (14) in, For a mixed loss function, For L1 loss, In order to perceive loss, To combat the losses, For spectral loss, These are the weighting coefficients corresponding to L1 loss, perceptual loss, adversarial loss, and spectral loss, used to balance the influence of different loss terms, and were determined through experimental tuning. The values ​​are 1.0, 0.1, 0.01, and 0.5. L1 loss measures the pixel-level difference between the restored image and the ideal sharp image. It can effectively suppress noise and improve the objective accuracy of image restoration. Its expression is: , in, H, W These are the height and width of the input image, respectively. B This represents the number of spectral bands. G(x) h,w,b To predict image pixels ( h, w ) No. b Band values, y h,w,b The value corresponds to the location in the real image; Perceptual loss, based on the extraction of high-level semantic features of an image, measures the difference between the restored image and the ideally sharp image at the semantic feature level. It can effectively preserve the details, textures, and visual realism of the image, avoiding pseudo-sharpness. Its expression is: , in, Φ i (·) indicates the first (·) of the pre-trained VGG-16 network. i Layer extractor; C i 、H i 、W i The first i The channels, height, and width of the layer feature map; for L2 The square of the norm; To combat loss, we take the expected value to eliminate batch fluctuations; its expression is:

[0043] in, E [·] represents the mathematical expectation. D (·) represents the discriminator; The spectral loss is a weighted SAM spectral loss, which minimizes the angle between the predicted and the real image's pixel-level spectral vectors, weights and highlights sensitive bands for marine water pollution, and includes pixel normalization. Its expression is: , in, s h,w (·) represents pixels ( h, w ) B 3D spectral vector; (·) T Transpose of a vector; It is an L2 norm; ε =10 -6 , ω b For the first b Band weighting (in marine pollution) w b =1.0); In this embodiment, an NVIDIA RTX 3090 GPU is used, and the batch size is set to 7. The training set is imported into the pyramid attention generative adversarial network for the first and second stages of training by sequentially changing the weight coefficients corresponding to the adversarial loss and the weight coefficients corresponding to the adversarial loss and the perceptual loss, respectively. A total of 100 training rounds are conducted, and the water image restoration model is obtained after training.

[0044] In the first phase of training (rounds 1-80), the weight coefficients corresponding to L1 loss, perceptual loss, and spectral loss in the hybrid loss function are kept unchanged, while the weight coefficients corresponding to adversarial loss are reduced. The specific reduction range can be preset by each reduction value, such as 0.1 each time. The training set is then imported into the Pyramid Attention Generative Adversarial Network for the first phase of training. In this way, the first phase of training can prioritize the repair of overall image blur and large-scale turbulence distortion, ensuring the clarity and accuracy of the global contour of the water area in the image. In the second phase of training (rounds 81-100), the weight coefficients corresponding to L1 loss and spectral loss in the hybrid loss function are kept constant, while the weight coefficients corresponding to adversarial loss and perceptual loss are increased. The specific range of increase can be preset by decreasing the value each time, such as increasing by 0.1 each time. The training set is then imported into the pyramid attention generative adversarial network for the second phase of training. This second phase of training can optimize the microscopic texture of the water surface and the detailed features of the water-land boundary in the image, avoiding excessive smoothing of the image. The model with the best image clarity, accuracy, and edge detail features after two phases of training is determined to be the water image restoration model.

[0045] S5. The collected water area images are re-input into the water area image restoration model to obtain a distortion-free water area restoration image. Then, the water area restoration image is input into the intelligent assessment system for water and soil composite pollution, and the water area segmentation and pollution features are extracted. The spatial location of water pollution and the pollution diffusion range are entered into the water and soil pollution knowledge graph to determine the spatial entity relationship. Then, the water area restoration image is input into the knowledge-enhanced big data language model. Combined with soil physicochemical test data and historical water quality data, the system completes the dynamic pollution assessment, automatic spatial profiling, and pollution source tracing analysis, and finally outputs the zoning governance decision. In this embodiment, the spatial entity relationship represents, for example, the corresponding relationship between the spatial location of water pollution, the pollution diffusion range, and the pollution situation to form a water and soil pollution knowledge graph. However, the enhanced big data language model is an existing model. It determines the governance decision based on the image and related parameters. The formation of the governance decision is determined by existing principles. For example, if there is a pollution location and the current pollution situation in the input water area restoration image, it is determined that subsequent surface garbage cleaning is needed in the current area. The specific governance decisions required for different pollution locations and different pollution situations are preset in the system or can be learned by the system. The specific process is existing technology.

[0046] In this embodiment, reference Figure 4 Select AT-DDPM ( Figure 4 (c)), ESR-GAN ( Figure 4 (d)), DATUM ( Figure 4 (e)), SR-GAN ( Figure 4 (f)), TSR-WGAN ( Figure 4 (g)), TURBNET ( Figure 4(h) Six turbulence restoration models were used as benchmarks. All six models are existing publicly available remote sensing image deblurring and turbulence distortion repair algorithms. The core implementation schemes and technical principles of each model are as follows: (1) AT-DDPM: It is an adaptive turbulence deblurring algorithm based on the diffusion probability model. Relying on the core architecture of the DDPM diffusion model, it simulates atmospheric turbulence degradation noise by superimposing it through the forward diffusion process, generates clear images by reverse iteration, introduces an adaptive time step adjustment mechanism to adapt to remote sensing turbulence blurring scenarios of different intensities, and achieves overall image restoration by multi-round iterative noise reduction. It is the current mainstream benchmark model for image restoration based on the diffusion model. (2) ESR-GAN: It is an enhanced super-resolution generative adversarial network. Based on the traditional SR-GAN optimization and improvement, it uses generative adversarial network as the basis, extracts image features through deep residual convolution module, and combines global residual learning strategy to optimize image super-resolution and deblurring effect, improve the resolution and overall clarity of remote sensing images, and is widely used in the basic restoration scenario of aerial remote sensing images. (3) DATUM: A model for turbulent image restoration based on dynamic attention. It builds a feature extraction module based on the Transformer architecture and captures global image dependencies through a dynamic adaptive attention mechanism. It optimizes and repairs image distortion and detail blur caused by atmospheric turbulence. It can effectively adapt to remote sensing image restoration tasks with mild to moderate turbulence interference and is a classic comparative model in the field of remote sensing turbulence restoration. (4) SR-GAN: A classic super-resolution generative adversarial network. It is a basic model for traditional deep learning image restoration. It uses convolutional neural networks as the feature extraction backbone and combines adversarial training mechanisms to achieve image deblurring and super-resolution reconstruction. Through basic convolutional feature extraction and adversarial loss constraints, it completes global blur restoration of remote sensing images and serves as the basic benchmark architecture for various GAN-type image restoration models. (5) TSR-WGAN: This is an improved turbulence image restoration model based on WGAN. It introduces Wasserstein distance to replace the traditional GAN ​​adversarial loss, solves the problem of vanishing training gradients, and combines multi-scale convolution modules to extract shallow and deep features of the image. It performs global restoration of the turbulence blur problem in remote sensing images, but its adaptability to the restoration of water-specific color and micro-texture details is poor. (6) TURBNET: This is a dedicated deep learning model for atmospheric turbulence image restoration. It adopts an encoding and decoding infrastructure and combines a multi-scale feature fusion strategy to process turbulence-degraded images. It optimizes the image distortion and blur problem caused by atmospheric turbulence. It has excellent performance in conventional land remote sensing turbulence image restoration and is the mainstream comparative model in the field of water and soil remote sensing turbulence restoration.

[0047] The results of the comparison of core objective indicators are as follows: This invention ( Figure 4(i) The PSNR value reached 23.894, which is 0.15% higher than the second-best model TURBNET (23.859) and 17.65% higher than the worst model TSR-WGAN (20.309), indicating that it has the best pixel-level restoration accuracy and stronger anti-turbulence distortion ability; the SSIM value of this invention reached 0.673, which is 4.50% higher than the second-best model DATUM (0.644) and much higher than the worst model TSR-WGAN (0.511), indicating that it can more fully preserve the image structure similarity and has a more prominent restoration effect on key details of oil pollution boundaries; the LPIPS value of this invention is only 0.311, which is 15.26% lower than the second-best model DATUM (0.367) and 38.90% lower than the worst model TSR-WGAN (0.509); it can completely preserve the details of water surface pollutants and pollution zones at the water-land interface, without color distortion or edge blurring problems, and the restoration effect is far better than the above models.

[0048] The working principle of this invention is as follows: After extracting the water area image, a pyramid-based generative adversarial network (GAN) is used to generate feature maps of different scales from the pyramids of multi-scale features. Then, a multi-head attention module forms a comprehensive attention weight from window-based multi-head attention and spatial attention. This allows the feature maps of different scales to be filled with local image details and spatial correlations from both the window and spatial perspectives, thereby forming attention-enhanced feature maps of different scales. By adaptively focusing on key pollution areas in the water area through the multi-head attention module, it can accurately handle the unique non-uniform atmospheric turbulence noise near the water surface, adapting to the complex interference environment of aerial photography close to the water surface and avoiding the limitation of not being able to repair local turbulence blur. Furthermore, through global, chromatic, and texture triple discrimination, the water area map can be identified separately. The model preserves the outline, color, and micro-texture of the water bodies, thus retaining the water area division, color difference, and turbidity gradient pollution characteristics from aerial photography. This ensures that the restored image perfectly matches the actual aerial view and can distinguish whether the color difference is caused by water color difference or surface noise. By performing segmented training on the restored image, adjusting the adversarial loss separately and combining adversarial loss and perceptual loss in two stages, the model first ensures that the overall water area image is distortion-free, and then refines key details such as the water-land boundary and water surface ripples, avoiding excessive smoothing of the restored image edges. This ensures the reliability of the entire water area image restoration model, allowing subsequent models to obtain the restored image of Wushi Town's water area simply by inputting the collected water area map into the model, further improving the accuracy of pollution feature extraction.

Claims

1. A method for restoring remote sensing turbulent images for monitoring combined water and soil pollution, characterized in that: Includes the following steps: S1. Collect water area images, and then construct a water system remote sensing paired dataset with water area images that are disturbed by turbulence blur, motion blur, and water surface reflection degradation and water area images without disturbance. S2. Standardize and preprocess the collected water area images, and divide the water system pairing dataset into training set and test set according to the preset ratio; S3. Construct a pyramid attention generative adversarial network based on the Swin-U-Net backbone network. Extract multi-scale features from the image of the interfering water area through the encoder, and then generate feature maps of different scales through feature pyramid generation. The multi-head attention module determines the comprehensive attention weight of each scale feature map by combining window multi-head attention and spatial attention. After fusing the feature maps of different scales with the comprehensive attention weight, attention-enhanced feature maps of different scales are obtained. The decoder decodes the attention-enhanced feature maps of different scales and outputs feature maps. Then, the multi-scale discriminator performs global, color and texture discrimination on the feature maps output by the decoder to restore the image. S4. Construct a hybrid loss function that includes L1 loss, perceptual loss, adversarial loss and spectral loss, and change the weight coefficients corresponding to the adversarial loss and the weight coefficients corresponding to the adversarial loss and perceptual loss in turn to import the training set into the pyramid attention generative adversarial network for two-stage training. After training, the water image restoration model is obtained. S5. Re-input the acquired water area images into the water area remote sensing image restoration model to obtain a distortion-free water area restoration image.

2. The remote sensing turbulence image restoration method for monitoring water and soil composite pollution according to claim 1, characterized in that: Step S5 also includes: inputting the water area restoration image into the intelligent assessment system for water and soil composite pollution, extracting the water area segmentation and pollution characteristics, recording the spatial location of water pollution and the pollution diffusion range into the water and soil pollution knowledge graph, determining the spatial entity relationship, then inputting the water area restoration image into the knowledge-enhanced big data language model, combining soil physicochemical test data and historical water quality data to complete the dynamic pollution assessment, automatic spatial profiling, pollution source tracing analysis, and finally outputting the zoned governance decision.

3. The remote sensing turbulence image restoration method for monitoring water and soil composite pollution according to claim 1, characterized in that: Step S3 includes: steps S3.1-S3.

4. S3.

1. Multi-scale features of water area images affected by turbulence are extracted by an encoder. The encoder includes one or more Swin Block modules based on the Swin Transformer architecture. The water area image is used as input feature. After passing through the Swin Block module, the first feature map is obtained. Then, it is pooled by the max pooling layer connected to the output of the Swin Block module and the second feature map is output. The Swin Block module includes two window attention layers, two shift window attention layers, one normalization layer and one ReLU activation function. The encoder processing process is shown in the following formulas (1) and (2): （1）， （2）， F swin This is the first feature map of the Swing Block module, where ReLU is the activation function. For normalization layer, For shift window attention layer, For window attention layer, F in This is the input feature map for the Swing Block module. θ ω For window attention layer parameters, θ sω For the shift window attention layer parameters, F pool This is the second feature map after pooling. This is a pooling layer, where stride is the layer number.

4. The remote sensing turbulence image restoration method for monitoring water and soil composite pollution according to claim 3, characterized in that: In step S3.1, the encoder includes four Swing Block modules, and the processing procedure through the Swing Block modules includes: Water area image as input feature image I t After passing through the first Swin Block module, the output feature map has 64 channels and a size of 256×256. F 1. Extract shallow texture features through window attention; after passing through the second Swin Block module, output a feature map with 128 channels and a size of 128×128. F 2. Extract mid-layer features; after passing through the third Swin Block module, the output feature map has 256 channels and a size of 64×64. F 3. Extract deep semantic features; after the fourth SwinBlock module, the output feature map has 512 channels and a size of 32×32. F 4. Global features are captured through long-distance attention. The feature extraction process is shown in the following formula (3): （3）， when hour, F 0= I t , θ ω,i , θ sω,i The first i Window attention parameters and shift window attention parameters for each Swin Block module.

5. The remote sensing turbulence image restoration method for monitoring water and soil composite pollution according to claim 3, characterized in that: Step S3.2 includes: constructing a pyramid multi-head attention module (PAM). The PAM includes a feature pyramid, a multi-head attention module, and feature fusion. The PAM processing procedure includes steps S3.2.1 to S3.2.

3. S3.2.1, Apply convolutional kernels of three different scales to the feature map from step S3.

1. F 4. Perform parallel convolution operations to generate three feature maps of different scales, corresponding to small-scale, medium-scale, and large-scale feature maps respectively, to form a feature pyramid, as shown in the following formula (5): （5）， θ s / m / l These are the weight parameters for three preset convolutional kernels with different scales. F s , F m , F l These correspond to small-scale feature maps, medium-scale feature maps, and large-scale feature maps, respectively. S3.2.

2. Adaptive average pooling is performed on feature maps at different scales. In the multi-head attention of the window in the Swin-U-Net backbone network, the global spatial features of the feature maps are compressed into channel features, and then connected to the network through two fully connected layers. Sigmoid The activation function calculates the channel attention weights, and then, in the spatial attention of the Swin-U-Net backbone network, 1×1 convolutions are performed on the feature maps of different scales for dimensionality reduction. Sigmoid The activation function calculates the spatial attention weights, and the channel attention weights are multiplied by the spatial attention weights to obtain the comprehensive attention weights corresponding to each feature map at different scales, as shown in the following formula (6): （6）， The comprehensive attention weights are for feature maps of different scales. These are the channel attention weights corresponding to feature maps of different scales. Spatial attention weights corresponding to feature maps at different scales are determined by... Sigmoid The activation function is normalized to the [0,1] interval; S3.2.3 Multiply each feature map at different scales with its corresponding comprehensive attention weight to obtain enhanced feature maps at different scales. Then, concatenate the enhanced feature maps at different scales and reduce their dimensionality through 1×1 convolution to obtain the fused attention-enhanced feature map, as shown in the following formula (7): （7）， Concat(·) is a channel concatenation operation. These are the weight parameters for the fusion convolution.

6. The remote sensing turbulence image restoration method for monitoring water and soil composite pollution according to claim 3, characterized in that: Step S3.3 includes: The decoder outputs the attention-enhanced feature map from the Pyramid Multi-Head Attention Module (PAM). F PAM The spatial resolution of the image is gradually restored by upsampling through one or more deconvolution blocks, and the output is the same as the input feature image. I t Restored images of the same size I c In step S3.3, the decoder processing includes: the decoder adopts a structure symmetrical to the encoder, including 4 deconvolution blocks and 4 Swin Transformer blocks; each deconvolution block includes a 2×2 deconvolution layer, a Swin Transformer submodule, a normalization layer and a ReLU activation function, as shown in the following formula (8): (8), This represents the output feature map of the deconvolution block, where ReLU is the activation function. For normalization layer, These are the weight parameters of the deconvolution layer. For the shift window attention parameters of the Swing Transformer submodule, It is a deconvolutional layer; Input feature map to the decoder; This is a submodule of Swin Transformer; After upsampling, the feature map output by the encoder is channel-adjusted, and then concatenated with the feature map output by the corresponding deconvolution block of the decoder. The shallow texture features extracted by the encoder are fused with the deep semantic features by skip connections, as shown in the following formula (9): （9）， For the decoder i The output feature map of each deconvolution block This is the feature map of the corresponding layer of the encoder. Adjust the weight parameters of the convolution for each channel. This refers to the output features after skip connection fusion.

7. A method for restoring remote sensing turbulent images for monitoring combined water and soil pollution according to claim 3, characterized in that: Step S3.4 includes a multi-scale discriminator performing multiple discriminations on the feature map output by the decoder. The multi-scale discriminator includes a global discriminator, a chromaticity discriminator, and a texture discriminator set in parallel branches. The overall discrimination function of the multi-scale discriminator is shown in the following formula (10): （10）， x For input to the multi-scale discriminator, For multi-scale discriminator parameters, D g (·), D c (·), D t (·) represent the outputs of the global discriminator, chroma discriminator, and texture discriminator, respectively. , , These are the loss weights corresponding to the global discriminator, chroma discriminator, and texture discriminator, respectively.

8. The remote sensing turbulence image restoration method for monitoring water and soil composite pollution according to claim 7, characterized in that: Step S3.4 also includes: The global discriminator consists of 5 convolutional blocks and 1 output layer. Each convolutional block includes a 4×4 convolution, a normalization layer, and... Sigmoid The activation function and the global discriminant function are shown in the following formula (11): （11）， For global branch convolution sequence operations, The parameters are preset global discriminator parameters, and σ(•) is... Sigmoid Activation function, global discriminator output ; The chromaticity discriminator consists of two convolutional blocks, and the chromaticity discriminant function is shown in the following formula (12): （12）， RGB2CbCr (·)for RGB Space to CbCr Space transformation functions, ConvSeq c (·) represents the chroma branch convolution sequence operation, and σ(•) is... Sigmoid Activation function Preset the parameters for the colorimetric discriminator; The texture discriminator consists of two convolutional blocks, and the texture discrimination function is shown in the following formula (13): （13）， DoG (·) represents the Gaussian difference filtering operation. ConvSeq t (·) represents the convolution sequence operation for texture branches, and σ(·) is... Sigmoid Activation function These are the preset texture discriminator parameters.

9. A method for restoring remote sensing turbulent images for monitoring combined water and soil pollution according to claim 1, characterized in that: The hybrid loss function in step S4 is shown in formula (14) below: （14）， For a mixed loss function, For L1 loss, In order to perceive loss, To combat the losses, For spectral loss, These are the preset weight coefficients corresponding to L1 loss, perceptual loss, adversarial loss, and spectral loss, respectively.

10. A method for restoring remote sensing turbulent images for monitoring combined water and soil pollution according to claim 1, characterized in that: Step S4 also includes: Keep the weight coefficients corresponding to L1 loss, perceptual loss and spectral loss in the hybrid loss function unchanged, reduce the weight coefficients corresponding to adversarial loss, and import the training set into the pyramid attention generative adversarial network for the first stage of training. Keeping the weight coefficients corresponding to L1 loss and spectral loss in the hybrid loss function unchanged, while increasing the weight coefficients corresponding to adversarial loss and perceptual loss, the training set is imported into the pyramid attention generative adversarial network for the second stage of training.

Images