A remote sensing image defogging method and device and a storage medium

Abstract

The application discloses a remote sensing image defogging method and device and a storage medium, relates to the field of image processing, and drives deep learning by using dark channel prior knowledge, and designs a remote sensing image defogging method based on a dark channel attention network. The network adopts a parallel architecture of attention flow and dark channel prior constraint flow. In the attention flow, an image extracts feature information through an encoder-decoder and combines a channel space attention structure to enhance the features. In the dark channel prior constraint flow, a dark channel prior loss is used to improve the performance of the dark channel attention network, to transmit feature information for the attention flow and enhance the ability of the attention flow to learn image features. The application has good remote sensing image defogging performance, good robustness and adaptability for various types of foggy remote sensing image data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and particularly relates to a remote sensing image defogging method and device based on a dark channel attention network and a storage medium. BACKGROUND

[0002] Remote sensing images have the advantages of high spatial resolution and wide coverage, and have been widely applied to resource exploration, ocean monitoring, land classification, semantic segmentation and other fields. Clear remote sensing images are a prerequisite for many remote sensing image analysis and processing systems and algorithms. However, during the acquisition of remote sensing images, the images often appear blurred and lose details due to the interference of fog, which seriously affects subsequent analysis and use.

[0003] At present, the problem of remote sensing image defogging is:

[0004] 1. Prior-based defogging methods rely on atmospheric scattering models and image prior information, and cannot adapt to complex and variable scene environments and have unstable defogging effects;

[0005] 2. Learning-based methods cannot effectively utilize existing key feature details in images, and learning-based methods need to learn a large amount of image feature information, which is prone to data overfitting problems during training.

[0006] Based on the above problems, the existing remote sensing image defogging method is still difficult to effectively remove the haze information in the remote sensing image. SUMMARY

[0007] The present application provides a remote sensing image defogging method and device based on a dark channel attention network and a storage medium. The present application uses a parallel cascade architecture of attention flow and dark channel prior constraint flow in the dark channel attention network structure, and processes the foggy remote sensing image in an end-to-end manner. Details are described below:

[0008] A remote sensing image defogging method, the method comprising:

[0009] According to the flow direction of the feature map in the dark channel attention network, the real haze-free remote sensing image and the defogging remote sensing image output by the dark channel attention network are taken as a first image pair, and the defogging remote sensing image processed by the dark channel prior algorithm and the constraint remote sensing image output by the dark channel prior constraint flow are taken as a second image pair. The loss function values of the two groups of image pairs are calculated using the loss function formula respectively;

[0010] The loss function value of the dark channel attention network is minimized, and the values of the parameters in the network are iteratively updated using deep neural network backpropagation until the termination condition is met, and the training of the dark channel attention network is completed. The dark channel attention network is trained using a synthetic foggy remote sensing image training set.

[0011] The dark channel attention network comprises an attention flow and a dark channel prior constraint flow,

[0012] The attention flow comprises a first encoder-decoder structure and a channel-spatial attention structure.

[0013] The dark channel prior constraint flow comprises a second encoder-decoder structure.

[0014] The channel-spatial attention structure is an attention structure that has a convolution residual block, a channel attention mechanism and a spatial attention mechanism.

[0015] Further, the first encoder-decoder structure is composed of a convolution block and a deconvolution block. The convolution block is composed of five groups of structure units, each of which comprises a LeakyReLU activation function, a convolution layer, a normalization layer and a Dropout layer. The deconvolution block is composed of five groups of structure units and a group of convolution layers, each of which comprises a ReLU activation function, a transpose convolution layer, a normalization layer and a Dropout layer.

[0016] The training of the dark channel attention network with the synthetic foggy remote sensing image training set comprises the following steps:

[0017] The dark channel attention network adopts a parallel architecture of the attention flow and the dark channel prior constraint flow. In the attention flow, an image extracts feature information through an encoder-decoder and enhances the features in combination with a channel-spatial attention structure. In the dark channel prior constraint flow, a dark channel prior loss is used to pass feature information to the attention flow. The attention flow and the dark channel prior constraint flow share feature information and jointly complete the remote sensing image defogging task.

[0018] Further, the processing of the foggy remote sensing image through the dark channel attention network and the output of the defogged remote sensing image comprise the following steps: loss, edge loss The defogged remote sensing image d is calculated by h the error between the defogged remote sensing image d and the paired real haze-free remote sensing image d, through loss, the constraint remote sensing image d output by the dark channel prior constraint flow in the dark channel attention network is calculated by h the difference between the haze-free remote sensing image d' processed by the dark channel prior algorithm, and the defogged remote sensing image d' output by the dark channel attention network, and the parameters of the dark channel attention network are updated through back propagation to minimize the loss error:

[0019]

[0020] wherein ||*||1 is the 1-norm, d and d h respectively represent the real haze-free remote sensing image and the defogged remote sensing image obtained by processing through a preset network;

[0021]

[0022] wherein, E Sobelx (·) represents edge detection in the horizontal direction, E Sobely (·) represents edge detection in the vertical direction.

[0023]

[0024] wherein, d' and d h respectively represent a non-fog remote sensing image processed by a dark channel prior algorithm and a constraint remote sensing image obtained by processing a dark channel prior constraint flow in a dark channel attention network;

[0025] A remote sensing image defogging device, the device comprising: a processor and a memory, the memory having stored therein program instructions, the processor invoking the program instructions stored in the memory to cause the device to perform the method steps of any one of the claims.

[0026] A computer-readable storage medium storing a computer program, the computer program comprising program instructions, the program instructions being executed by a processor to cause the processor to perform the method steps of any one of the claims.

[0027] The beneficial effects of the technical solutions provided by the present application are:

[0028] 1) The remote sensing image defogging method provided by the present application effectively utilizes the key feature information in the image based on the dark channel attention network, and retains more image texture;

[0029] 2) The channel space attention structure and edge loss are integrated into the dark channel attention network, the attention degree of important feature information of the network is improved, the clarity of the scene edge in the image is improved, and the defogging performance is effectively improved;

[0030] 3) The remote sensing image defogging method provided by the present application has high robustness and stability, not only has significant defogging performance for remote sensing images with uniform haze distribution, but also has high generalization ability for remote sensing image datasets with uneven haze distribution. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 It is a flowchart of a remote sensing image defogging method;

[0032] Figure 2 It is a schematic diagram of the overall structure of the dark channel attention network model;

[0033] Figure 3 It is a synthetic fog remote sensing image and defogging effect diagram;

[0034] Wherein, (a) is a synthetic fog remote sensing image schematic diagram; (b) is a dark channel attention network defogging effect diagram.

[0035] Figure 4 For real fog remote sensing image and defogging effect diagram;

[0036] Wherein, (a) is a real fog remote sensing image schematic diagram; (b) is a dark channel attention network defogging effect diagram; (c) is a dark channel prior algorithm defogging effect diagram.

[0037] Figure 5 It is a structural schematic diagram of a remote sensing image defogging device. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application are further described in detail below.

[0039] The following first explains and describes the technical terms related to the embodiments of the present application:

[0040] Neuron: In a convolutional neural network, a neuron is a processing unit in the network, which is a placeholder for a mathematical function, uses a function on the input to get an output.

[0041] Convolution layer: The convolution calculation formula is: N=(W-F+2P) / S+1, wherein N represents the output channel number, W represents the input channel number, F represents the convolution kernel size, P represents the padding value size, and S represents the step size. The convolution kernel is a matrix of FxF fixed weights, which moves in the horizontal and vertical directions of the original image pixel matrix according to the preset step size (i.e. from left to right and from top to bottom), and performs convolution operation by element-wise multiplication and addition of the data of different data windows in the original image and the convolution kernel. The padding value is introduced to avoid the problem of image shrinking and loss of image edge information after convolution operation by filling the pixel points around the original image (the pixel value of the pixel point is preset to 0 in the embodiments of the present application).

[0042] Pooling layer: used to reduce the image size, reduce information redundancy and improve calculation speed. The maximum value or average value is taken in the filter sliding area.

[0043] ReLU activation layer: the expression form is: f(x)=max(0,x), wherein x represents the input value of the network in this layer, and the ReLU function is a piecewise linear function that changes all negative values to 0 while keeping the positive values unchanged. This operation is called unilateral inhibition, which makes part of the neurons in the neural network activated, and is used to improve the calculation efficiency.

[0044] LeakyReLU activation layer: the expression form is: f(x) = max(a*x, x), wherein a represents a fixed parameter in the interval (0, 1), and a = 0.2 in the embodiment of the application. It is used to solve the problem that the ReLU function enters the negative interval, causing the neuron not to learn.

[0045] Sigmoid activation layer: the expression form is: When x tends to negative infinity, f(x) tends to 0; when x tends to positive infinity, f(x) tends to 1; when x = 0, f(x) = 0.5.

[0046] Normalization layer: normalization is to scale the value of a column of numerical features in the training set to between 0 and 1. After data normalization, the optimization process of the optimal solution will obviously become smooth, and it is easier to correctly converge to the optimal solution.

[0047] Attention mechanism: human vision obtains the attention focus of the target area that needs to be focused on by quickly scanning the global image, and then invests more attention resources in this area to obtain more detailed information of the target that needs to be focused on, while ignoring other useless information, greatly improving the efficiency and accuracy of visual information processing. Adding an attention mechanism to a deep neural network can help the model assign different weights to each part of the input, extract more key and important information, and make accurate judgments, while not causing a large overhead in model calculation and storage.

[0048] Embodiment 1

[0049] In view of the problem that the existing technology brings fog to remote sensing images, affecting subsequent analysis and processing, the embodiment of the application provides a remote sensing image defogging method, which is described with reference to Figure 1 The method comprises the following steps:

[0050] 101: According to the flow direction of the feature map in the dark channel attention network, the dark channel attention network is composed of an attention flow and a dark channel prior constraint flow;

[0051] The dark channel attention network comprises: an attention flow and a dark channel prior constraint flow, the attention flow comprises: a first encoder-decoder structure E1→D1 and a channel-space attention structure; the dark channel prior constraint flow comprises: a second encoder-decoder structure E2→D2. The channel-space attention structure is an attention structure with a convolution residual block, a channel attention mechanism and a spatial attention mechanism residual joint; the first encoder-decoder structure E1→D1 and the second encoder-decoder structure E2→D2 are a deep neural network with convolution and deconvolution operations known to the public, that is, a convolutional neural network is used to extract features from an input image, and then a deconvolution layer is used to restore low-level features from a feature vector.

[0052] 102: calculate the loss function value;

[0053] The real haze-free remote sensing image and the haze-removed remote sensing image output by the dark channel attention network are taken as a first image pair, the haze-removed remote sensing image processed by the dark channel prior algorithm and the constraint remote sensing image output by the dark channel prior constraint stream are taken as a second image pair, and the loss function values of the two image pairs are calculated by using the loss function formula respectively.

[0054] 103: update the dark channel attention network parameters;

[0055] The loss function value of the dark channel attention network is minimized, the values of the parameters in the dark channel attention network are iteratively updated by using the deep neural network back propagation algorithm, until the termination condition is met, and the training of the dark channel attention network is completed.

[0056] The deep neural network back propagation algorithm is well known to those skilled in the art, and will not be described here.

[0057] 104: training: the dark channel attention network is trained by using a synthetic haze-removed remote sensing image training set;

[0058] The dark channel attention network adopts a parallel architecture of attention stream and dark channel prior constraint stream. In the attention stream, the image extracts feature information by using an encoder-decoder and enhances the features by combining a channel spatial attention structure. In the dark channel prior constraint stream, the dark channel prior loss is used to improve the performance of the dark channel attention network. The attention stream transmits feature information to the dark channel prior constraint stream, and enhances the ability of the attention stream to learn image features. The attention stream and the dark channel prior constraint stream share feature information, and jointly complete the remote sensing image haze removal task.

[0059] 105: input a haze-removed remote sensing image h, and the haze-removed remote sensing image h is processed by the dark channel attention network and output as a haze-removed remote sensing image d h , the error between the haze-removed remote sensing image d h and the paired real haze-free remote sensing image d is calculated by using the following loss formula (1) and edge loss formula (2), and the parameters of the dark channel attention network are updated by back propagation to minimize the loss error.

[0060]

[0061] Wherein, ||*||1 is the 1-norm, d and d h represent the real haze-free remote sensing image and the haze-removed remote sensing image obtained by processing the dark channel attention network respectively, and the two images are required to be paired.

[0062]

[0063] ​wherein E Sobelx (·) represents edge detection in the horizontal direction, E Sobely (·) represents edge detection in the vertical direction.

[0064] 106: Integrate channel space attention structure to achieve better remote sensing image defogging effect;

[0065] wherein, in order to further realize the above-mentioned invention, in the attention flow, the channel space attention structure is integrated to improve the prediction accuracy to achieve better defogging effect. The channel space attention structure is composed of a convolution residual block, a channel attention mechanism and a spatial attention mechanism, which can improve the defogging effect in the convolution feature dimension, the channel feature dimension and the spatial feature dimension.

[0066] wherein, the channel attention mechanism determines the key attention content of the feature map in the channel dimension by modeling the mutual relationship between different channels. In the remote sensing image defogging task, the channel attention mechanism can assign different weights to each channel of the image, so that the dark channel attention network can distinguishably process the channel information of the foggy area and the channel information of the non-foggy area. The spatial attention mechanism assigns different weights to different regions in the image by calculating the mean of different channels of the same pixel point, helping the dark channel attention network pay more attention to the task area in the image. A skip connection is added between the channel attention mechanism and the spatial attention mechanism to fuse the shallow feature information and the channel feature information, and increase the reusability of the feature information.

[0067] 107: In the channel attention mechanism module, the focus is on the dependency relationship between the feature map channels, and the channel features can be adaptively corrected. The channel attention mechanism module first compresses the feature map with a size of CxHxW to Cx1x1 through a global average pooling operation. Then the obtained feature map is fed into two fully connected layers, and then fed into the ReLU activation function and the Sigmoid activation function to obtain a weight matrix, and the weight matrix is multiplied with the input feature map to obtain a channel attention feature map. The processing process of the channel attention mechanism module is shown in formula (3):

[0068] F CA =σ sigmoid (H FC (σ ReLU (H FC (H avg (F))))), (3)

[0069] wherein F is the input of the channel attention mechanism module, σ represents the activation function, H avg (·) is a global average pooling operation, H FC (·) represents a fully connected operation, F CAAn output of the channel attention mechanism module.

[0070] The spatial attention mechanism module assigns different weights to different regions in the image by calculating the mean value of different channels of the same pixel point, and helps the dark channel attention network pay more attention to the task region in the image.

[0071] F SA sigmoid C avg max

[0072] wherein, F is the input of the spatial attention mechanism module, sigma represents an activation function, H avg (·) is a global average pooling operation, H max (·) is a maximum pooling operation, H C (·) represents a convolution operation, [·] represents a connection operation, and F SA represents the output of the spatial attention mechanism module.

[0073] 108: The foggy remote sensing image h is processed by the dark channel prior constraint flow to obtain a constraint remote sensing image d h ′, according to the loss formula (5), the difference between the remote sensing image d h ′ processed by the dark channel prior algorithm and d ′ is calculated, the loss value is obtained, and the network parameters are updated by back propagation;

[0074]

[0075] 109: Steps 101 and 102 and steps 103 and 104 are simultaneously iteratively trained, and the training is stopped when the set number of training cycles is reached;

[0076] 110: The output test set remote sensing image defogging image: when the dark channel attention network is trained, the test foggy remote sensing image to be recovered is input into the dark channel attention network which has been trained, and the remote sensing image defogging image of the test set is output by iterative calculation in the network.

[0077] In summary, the embodiment of the present application uses a dark channel attention network to process a foggy remote sensing image, and adds a channel spatial attention structure in the network to obtain multi-dimensional feature information of the image, thereby improving the defogging effect.​​​​​

[0078] Embodiment 2

[0079] The scheme in Embodiment 1 is further introduced below in combination with specific calculation formulas, drawings and examples, and details are described below:

[0080] 201: The remote sensing image defogging problem can be regarded as a conversion problem from an image to an image, that is, a problem of converting a remote sensing image with fog into a remote sensing image without fog;

[0081] A certain number of pairs of remote sensing images with fog and corresponding remote sensing images without fog are required as data basis in the training process of the dark channel attention network. However, in reality, it is difficult to obtain paired remote sensing image data. Therefore, the first step is to construct a remote sensing image dataset. In order to ensure good generalization performance of the subsequent dark channel attention network, the fog distribution type in the remote sensing image should be as diverse as possible. In order to ensure the authenticity and reliability of the dataset, a publicly available public dataset is selected as the original remote sensing image without fog, and a known fog generation algorithm is used to generate a remote sensing image with fog, thereby obtaining a synthetic paired remote sensing image with fog and the corresponding real remote sensing image without fog.

[0082] 202: The dark channel attention network is trained using the synthetic remote sensing image dataset with fog. As shown in Figure 2 The dark channel attention network adopts a parallel architecture of attention flow and dark channel prior constraint flow, (1) the attention flow, which completes the defogging operation on the input remote sensing image with fog h and outputs the remote sensing image d after defogging h (2) the dark channel prior constraint flow: extracts the dark channel information of the input remote sensing image with fog, and shares the dark channel information with the attention flow to complete the defogging task together. The attention flow and the dark channel prior constraint flow of the dark channel attention network each have an encoder-decoder structure, and the encoder-decoder structures of the two are the same. The channel spatial attention structure is added in the attention flow to improve the performance of the dark channel attention network.

[0083] 203: Figure 2 The attention flow structure used in this embodiment is illustrated. As shown in Figure 2 The attention flow in this embodiment is mainly built using the first encoder-decoder structure and the channel spatial attention structure. The encoder is built using a convolution block, which is used to increase the receptive field. The decoder is built using a deconvolution block, which is used to map the image from a small resolution to a large resolution. The channel spatial attention structure focuses on the multi-dimensional feature information of the image. After being processed by the encoder, a feature map of 512x64x64 pixels is output, and then input into the channel spatial attention structure. After being processed by the channel spatial attention structure, it is input into the decoder.

[0084] 204: The first encoder-decoder structure is composed of a convolutional block and a deconvolutional block. The convolutional block is composed of five groups of structure units, each of which includes a LeakyReLU activation function, a convolutional layer, a normalization layer, and a Dropout layer, wherein the convolutional layer is a commonly used structure in deep learning. The convolution kernel size of all intermediate convolutional layers in the convolutional block is set to 4x4 pixels, the convolution kernel moving step is 2, the feature map padding width is 1, and the channel number is 64, 128, 256, 512, and 512 in turn. The deconvolutional block is composed of five groups of structure units and a convolutional layer, each of which includes a ReLU activation function, a transposed convolutional layer, a normalization layer, and a Dropout layer, wherein the transposed convolutional layer is a commonly used structure in deep learning. The convolution kernel size of all intermediate convolutional layers in the deconvolutional block is set to 4x4 pixels, the convolution kernel moving step is 2, the feature map padding width is 1, and the channel number is 512, 512, 256, 128, 64, and 3 in turn. After the image feature information is operated multiple times by the convolutional block and the deconvolutional block, some important feature information will be lost. Therefore, the embodiment of the present application designs a skip connection between the convolutional block and the deconvolutional block to fuse the shallow features and the deep features, so as to help the dark channel attention network to retain more details of the image and thus restore a clear image.

[0085] 205: The channel-spatial attention structure in step 203 includes a convolutional residual block, a channel attention mechanism module, and a spatial attention mechanism module. The channel attention mechanism module determines the key attention content of the feature map in the channel dimension by modeling the mutual relationship between different channels. In the remote sensing image defogging task, the channel attention mechanism can assign different weights to each channel of the image, so that the dark channel attention network can distinguishably process the channel information of the foggy area and the channel information of the non-foggy area. The spatial attention mechanism module assigns different weights to different regions in the image by calculating the mean of different channels of the same pixel point, helping the dark channel attention network to pay more attention to the task area in the image. A skip connection is added between the channel attention mechanism module and the spatial attention mechanism module to fuse the shallow feature information and the channel feature information and increase the reusability of the feature information.

[0086] 206: Figure 2 The channel-spatial attention (CSA) structure includes a convolutional residual block, a channel attention mechanism module, and a spatial attention mechanism module. The channel attention mechanism module and the spatial attention mechanism module are respectively used to extract the channel features and the spatial features of the image, and the convolutional residual block is composed of three double convolutional blocks, which are used to perform convolutional operations on the image to extract the convolutional features of the image, wherein each double convolutional block is composed of two convolutional layers and one ReLU activation function layer. The processing process of the channel attention mechanism block is shown in formula (3) above, and the processing process of the spatial attention mechanism block is shown in formula (4) above.

[0087] 207: Figure 2 (DCP constraint flow) illustrates the dark channel prior constraint flow structure used in this embodiment. For example... Figure 2 As shown, the dark channel prior constraint flow structure adopts a second encoder-decoder structure. The second encoder-decoder structure is consistent with the first encoder-decoder structure in 204.

[0088] 208: In this embodiment, the size of the training set images is uniformly adjusted to 512×512 pixels during the training of the dark channel attention network. The training iteration is 150 times, and the Adam optimization algorithm is used for training optimization, with a learning rate of 0.0002.

[0089] 209: Construct a dark channel attention network and train it; the target loss function of the dark channel attention network is shown in formula (6):

[0090]

[0091] in, This represents the target loss function of the dark channel attention network. This represents the difference between a true haze-free image and a processed dehazed image. Represents the edge loss function. This represents the constrained remote sensing image d′ obtained after processing with the dark channel prior constrained flow. h The difference between the image and the image processed by the dark channel prior algorithm. In the task of dehazing remote sensing images, it is necessary to minimize the loss function to ensure that the dehazed remote sensing image is as close as possible to the real fog-free remote sensing image. The target loss function is expressed by the above formula (6).

[0092] Remote sensing images are characterized by high spatial resolution, and a single remote sensing image typically contains rich ground features and complex and diverse scenes. Due to the complexity of remote sensing images, clear boundary information plays a crucial role in advanced processing applications (such as semantic segmentation and wastewater extraction). Fog, however, severely affects the boundary clarity of remote sensing images. To improve this issue and effectively enhance the edge clarity of defogging remote sensing images using dark channel attention networks, an edge loss function is used for calculation. The edge loss function is expressed as shown in formula (2) above.

[0093] 210: Testing: After training the dark channel attention network, input the hazy remote sensing image dataset into the trained dark channel attention network for testing, and output the dehazing results.

[0094] In summary, the embodiment of the present application solves the problem of the influence of the foggy remote sensing image by using the paired images to perform the defogging task of the remote sensing image in the training process of the dark channel attention network through steps 201-209. The channel space attention structure is integrated into the dark channel attention network to improve the representation of the attention points and the expressiveness of the content of interest, so as to achieve better remote sensing image defogging effect and improve the image edge definition.

[0095] Embodiment 3

[0096] The feasibility of the schemes in embodiments 1 and 2 is verified below in combination with specific evaluation criteria, drawings and tables, and details are described below:

[0097] In order to objectively evaluate the remote sensing image defogging effect, the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) are used as the evaluation criteria of the restored image quality. Given an image I and an image K with a size of m*n pixels, the PSNR and the SSIM are defined as follows:

[0098] (1)

[0099] Wherein, is the mean square error, I(i,j) and K(i,j) are the pixel values of the image I and the image K at the (i,j) position.

[0100] (2)

[0101] Wherein, μ I is the mean value of I, μ K is the mean value of K, is the variance of I, is the variance of K, σ IK is the covariance of I and K, c1=(k1L) 2 , c2=(k2L) 2 are two constants to avoid division by zero, and L is the range of the pixel value, i.e. 256. The default k1=0.01 and k2=0.03.

[0102] The experimental results are shown in Figure 3 (b), the defogging result of the dark channel attention network is natural and the texture details are clear. The comparative experimental results of the embodiment of the present application and the traditional dark channel prior algorithm on the real remote sensing image dataset are shown in Figure 4 , the defogging result of the dark channel prior algorithm is shown in Figure 4 (c), and the effect after defogging is not ideal, and the haze information is not completely removed. The restored result of the dark channel attention network is shown in Figure 4(b) As shown, the defogging target can be achieved in both thin fog area and thick fog area, and the image topographic information can be better restored and the texture detail information of the image can be retained. Table 1 shows the objective evaluation standard parameter comparison of the defogging results of the method and the dark channel prior algorithm.

[0103] Table 1 Quantitative comparison of results of different image defogging methods on synthetic fog remote sensing images

[0104]

[0105]

[0106] Example 4

[0107] A remote sensing image defogging device, referring to Figure 5 The device comprises a processor 1 and a memory 2, and the memory 2 stores program instructions, and the processor 1 calls the program instructions stored in the memory 2 to make the device execute the following method steps in Example 1:

[0108] According to the flow direction of the feature map in the dark channel attention network, the real fog-free remote sensing image and the defogging remote sensing image output by the dark channel attention network are taken as a first image pair, and the defogging remote sensing image processed by the dark channel prior algorithm and the constraint remote sensing image output by the dark channel prior constraint flow are taken as a second image pair, and the loss function values of the two groups of image pairs are calculated respectively by using the loss function formula.

[0109] The loss function value of the dark channel attention network is minimized, the deep neural network is backward propagated, the values of the parameters in the network are iteratively updated until the termination condition is met, and the training of the dark channel attention network is completed; and the dark channel attention network is trained by using the synthetic fog remote sensing image training set.

[0110] The dark channel attention network comprises:

[0111] The dark channel prior constraint flow extracts the prior image feature f' from the input image through the second encoder-decoder structure.

[0112] The attention flow outputs the defogging remote sensing image d from the input fog remote sensing image h through the first encoder-decoder structure and the channel-spatial attention structure. h

[0113] The first encoder-decoder structure and the second encoder-decoder structure are the same.

[0114] Further, the first encoder-decoder structure comprises a first encoder and a first decoder, and a skip connection structure is used to simultaneously connect the high-level features and the low-level details of the image.

[0115] ​Further, the first encoder is built using a convolutional block, the convolutional block is used to increase the receptive field, so that the network model can learn more global information, the convolutional block is composed of five groups of structure units, each group of structure units includes a LeakyReLU activation function, a convolutional layer, a normalization layer and a Dropout layer.

[0116] Further, the first decoder is built using a deconvolutional block, the deconvolutional block is used to map the image from a small resolution to a large resolution, the deconvolutional block is composed of five groups of structure units and one group of convolutional layers, each group of structure units includes a ReLU activation function, a transposed convolutional layer, a normalization layer and a Dropout layer.

[0117] Further, the channel-spatial attention structure is composed of a channel attention mechanism, a spatial attention mechanism and a convolutional residual block, the convolutional residual block is composed of three groups of structure units, a skip connection is used after the second group of structure units to avoid the problem of gradient disappearance, and each group of structure units includes a convolutional layer and a ReLU activation function.

[0118] In the method, paired images are used to perform a remote sensing image defogging task in a training process of a network.

[0119] It should be noted that the device description in the above embodiments corresponds to the method description in the embodiments, and the embodiments of the present application will not be described here.

[0120] The execution subject of the processor 1 and the memory 2 described above can be a computer, a single-chip microcomputer, a microcontroller or the like having a computing function, and in actual implementation, the embodiments of the present application do not limit the execution subject, and the execution subject is selected according to the actual application.

[0121] The memory 2 and the processor 1 transmit data signals through the bus 3, and the embodiments of the present application will not be described here.

[0122] Embodiment 5

[0123] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium, the storage medium includes a stored program, and the program controls the device where the storage medium is located to execute the method steps in the above embodiments when the program runs.

[0124] The computer readable storage medium includes but is not limited to a flash memory, a hard disk, a solid state disk and the like.

[0125] It should be noted that the readable storage medium description in the above embodiments corresponds to the method description in the embodiments, and the embodiments of the present application will not be described here.

[0126] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated.

[0127] The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in or transmitted by a computer readable storage medium. The computer readable storage medium can be any available medium or a set of media including a server, a data center, etc. data storage device integrated with one or more available media. The available media can be a magnetic medium or a semiconductor medium, etc.

[0128] The model of each device in the embodiments of the present application is not limited unless otherwise specified, and any device that can complete the above functions can be used.

[0129] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred embodiment, and the above-mentioned serial numbers of the embodiments of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0130] The above is only a preferred embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for dehazing remote sensing images, characterized in that, The method comprises: According to the flow direction of the feature map in the dark channel attention network, the real haze-free remote sensing image and the haze-removed remote sensing image output by the dark channel attention network are taken as a first image pair, and the haze-removed remote sensing image processed by the dark channel prior algorithm and the constraint remote sensing image output by the dark channel prior constraint flow are taken as a second image pair, and the loss function values of the two groups of image pairs are calculated respectively by using the loss function formula; The loss function value of the dark channel attention network is minimized, the values of the parameters in the network are iteratively updated by using the back propagation of the deep neural network, until the termination condition is met, and the training of the dark channel attention network is completed; and the dark channel attention network is trained by using a synthetic haze remote sensing image training set; The dark channel attention network comprises an attention flow and a dark channel prior constraint flow, The attention flow comprises a first encoder-decoder structure and a channel-spatial attention structure; The dark channel prior constraint flow comprises a second encoder-decoder structure; wherein the channel-spatial attention structure is an attention structure with a convolution residual block, a channel attention mechanism and a spatial attention mechanism combined; The first encoder-decoder structure is composed of a convolution block and a deconvolution block, the convolution block is composed of five groups of structure units, each group of structure units comprises a LeakyReLU activation function, a convolution layer, a normalization layer and a Dropout layer; the deconvolution block is composed of five groups of structure units and a convolution layer, each group of structure units comprises a ReLU activation function, a transposed convolution layer, a normalization layer and a Dropout layer. The first encoder-decoder structure is the same as the second encoder-decoder structure.

2. The remote sensing image defogging method of claim 1, wherein, The training of the dark channel attention network by using the synthetic haze remote sensing image training set comprises: The dark channel attention network adopts a parallel architecture of the attention flow and the dark channel prior constraint flow, in the attention flow, the image extracts feature information through the encoder-decoder and enhances the features combined with the channel-spatial attention structure, in the dark channel prior constraint flow, the dark channel prior loss is used to pass the feature information to the attention flow, the attention flow and the dark channel prior constraint flow share the feature information, and the remote sensing image haze-removal task is completed together.

3. The remote sensing image defogging method of claim 1, wherein, The real haze-free remote sensing image and the haze-removed remote sensing image output by the dark channel attention network are: by loss, edge loss computing a dehazed remote sensing image from a paired true haze-free remote sensing image by loss computes a constraint remote sensing image from the dark channel prior constraint flow output of the dark channel attention network from a haze-free remote sensing image processed by a dark channel prior algorithm the difference between, backpropagates to update the parameters of the dark channel attention network, seeks to minimize the loss error: ； wherein, is a 1-norm, and respectively represent a real no-fog remote sensing image and a defogging remote sensing image obtained by processing a preset network. ； wherein represents an edge detection in the horizontal direction, represents an edge detection in the vertical direction; ； wherein, and respectively represent the haze-free remote sensing image processed by the dark channel prior algorithm and the constraint remote sensing image obtained by processing the dark channel prior constraint stream in the dark channel attention network.

4. A remote sensing image defogging device, characterized in that, The device comprises a processor and a memory, the memory stores program instructions, and the processor invokes the program instructions stored in the memory to make the device execute the method steps of any one of claims 1-3.

5. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, the computer program comprises program instructions, and the program instructions are executed by the processor to make the processor execute the method steps of any one of claims 1-3.

Images