Integrated sand and dust image antagonism restoration method and device

Abstract

The invention discloses an integrated sand and dust image antagonism restoration method and device. The method comprises the following steps: constructing a sand and dust image restoration network; wherein the sand and dust image restoration network comprises an integration part and a remodeling part, the output end of the integration part is connected with the input end of the remodeling part, and the integration part is used for generating an inverse evolution quantity corresponding to an image to be restored; constructing a restoration loss function through a preset discriminator model, and training the sand and dust image restoration network according to the restoration loss function; and inputting a to-be-restored image into the trained sand and dust image restoration network to obtain a restored image. According to the invention, the restoration effect of the dust image can be improved.

Description

Technical Field

[0001] The present invention relates to a field, and in particular to an integrated dust image adversarial restoration method and device. Background Art

[0002] In the theoretical research and practical application of image processing, the negative impact of dust weather conditions on image quality has always been a challenge that cannot be ignored. Dust particles in the atmosphere not only significantly reduce the contrast and detail clarity of the image, but also cause a series of image degradation phenomena such as color distortion. These phenomena greatly affect the visual parsing ability and available information of the image, which in turn brings limitations to advanced image processing links such as intelligent recognition, scene reconstruction, and information extraction.

[0003] Although traditional image processing technology can alleviate some image degradation problems to a certain extent, its processing effect is often unsatisfactory when faced with complex and changeable degradation environments such as dust. Most traditional methods are based on a single or limited degradation model, which makes it difficult to fully capture and effectively deal with multi-dimensional degradation features such as image brightness changes, contrast loss, and color shift caused by dust. Therefore, there are significant limitations in restoring the true appearance and detailed information of the image.

[0004] With the rapid development of artificial intelligence, especially deep learning technology, the field of image restoration has ushered in a new breakthrough opportunity. With its powerful feature learning ability and data-driven optimization strategy, deep learning models can show superior performance in complex and changeable image degradation scenarios. However, for the complex degradation scene of sand and dust, the existing deep learning restoration methods still face challenges such as insufficient generalization ability and poor stability. The existing deep learning methods are often designed to focus on specific types of image degradation, lacking in-depth understanding and targeted optimization of sand and dust degradation characteristics, making it difficult to achieve ideal restoration effects when processing complex sand and dust images. Summary of the invention

[0005] In order to overcome the defects of the prior art, the present invention provides an integrated dust image antagonistic restoration method and device, which can improve the restoration effect of the dust image.

[0006] An embodiment of the present invention provides an integrated dust image adversarial restoration method, comprising the following steps:

[0007] Constructing a dust image restoration network; wherein the dust image restoration network includes an integration part and a reshaping part, the output end of the integration part is connected to the input end of the reshaping part, and the integration part is used to generate an inversion variable corresponding to the image to be restored;

[0008] Construct a restoration loss function through a preset discriminator model, and train the dust image restoration network according to the restoration loss function;

[0009] Input the image to be restored into the trained dust image restoration network to obtain a restored image.

[0010] Furthermore, the integration part specifically includes: a first convolutional downsampling unit and a first convolutional upsampling unit;

[0011] The first convolutional downsampling unit includes a number of convolutional encoding layers and a number of pooling downsampling layers, and is used to generate corresponding abstract representation information according to the input image; wherein, the convolutional encoding layer includes a convolutional layer and a Dropout regularization layer, and the input end of the Dropout regularization layer is connected to the output end of the convolutional layer;

[0012] The first convolutional upsampling unit includes a number of the convolutional encoding layers and a number of upsampling layers, and is used to generate corresponding inversion variables according to the abstract representation information.

[0013] Furthermore, the reshaping part includes a second convolutional downsampling unit and a second convolutional upsampling unit;

[0014] The second convolutional downsampling unit includes a number of convolutional encoding layers and a number of pooling downsampling layers, and is used to generate corresponding latent content information according to the inversion variables;

[0015] The second convolutional upsampling unit includes a number of the convolutional encoding layers and a number of upsampling layers, and is used to generate corresponding reshaped images according to the latent content information.

[0016] Preferably, the dust image restoration network further includes: an adaptive fusion mechanism;

[0017] The mathematical expression of the adaptive fusion mechanism is specifically:

[0018] F = mix(F1, F2) = σ(ξ)·F1 + [1 - σ(ξ)]·F2

[0019] wherein, F represents the fused data; F1 and F2 respectively represent the output features of the convolutional encoding layer and the output features of the upsampling layer; σ(·) represents the Sigmoid activation function; ξ is a preset trainable factor.

[0020] Furthermore, the constructing of the restoration loss function through the preset discriminator model specifically includes:

[0021] The restoration loss function includes an L1 loss, an adversarial loss L adv and a restoration loss L res ;

[0022] For N pairs of training samples The mathematical expression of the L1 loss is specifically:

[0023]

[0024] Wherein, is the inversion variable output by the integration part, represents the clear image output by the reshaping part, and are respectively and corresponding reference images;

[0025] The adversarial loss L adv The mathematical expression of is specifically:

[0026]

[0027] Wherein, D is the preset discriminator model, P gen and P gt respectively represent and corresponding image feature distributions;

[0028] The restoration loss L res The mathematical expression of is specifically:

[0029]

[0030] Wherein, both λ1 and λ2 are preset control coefficients;

[0031] Let the restoration loss function be L t Then there is:

[0032] L t = α1·L1 + α2·L adv + α3·L res

[0033] Wherein, α1, α2 and α3 are respectively the preset weight coefficients corresponding to L1, L adv and L res ;

[0034] Preferably, the preset discriminator model specifically includes: a plurality of the convolutional coding layers and a plurality of upsampling cascade units; wherein, the upsampling cascade unit is composed of a convolutional layer and a pixel recombination layer.

[0035] Further, training the dust image restoration network according to the restoration loss function specifically includes:

[0036] Obtain a training dataset, and perform pixel value normalization processing on the training samples in the training dataset to correspondingly obtain preprocessed samples;

[0037] According to the preprocessed samples, use the ADAM optimizer and the restoration loss function to iteratively train the dust image restoration network. When the number of iterations exceeds a preset iteration threshold or the value of the restoration loss function output in the latest iteration is less than a preset loss threshold, end the iteration.

[0038] Further, inputting the image to be restored into the trained dust image restoration network to obtain a restored image specifically includes:

[0039] According to the image to be restored, generate the inversion variable corresponding to the image to be restored through the integration part;

[0040] According to the inversion variable, generate the restored image corresponding to the image to be restored through the reshaping part.

[0041] Another embodiment of the present invention provides an integrated dust image adversarial restoration device, including: a construction module, a training module, and a restoration module;

[0042] The construction module is used to construct a dust image restoration network; wherein, the dust image restoration network includes an integration part and a reshaping part, and the integration part is used to generate the inversion variable corresponding to the image to be restored;

[0043] The training module is used to construct a restoration loss function through a preset discriminator model, and train the dust image restoration network according to the restoration loss function;

[0044] The restoration module is used to input the image to be restored into the trained dust image restoration network to obtain a restored image.

[0045] Further, the restoration module is used to input the image to be restored into the trained dust image restoration network to obtain a restored image, specifically including:

[0046] According to the image to be restored, generate the inversion variable corresponding to the image to be restored through the integration part;

[0047] According to the inversion variable, generate the restored image corresponding to the image to be restored through the reshaping part.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] The present invention deeply integrates the advantages of deep learning and traditional restoration methods, integrates the parameters to be estimated and the potential imaging logic in the dust scattering model into an inversion variable, and effectively mines and learns the complex non-linear mapping relationship between dust images and clear images by constructing a W-Net network structure and combining an adversarial training mechanism, improving the perceptual quality of dust images and thus enhancing the restoration effect of dust images. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic flow chart of an integrated dust image adversarial restoration method provided by an embodiment of the present invention.

[0051] Figure 2 It is a schematic structural diagram of a dust image restoration network provided by an embodiment of the present invention.

[0052] Figure 3 It is a schematic structural diagram of a preset discriminator model provided by an embodiment of the present invention.

[0053] Figure 4 It is a schematic structural diagram of an integrated dust image adversarial restoration device provided by another embodiment of the present invention.

[0054] Figure 5 It is a first visual comparison diagram of various algorithms on the synthetic test set SIRB-E(H) provided by an embodiment of the present invention.

[0055] Figure 6 It is a first visual comparison diagram of various algorithms on the synthetic test set SIRB-E(H) provided by an embodiment of the present invention.

[0056] Figure 7 It is a first enhanced result comparison diagram of various algorithms in real dust images provided by an embodiment of the present invention.

[0057] Figure 8 It is a second enhanced result comparison diagram of various algorithms in real dust images provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The drawings are only for illustrative purposes and should not be construed as limiting the present patent;

[0059] For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0060] The technical solutions of the present invention will be further described below with reference to the drawings and embodiments.

[0061] Refer to Figure 1, which is a schematic flowchart of an integrated sand-dust image adversarial restoration method provided by an embodiment of the present invention, includes the following steps:

[0062] S1: Construct a sand-dust image restoration network; wherein, the sand-dust image restoration network includes an integration part and a reshaping part, the output end of the integration part is connected to the input end of the reshaping part, and the integration part is used to generate an inversion variable corresponding to the image to be restored;

[0063] S2: Construct a restoration loss function through a preset discriminator model, and train the sand-dust image restoration network according to the restoration loss function;

[0064] S3: Input the image to be restored into the trained sand-dust image restoration network to obtain a restored image

[0065] For step S1, specifically, the integration part specifically includes: a first convolutional downsampling unit and a first convolutional upsampling unit;

[0066] The first convolutional downsampling unit includes a plurality of convolutional encoding layers and a plurality of pooling downsampling layers, and is used to generate corresponding abstract representation information according to the input image; wherein, the convolutional encoding layer includes a convolutional layer and a Dropout regularization layer, and the input end of the Dropout regularization layer is connected to the output end of the convolutional layer;

[0067] The first convolutional upsampling unit includes a plurality of the convolutional encoding layers and a plurality of upsampling layers, and is used to generate the corresponding inversion variable according to the abstract representation information.

[0068] Further, the reshaping part includes a second convolutional downsampling unit and a second convolutional upsampling unit;

[0069] The second convolutional downsampling unit includes a plurality of convolutional encoding layers and a plurality of pooling downsampling layers, and is used to generate corresponding potential content information according to the inversion variable;

[0070] The second convolutional upsampling unit includes a plurality of the convolutional encoding layers and a plurality of upsampling layers, and is used to generate a corresponding reshaped image according to the potential content information.

[0071] In a preferred embodiment, refer to Figure 2 , which is a schematic structural diagram of a sand-dust image restoration network provided by an embodiment of the present invention. As Figure 2 can be seen, the sand-dust image restoration network is actually a segmented restoration network with a W-Net structure, which specifically optimizes the estimation of unknown variables and the restoration process of clear images, ensuring that the network can effectively extract key features at different stages and gradually improve the image restoration quality.

[0072] The sand-dust image restoration network includes a second-order non-linear mapping process, which respectively corresponds to the integration part and the reshaping part. In the integration part, for the input sand-dust image, through multi-level convolutional encoding calculation and pooling downsampling operations, the original features of the input image are gradually compressed into a dense abstract representation in the high-dimensional space. At the same time, a Dropout regularization layer is connected to the output end of the convolutional network layer to form the convolutional encoding layer. In the convolutional encoding layer, some neuron connections are randomly closed to reduce the risk of the network's generalization ability decline due to overfitting to the training data. Then, a cascade unit composed of upsampling pixel recombination and convolutional layers is used to decode the abstract representation information into the RGB three-dimensional space and output the estimated inversion variable. In the reshaping part, the inversion variable is used as the port input, and the latent content information is feature-fitted and reshaped into a clear image through an encoder-decoder network.

[0073] For step S1, preferably, the sand-dust image restoration network further includes: an adaptive fusion mechanism;

[0074] The mathematical expression of the adaptive fusion mechanism is specifically:

[0075] F = mix(F1, F2) = σ(ξ)·F1 + [1 - σ(ξ)]·F2

[0076] Where F represents the fused data; F1 and F2 respectively represent the output features of the convolutional encoding layer and the output features of the upsampling layer; σ(·) represents the Sigmoid activation function; ξ is a preset trainable factor.

[0077] In a preferred embodiment, the Mixup adaptive fusion mechanism is adopted to replace the traditional skip connection, making the information fusion process smoother and more gentle through linear interpolation of the feature maps in the network, and at the same time being able to strengthen the non-linear fitting ability of the network. ξ is a preset trainable factor, which can be used to dynamically adjust the weight ratio of the corresponding fused features.

[0078] For step S2, specifically, constructing the restoration loss function through a preset discriminator model specifically includes:

[0079] The restoration loss function includes the L1 loss, the adversarial loss L adv and the restoration loss L res ;

[0080] For N pairs of training samples The mathematical expression of the L1 loss is specifically:

[0081]

[0082] Where The inversion variable output for the integration part represents the clear image output by the reshaping part and are respectively and the corresponding reference images

[0083] The adversarial loss L adv has the specific mathematical expression as follows

[0084]

[0085] where D is the preset discriminator model, P gen and P gt respectively represent and the corresponding image feature distributions

[0086] The restoration loss L res has the specific mathematical expression as follows

[0087]

[0088] where both λ1 and λ2 are preset control coefficients

[0089] Let the restoration loss function be L t , then there is

[0090] L t = α1·L1 + α2·L adv + α3·L res

[0091] where α1, α2, and α3 are respectively the preset weight coefficients corresponding to L1, L adv and L res

[0092] In a preferred embodiment, during the training phase, the proposed network model is trained for parameters by constructing a joint loss constraint term (i.e., the restoration loss function), including the L1 loss, the adversarial loss L adv and the restoration loss L res . Among them, the adversarial loss L adv is used to guide the backbone network to extract potential information in the high-dimensional feature space, and continuously reduce the distribution difference between the network output result and the real sample in an adversarial manner; the restoration loss L res starting from the perspective of imaging, based on the dust inversion model, a feature constraint on the first-order network is established, and by standardizing the parameter derivative direction, the convergence speed of the network during the training phase is improved

[0093] ​For step S2, preferably, the preset discriminator model specifically includes: a plurality of the convolutional coding layers and a plurality of upsampling cascade units; wherein, the upsampling cascade unit is composed of a convolutional layer and a pixel recombination layer.

[0094] In a preferred embodiment, during the training process, an adversarial training mechanism is constructed. Through the adversarial learning between networks, it can force the generator to continuously improve to generate more realistic samples. And the preset discriminator model learns to distinguish real samples from generated samples, promoting the samples generated by the generator to continuously improve in terms of realism and diversity.

[0095] Due to the modular characteristics of adversarial training, flexible loss function integration, and adjustable training strategies, this training method has good seamless integration and compatibility with existing deep learning architectures. Considering the many excellent performances of adversarial training in image generation tasks, introducing the preset discriminator model in the training stage to co-optimize the built-in parameters in the backbone network in an adversarial training manner can ensure the accuracy of the output data of the network at different stages.

[0096] Refer to Figure 3 , which is a schematic structural diagram of a preset discriminator model provided by an embodiment of the present invention. As can be seen from Figure 3 , for the given input data, first, through a cascade unit of multiple convolutional layers and Dropout regularization layers, it is mapped from a low-dimensional visual space to a high-dimensional feature space, strengthening the model's learning and understanding ability of abstract features; subsequently, through a cascade unit of a convolutional layer and a pixel recombination layer, the abstract representations of different inputs are gradually mapped into a single-channel score map, which gives a point-by-point evaluation of the quality of the input data and provides pixel-level reconstruction guidance for the parameter optimization of the backbone network. Through continuous training iterations, the output feature distribution of the backbone network gradually approaches the real sample distribution.

[0097] For step S2, further, the training of the sand-dust image restoration network according to the restoration loss function specifically includes:

[0098] Obtain a training data set, and perform pixel value normalization processing on the training samples in the training data set to correspondingly obtain preprocessed samples;

[0099] According to the preprocessed samples, use the ADAM optimizer and the restoration loss function to perform iterative training on the sand-dust image restoration network. When the number of iterations exceeds a preset iteration threshold or the value of the restoration loss function output in the latest iteration is less than a preset loss threshold, end the iteration.

[0100] In a preferred embodiment, the sand and dust reference SIRB-T(H) is used as the training data set. All input images are cropped to a size of 256×256, and the pixel values are normalized. The PyTorch deep learning development framework is used, and the hardware environment is an NVIDIA Geforce RTX2070 GPU with 8GB video memory, an Intel(R) Core(TM) i5-9400 CPU with a main frequency of 2.90GHz, and 16G RAM.

[0101] During training, this preferred embodiment uses the ADAM optimizer to randomly initialize the network parameters through a Gaussian distribution. The learning rates of the sand and dust image restoration network and the preset discriminator model are set to 0.002 and 0.001 respectively, and the preset iteration threshold is set to 100.

[0102] For step S3, specifically, the step of inputting the image to be restored into the trained sand and dust image restoration network to obtain the restored image specifically includes:

[0103] According to the image to be restored, the inversion variable corresponding to the image to be restored is generated through the integration part;

[0104] According to the inversion variable, the restored image corresponding to the image to be restored is generated through the reshaping part.

[0105] In a preferred embodiment, the derivation process of the inversion variable is as follows:

[0106] Perform backward derivation on the sand and dust scattering model, and integrate the model parameters and the potential imaging logic into a unified unknown variable to more comprehensively describe the key degradation process of the sand and dust image. The mathematical expression form of the sand and dust imaging model is as follows:

[0107] I s (x) = [J s (x) - A s ′]·t s (x) + A s

[0108]

[0109] Among them, I s (x) is the image to be restored; J s (x) represents the clear image without sand and dust; A s represents the global color polarization value, and A s ′ is the corresponding complementary color; t s (x) is the transmittance; β represents the scattering coefficient; d s (x) is the scene depth.

[0110] Performing reverse derivation on the sand and dust scattering model can construct the restoration process of a clear image:

[0111]

[0112] Integrate the model parameters and the potential imaging logic into the inversion variable To minimize the cumulative error generated during the inference stage. The mathematical expression of the sand and dust inversion model is as follows:

[0113]

[0114] where λ1 and λ2 are the preset control coefficients used to fix the element values in the interval [0, 1].

[0115] It can be seen from the above formula that the inversion variable has a non-linear logical relationship with the image I s (x) to be restored. Therefore, the neural network can be trained to fit this complex relationship to construct a point-to-point mapping between the two, and combined with the inversion model to restore a clear image.

[0116] Referring to Figure 4 , it is a schematic structural diagram of an integrated sand and dust image adversarial restoration device provided by another embodiment of the present invention, including: a construction module 101, a training module 102, and a restoration module 103;

[0117] The construction module 101 is used to construct a sand and dust image restoration network; wherein, the sand and dust image restoration network includes an integration part and a reshaping part, and the integration part is used to generate an inversion variable corresponding to the image to be restored;

[0118] The training module 102 is used to construct a restoration loss function through a preset discriminator model, and train the sand and dust image restoration network according to the restoration loss function;

[0119] The restoration module 103 is used to input the image to be restored into the trained sand and dust image restoration network to obtain a restored image.

[0120] Further, the restoration module 103 is used to input the image to be restored into the trained sand and dust image restoration network to obtain a restored image, specifically including:

[0121] Generate the inversion variable corresponding to the image to be restored through the integration part according to the image to be restored;

[0122] Generate the restored image corresponding to the image to be restored through the reshaping part according to the inversion variable.

[0123] Finally, to verify the effectiveness of the algorithm, the embodiments of the present invention use the current advanced sand and dust image processing method to conduct subjective and objective performance comparisons with the integrated sand and dust image adversarial restoration method proposed by the present invention. The processing methods for comparison include: TTFIO, ROP, ROP + , HRDCP, CIDC, CBCS, NGT, and FS. At the same time, algorithm performance evaluation experiments are carried out on the synthetic sand and dust test set SIRB-E(H) and the real sand and dust test set RSTS.

[0124] Refer to Figure 5 , which is the first visual comparison diagram of each algorithm on the synthetic test set SIRB-E(H) provided by an embodiment of the present invention. At the same time, refer to Figure 6 , which is the first visual comparison diagram of each algorithm on the synthetic test set SIRB-E(H) provided by an embodiment of the present invention.

[0125] From Figure 5 (b), it can be seen that the TTFIO algorithm cannot effectively balance the sand and dust image with a large color offset, resulting in obvious color distortion problems in the enhancement result; the ROP algorithm can improve the visual quality of the sand and dust image to a certain extent, but there are obvious color distortion problems in the local area with high sand and dust concentration, as shown in Figure 5 (c); the enhancement result of the ROP + algorithm has a certain improvement compared with ROP, but from Figure 6 (d), it can be observed that there is an obvious fault problem in the brightness distribution of the algorithm in the sky area; although the HRDCP and NGT algorithms can effectively balance the color deviation of the sand and dust image, due to the algorithm being too sensitive to interference noise in the structural design, there are many noise points in the enhancement result, resulting in the loss of detailed information in the image; from Figure 5 (f) and Figure 6 (f), it can be seen that the output image of the CIDC algorithm is too dark overall, and there is a certain color offset problem in the result; the enhancement result of the CBCS algorithm has obvious block effect stratification in the area with large sand and dust concentration, resulting in damage to the local texture structure, as shown in Figure 5 (g); the FS algorithm can improve the visual quality of the image to a certain extent, but from Figure 6 (i), it can be observed that due to the over-enhancement of the algorithm, the result has a too high color saturation and obvious halo artifact problems. Compared with the foregoing methods, the method of the present invention can effectively balance the color deviation problem of the sand and dust image, eliminate the foggy haze caused by the sand and dust, and has better performance in dealing with complex sand and dust scenes.

[0126] The Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and color quantization index CIEDE2000 are used as objective evaluation indicators to quantitatively evaluate the performance of the algorithm. The objective evaluation results are shown in Table 1. It can be seen that the TTFIO and CIDC algorithms show large errors in color balance, resulting in a high CIEDE2000 index; the HRDCP algorithm is easily affected by noise interference during the enhancement process, causing large pixel errors in its output results; due to the relatively serious blocking effect problem in the enhancement results of the CBCS algorithm, the local structural features of the image are damaged, thereby reducing the SSIM index. Starting from the perspective of sand-dust image imaging, the present invention can perform adaptive enhancement processing on different types of degraded images. Compared with the sub-optimal algorithms, the PSNR and SSIM indexes are respectively increased by 1.97 dB and 3.86%, and the CIEDE2000 is reduced by 3.69 △E 00 . The objective evaluation is basically consistent with the subjective perception, verifying the effectiveness of the present invention in the sand-dust image processing task.

[0127] Table 1 Comparison of objective evaluation indicators

[0128]

[0129] Although the network constructed by the present invention is trained on a synthetic dataset, it still shows good generalization ability in real sand-dust scenarios. Refer to Figure 7 , which is a comparison diagram of the first enhancement results of each algorithm in real sand-dust images provided by an embodiment of the present invention. Refer to Figure 8 , which is a comparison diagram of the second enhancement results of each algorithm in real sand-dust images provided by an embodiment of the present invention.

[0130] From Figure 7 and Figure 8 it can be seen that the enhancement results of the TTFIO algorithm still show obvious color offset problems; the ROP and ROP + algorithms can improve the visual quality of sand-dust images to a certain extent, but due to the lack of collaborative processing of color channels, the output images of both algorithms are accompanied by slight color deviation, and the ROP + algorithm result has an overexposure problem in the sky area; from Figure 7 (e) and Figure 7 (h) it can be observed that the results of the HRDCP and NGT algorithms are generally dim, and at the same time, the enhancement results of both are accompanied by obvious noise amplification problems; the output images of the CIDC and FS algorithms are too dark, from Figure 8(f) It can be seen that there is still a certain color offset problem in the enhanced result of this algorithm; the CBCS algorithm can improve the visual clarity of the image, but due to over-enhancement, the overall algorithm shows a blue-violet tone and there is also a checkerboard effect problem in the sky area. Compared with the aforementioned methods, the method proposed by the present invention can effectively improve the visual clarity of sand-dust images and performs better in aspects such as balancing image color deviation, suppressing noise, and reducing block effects.

[0131] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or variations can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. An integrated adversarial restoration method for sand-dust images, characterized in that, It includes the following steps: Construct a sand-dust image restoration network; wherein, the sand-dust image restoration network includes an integration part and a reshaping part, the output end of the integration part is connected to the input end of the reshaping part, and the integration part is used to generate an inversion variable corresponding to the image to be restored; Construct a restoration loss function through a preset discriminator model, and train the sand-dust image restoration network according to the restoration loss function; Input the image to be restored into the trained sand-dust image restoration network to obtain a restored image.

2. The integrated sand-dust image adversarial restoration method according to claim 1, wherein, The integration part specifically includes: a first convolutional downsampling unit and a first convolutional upsampling unit; The first convolutional downsampling unit includes a plurality of convolutional encoding layers and a plurality of pooling downsampling layers, and is used to generate corresponding abstract representation information according to the input image; wherein, the convolutional encoding layer includes a convolutional layer and a Dropout regularization layer, and the input end of the Dropout regularization layer is connected to the output end of the convolutional layer; The first convolutional upsampling unit includes a plurality of the convolutional encoding layers and a plurality of upsampling layers, and is used to generate the corresponding inversion variable according to the abstract representation information.

3. The integrated sand-dust image adversarial restoration method according to claim 2, characterized in that, The reshaping part includes a second convolutional downsampling unit and a second convolutional upsampling unit; The second convolutional downsampling unit includes a plurality of convolutional encoding layers and a plurality of pooling downsampling layers, and is used to generate corresponding potential content information according to the inversion variable; The second convolutional upsampling unit includes a plurality of the convolutional encoding layers and a plurality of upsampling layers, and is used to generate a corresponding reshaped image according to the potential content information.

4. The integrated sand-dust image adversarial restoration method according to claim 2, wherein The sand-dust image restoration network further includes: an adaptive fusion mechanism; The mathematical expression of the adaptive fusion mechanism is specifically: F = mix(F1, F2) = σ(ξ)·F1 + [1 - σ(ξ)]·F2 wherein, F represents the fused data; F1 and F2 respectively represent the output features of the convolutional encoding layer and the output features of the upsampling layer; σ(·) represents the Sigmoid activation function; ξ is a preset trainable factor.

5. The integrated sand-dust image adversarial restoration method according to claim 1, characterized in that, The constructing the restoration loss function through the preset discriminator model specifically includes: The restoration loss function includes the L1 loss, the adversarial loss L adv and the restoration loss L res ; For N pairs of training samples The specific mathematical expression of the L1 loss is as follows: Wherein, is the inversion variable output by the integration part, represents the clear image output by the reshaping part, and are respectively and corresponding reference images; The adversarial loss L adv has the following specific mathematical expression: Among them, D is the preset discriminator model, and P gen and P gt respectively represent and the corresponding image feature distributions; The restoration loss L res has the following specific mathematical expression: wherein, both λ1 and λ2 are preset control coefficients; Let the restoration loss function be L t , then we have: L t = α1·L1 + α2·L adv + α3·L res Among them, α1, α2, and α3 are the preset weight coefficients corresponding to L1, L adv and L res respectively.

6. The integrated sand-dust image adversarial restoration method according to claim 2, wherein The preset discriminator model specifically includes: a plurality of the convolutional encoding layers and a plurality of upsampling cascade units; wherein, the upsampling cascade unit is composed of a convolutional layer and a pixel recombination layer.

7. The integrated sand-dust image adversarial restoration method according to claim 1, wherein The training the sand-dust image restoration network according to the restoration loss function specifically includes: Obtain a training data set, and perform pixel value normalization processing on the training samples in the training data set to correspondingly obtain preprocessed samples; According to the preprocessed samples, perform iterative training on the sand-dust image restoration network through an ADAM optimizer and the restoration loss function. When the number of iterations exceeds a preset iteration threshold or the value of the restoration loss function output in the latest iteration is less than a preset loss threshold, end the iteration.

8. The integrated sand-dust image adversarial restoration method according to claim 1, characterized in that The inputting the image to be restored into the trained sand-dust image restoration network to obtain a restored image specifically includes: According to the image to be restored, generate the inversion variable corresponding to the image to be restored through the integration part; According to the inversion variable, a restored image corresponding to the image to be restored is generated through the reshaping part.

9. An integrated sand-dust image adversarial restoration device, characterized in that, It includes: a construction module, a training module, and a restoration module; The construction module is used to construct a dust image restoration network; wherein, the dust image restoration network includes an integration part and a reshaping part, and the integration part is used to generate an inversion variable corresponding to the image to be restored; The training module is used to construct a restoration loss function through a preset discriminator model, and train the dust image restoration network according to the restoration loss function; The restoration module is used to input the image to be restored into the trained dust image restoration network to obtain a restored image.

10. The integrated sand-dust image adversarial restoration device according to claim 9, wherein The restoration module is used to input the image to be restored into the trained dust image restoration network to obtain a restored image, specifically including: According to the image to be restored, the integration part generates the inversion variable corresponding to the image to be restored; According to the inversion variable, the reshaping part generates a restored image corresponding to the image to be restored.

Images