Multi-scene-oriented image restoration method and device

Abstract

The invention relates to the field of image restoration, in particular to a multi-scene-oriented image restoration method and device. According to the method, a neural network model used for image restoration comprises an encoder, a decoder A and a decoder B; inputting the image sample into an encoder, and extracting multi-scale depth features of the image sample by using the encoder; respectively inputting the multi-scale depth features into a decoder A and a decoder B, recovering a noisy image by using the decoder A, and recovering a noiseless image by using the decoder B; obtaining a loss function based on the noisy image and the noiseless image, and training a neural network model based on the loss function; and using the trained encoder to extract multi-scale depth features of the to-be-restored image, and inputting the multi-scale depth features into the decoder B to obtain a restored clean image. According to the method, image quality enhancement can be carried out aiming at various quality reduction factors, and a high-quality image can be better restored.

Description

Technical Field

[0001] The present invention relates to the field of image restoration, and in particular to a multi-scene oriented image restoration method and device. Background Art

[0002] With the rapid development and popularization of artificial intelligence and Internet of Things technologies, visual tasks in outdoor scenes have received extensive attention from industry and academia. Among them, outdoor scenes, especially special weather such as rainy and foggy days and external factors such as low light, will have degradation factors such as raindrop interference, snowflake interference or low light interference, which have a huge impact on the imaging quality of the image, resulting in a large number of low-quality images being captured. Due to factors such as weather noise and low light, when using low-quality images for visual tasks such as image classification or target recognition, the results often have large deviations.

[0003] In view of this, how to overcome the defects of the prior art and solve the problem of low image quality caused by scene factors is a problem to be solved in the technical field. Summary of the invention

[0004] In view of the above defects or improvement needs of the prior art, the present invention solves the problem of low image quality caused by scene factors.

[0005] The embodiment of the present invention adopts the following technical solution:

[0006] In the first aspect, the present invention provides an image restoration method for multiple scenarios, specifically: the neural network model used for image restoration includes an encoder, a decoder A and a decoder B, including: inputting image samples into the encoder, and using the encoder to extract multi-scale depth features of the image samples; inputting the multi-scale depth features into decoder A and decoder B respectively, using decoder A to restore the noisy image, and using decoder B to restore the noise-free image; obtaining a loss function based on the noisy image and the noise-free image, and training the neural network model based on the loss function; using the trained encoder to extract the multi-scale depth features of the image to be restored, and inputting the multi-scale depth features into decoder B to obtain a repaired clean image.

[0007] Preferably, the encoder includes Stem convolution, large kernel convolution, Transformer block and downsampling convolution, and the use of the encoder to extract multi-scale deep features of image samples specifically includes: obtaining a tensor of image samples, and inputting the tensors into Stem convolution, large kernel convolution, Transformer block and downsampling convolution respectively; using Stem convolution to extract shallow features of image samples, using large kernel convolution to extract middle features of image samples, and using two groups of serially connected Transformer blocks and downsampling convolution to respectively extract deep features of image samples of different dimensions; representing shallow features, middle features and deep features as feature tensors respectively, and using the set of all feature tensors as multi-scale deep features of image samples.

[0008] Preferably, each of the Transformer blocks is composed of two basic Transformer modules connected in series, specifically including: each basic Transformer module is connected in the form of residuals by layer regularization and multi-head self-attention, and then connected in series with a 3×3 convolution kernel activation function residual.

[0009] Preferably, the downsampling convolution includes at least one convolution kernel, specifically including: the number of convolution kernels in the downsampling convolution is greater than the step size, and the output channel dimension of the downsampling convolution is equal to twice the input channel dimension.

[0010] Preferably, the inputting of the multi-scale deep features into decoder A and decoder B respectively specifically includes: obtaining the feature tensor of each scale and dimension in the multi-scale deep features, and inputting the obtained feature tensors into the corresponding modules of decoder A and decoder B respectively in a jump connection manner.

[0011] Preferably, the decoder A includes 4 transposed convolution modules with consistent structures, and the use of decoder A to restore the noisy image specifically includes: obtaining the output result of the corresponding layer of the encoder and the output result of the upper decoder, and inputting the obtained output result into the corresponding convolution of each transposed convolution module, so as to obtain the restored noisy image through the corresponding transposed convolution module.

[0012] Preferably, each transposed convolution module of the decoder A includes in sequence: batch normalization, a first convolution, an activation function, a first transposed convolution, a second convolution and a second transposed convolution, and the obtained output result is input into the corresponding convolution of each transposed convolution module, specifically including: inputting the output result of the corresponding layer of the encoder into the second convolution, and inputting the output result of the previous layer of the decoder into the batch normalization and the second transposed convolution.

[0013] Preferably, the loss function is constructed as follows:

[0014]

[0015] Among them, DA i is the image output by decoder A, DB i The image output by decoder B, Dmg i is the i-th image sample, Gt i is the clean image corresponding to the i-th image sample, N is the total number of image samples, L 1 (·) indicates that the mean absolute error loss function is used for calculation.

[0016] In a second aspect, the present invention provides an image restoration device for multiple scenes, specifically comprising: at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the image restoration method for multiple scenes in the first aspect.

[0017] In a third aspect, the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method described in the first aspect.

[0018] In a fourth aspect, a chip is provided, comprising: a processor and an interface, for calling and running a computer program stored in the memory from a memory, and executing the method of the first aspect.

[0019] According to a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed on a computer or a processor, the computer or the processor executes the method according to the first aspect.

[0020] Compared with the prior art, the beneficial effects of the present invention are: using an asymmetric codec structure to achieve deep feature learning of low-quality images and high-quality image restoration based on multiple constraints, being able to enhance image quality for a variety of degradation factors and better restore high-quality images. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flowchart of a multi-scenario image restoration method provided by an embodiment of the present invention;

[0023] Figure 2Another flow chart of a multi-scenario image restoration method provided by an embodiment of the present invention

[0024] Figure 3 A schematic diagram of the structure of an encoder in the method provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of the Transformer block structure of the decoder in the method provided in an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of the connection relationship between the encoder and the decoder A in the method provided by an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of the connection relationship between the encoder and the decoder B in the method provided in an embodiment of the present invention;

[0028] Figure 7 A schematic diagram of the structure of a transposed convolution module of encoder A in the method provided in an embodiment of the present invention;

[0029] Figure 8 A schematic diagram of the structure of a multi-scenario image restoration device provided by an embodiment of the present invention;

[0030] The reference numerals are as follows:

[0031] 11: processor; 12: memory. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0033] The present invention is an architecture of a specific functional system, so the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

[0034] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0035] Embodiment 1:

[0036] In order to solve the problem of low image quality caused by scene factors, this embodiment proposes an end-to-end solution, which can efficiently repair images for multiple weather scenes and restore the above low-quality images to high-quality images.

[0037] In the method provided in this embodiment, a neural network model is used for image restoration. As shown in the figure, the neural network model used for image restoration includes an encoder, a decoder A and a decoder B.

[0038] like Figure 2 As shown, the specific steps of the multi-scene image restoration method provided by the embodiment of the present invention are as follows:

[0039] Step 101: Input an image sample into an encoder, and use the encoder to extract multi-scale depth features of the image sample.

[0040] In order to use the neural network model for image restoration, it is first necessary to train the neural network model using image samples, where each image sample used as training data is an image to be restored. At the same time, in order to optimize the neural network model, it is also necessary to obtain the real clean image corresponding to the image sample.

[0041] During training, the image samples are first input into the encoder, which includes convolution and Transformer modules to extract multi-scale deep features of the image samples.

[0042] Step 102: Input the multi-scale depth features into decoder A and decoder B respectively, use decoder A to restore the noisy image, and use decoder B to restore the noise-free image.

[0043] After the encoder obtains the multi-scale deep features, it is input into the decoder, and the two decoders perform the image restoration task in parallel to complete the image restoration. In actual implementation, the two decoders have the same architecture, both consisting of Transformer blocks and deconvolution. The two decoders run in parallel, decoder A is used to restore high-quality images with noise, and decoder B is used to restore clean images without noise.

[0044] Step 103: Obtain a loss function based on the noisy image and the noise-free image, and train the neural network model based on the loss function.

[0045] When training a neural network model, it is necessary to optimize it through a loss function. In this embodiment, two decoders are used to obtain two output images. Therefore, it is also necessary to obtain the overall loss function of the neural network model based on the two output images and the clean images corresponding to the image samples, and then optimize the neural network model according to the overall loss function.

[0046] Step 104: Use the trained encoder to extract multi-scale depth features of the image to be repaired, and input the multi-scale depth features into the decoder B to obtain a repaired clean image.

[0047] After the training of the neural network model is completed, the trained model can be used to actually complete the image restoration through model reasoning. In the model reasoning stage, only the encoder + decoder B needs to be run. Compared with the existing technology, the computing power during model reasoning is not increased, and the actual image restoration work can be completed with lower computing power consumption. Figure 1 In the figure, the dotted part is the neural network model framework used in the model inference stage.

[0048] After steps 101 to 104 provided in this embodiment, the low-quality image restoration caused by scene factors can be completed and a corresponding clean image can be obtained.

[0049] In actual implementation, the neural network model for repair can be established by referring to the following method, or the structure and parameters of the following model can be adjusted according to actual needs. It can be understood that the following specific implementation methods are only used to illustrate the specific implementation process of the method in this embodiment in certain specific scenarios, and are not intended to limit the scope of protection.

[0050] The network structure of the encoder is as follows Figure 3 As shown, the encoder includes Stem convolution, large kernel convolution, Transformer block and downsampling convolution.

[0051] The image to be restored input to the encoder, its image matrix data can be represented in the form of a tensor as Dmg∈R 3×H×W , get the tensor of image samples, and input the tensor into Stem convolution, large kernel convolution, Transformer block and downsampling convolution respectively.

[0052] In the model training phase, Stem convolution is used to extract shallow features of image samples, large kernel convolution is used to extract middle features of image samples, and two sets of serially connected Transformer blocks and downsampling convolution are used to extract deep features of different dimensions of image samples. In the model inference phase, the above modules in the encoder also obtain features of the corresponding scales in the image to be repaired in the above manner.

[0053] (1) The first part of the encoder is the Stem convolution, which is composed of two convolutions in series. The convolution kernel of each convolution is 3×3, the step size is 2, and the output channel dimension is 96. It is used to extract the shallow features of the input image. The feature dimension of the shallow features is

[0054] (2) The second part of the encoder is the large kernel convolution. The convolution kernel of the large kernel convolution is 21, the step size is 2, and the output feature dimension is 256. In actual implementation, the large kernel convolution can be used for filtering to extract the mid-level features of the image. The feature dimension of the mid-level features is

[0055] (3) The third part of the encoder consists of a series of Transformer blocks and downsampling convolution blocks. This part is referred to as stage 1 below. The feature dimension of the output of this part is

[0056] (4) The fourth part of the encoder is the same as the third part, which is composed of a Transformer block and a downsampling convolution block in series. This part is referred to as stage 2 below. The feature dimension of this part is

[0057] After each part of the encoder completes the above process, it can obtain features of image samples at different depths and scales, represent shallow features, mid-level features, and deep features as feature tensors respectively, and take the collection of all feature tensors as multi-scale depth features of image samples.

[0058] In a specific implementation, the third and fourth parts of the encoder can complete feature extraction in the following manner.

[0059] The Transformer block used in Stage 1 is composed of two basic Transformer modules connected in series. Its structure is as follows Figure 4 The two dotted boxes in the figure are shown in Figure 1. The main parts of the two basic Transformer modules are composed of multi-head attention mechanisms and convolutions. Each basic Transformer module is connected by layer regularization and multi-head self-attention in the form of residuals, and then connected in series with a convolution kernel and activation function residuals. The convolution kernel is a 3×3 convolution kernel with a step size of 1.

[0060] Among them, the calculation formula of the multi-head attention mechanism is defined as follows:

[0061]

[0062] Among them, x is the image input to Stage 1, which is input in the form of a feature tensor; Q, K, V are three 1×1 convolutions, which output x as 3 feature tensors of the same dimension as x; the spatial dimensions of the input and output tensors of the multi-head attention module do not change.

[0063] The number of convolution kernels in the downsampling convolution is greater than the stride, for example, the convolution kernel is 7 and the stride is 2. The output channel dimension of the downsampling convolution is equal to twice the input channel dimension, so the convolution with twice the input channel dimension is output as the downsampling convolution. Since the convolution kernel of the downsampling convolution is larger than the stride, the overlapping semantic features can be embedded into adjacent output pixels during the downsampling process, which helps to strengthen the information interaction between different pixels.

[0064] Stage 2 is similar to Stage 1 in structure, both of which are composed of basic Transformer modules. In specific implementation, it can be composed of 4 basic Transformer blocks connected in series.

[0065] In the model training phase, two decoders A and B with similar structures are used. The two decoders are responsible for restoring the clean image after repair and restoring the original image respectively. Specifically, the output result of the corresponding layer of the encoder and the output result of the upper decoder are obtained, and the obtained output results are input into the corresponding convolution of each transposed convolution module, so as to obtain the restored noisy image through the corresponding transposed convolution module.

[0066] In the specific implementation, the feature tensor of each scale and dimension in the multi-scale deep features is obtained, and the obtained feature tensors are input into the corresponding modules of decoder A and decoder B respectively in the form of skip connection. Skip connection is a method of directly connecting the input information from a certain level or branch of the model to the subsequent level or branch across some levels. By skipping the processing of the intermediate levels, the bottom and high-level features are directly connected, thereby promoting the flow and fusion of information. In the actual implementation, the connection between the encoder and the two decoders forms a U-shaped network structure, in which the encoder is the bottom of the U-shaped network and the two decoders are the two arms of the U-shaped network. Figure 5 As shown in , it is the connection structure of the encoder and decoder A. The first part of the encoder inputs the output result to the second part and the 4th layer of the decoder A, the second part inputs the output result to Stage 1 and the 3rd layer of the decoder A, Stage 1 inputs the output result to Stage 2 and the 3rd layer of the decoder A, and Stage 2 inputs the output result to the 1st layer of the decoder A; Figure 6 As shown, this is the connection structure of the encoder and decoder B, and the connection relationship is similar to that of the encoder and decoder A.

[0067] In this embodiment, decoder A is taken as an example for description, and decoder B can be constructed with reference to the structure of decoder A. Decoder A includes four transposed convolution modules with the same structure.

[0068] like Figure 7As shown, each layer of decoder A is a transposed convolution module, and each transposed convolution module includes: batch normalization, first convolution, activation function, first transposed convolution, second convolution and second transposed convolution in sequence. Among them, the first convolution is a 3X3 convolution with a step size of 1; the first transposed convolution is a 4X4 transposed convolution with a step size of 2; the second convolution is a 3X3 convolution with a step size of 2; the second transposed convolution is a 4X4 transposed convolution with a step size of 2. When decoding, in the first transposed convolution module, the output result of the decoder Stage 2 layer is input into the second transposed convolution. The input of each transposed convolution module in the 2nd to 4th transposed convolution modules consists of two parts, one is the output result of the upper-level transposed convolution module, and the other is the output result of the corresponding transposed convolution module in the encoder. Specifically: according to Figure 5 As shown in the corresponding relationship, the output result of the corresponding layer of the encoder is input into the second convolution, and the output result of the previous layer decoder is input into batch normalization and the second transposed convolution. Due to the excellent generalization ability and outstanding image feature expression ability of the encoder, even if a general deconvolution module is used as a decoder, outstanding image enhancement effects can be achieved.

[0069] Compared with the traditional single encoder-single decoder structure, the asymmetric codec design provided in this embodiment uses two parallel decoders in the model training stage to perform two image restoration tasks. Since both tasks need to restore the overall semantic features of the image, there are differences in the details of the restored image: decoder A uses the encoder output as input and is responsible for restoring noisy or degraded images; decoder B also uses the decoder output as input and is responsible for restoring clean images without noise. The coupling of the two decoders in coarse-grained features and the complementarity of fine-grained features can generate more accurate regularization constraints for the decoder during model training, improve the encoder performance, and thus improve the overall performance of the model.

[0070] In the model training stage, the training data is composed of a large number of image pairs (Dmg, Gt). 3×H×W Represents the image sample to be repaired, Gt∈R 3×H×W Represents the real clean image after restoration. For any input Dmg i , the image output by decoder A is denoted as DA i , the image output by decoder B is denoted as DB i , the data dimensions of both are R 3×H×W .

[0071] In the specific implementation, a suitable loss function can be selected for the output of each decoder as needed, and then the loss functions of the two decoders can be combined to obtain the loss function of the entire model. In a specific scenario, the mean absolute error loss function (L1 loss) is used as the loss function of the two decoder branches. The mean absolute error loss function has a stable gradient for any input value, will not cause the gradient explosion problem, and has a more robust solution.

[0072] The overall loss function of the model is constructed as follows:

[0073]

[0074] Among them, DA i is the image output by decoder A, DB i The image output by decoder B, Dmg i is the i-th image sample, Gt i is the clean image corresponding to the i-th image sample, N is the total number of image samples, L 1 (·) indicates that the mean absolute error loss function is used for calculation.

[0075] The multi-scenario image restoration method provided by this embodiment has the following beneficial effects compared with the prior art.

[0076] (1) Strong generalization ability, suitable for image restoration in various scenarios. The asymmetric encoding and decoding structure proposed in this embodiment has two decoders, one for restoring high-quality images and the other for restoring original images. The two have the characteristics of deep feature aliasing and shallow feature complementarity at the feature reconstruction level, which helps to improve the generalization ability and detail restoration ability of the model.

[0077] (2) The encoding feature sharing and decoder coupling structure contribute to high-quality image restoration. The two types of decoders are set to regularize the encoder features from two dimensions, which can improve the encoder's feature learning ability and enable the model to better restore image details.

[0078] (3) The asymmetric codec structure can significantly improve the image restoration performance of the model without increasing the computing power consumption. Compared with the traditional codec structure, the asymmetric codec structure proposed in this embodiment only increases the computational complexity of model training, and does not increase the computing power consumption of model reasoning, and has lower computing power consumption. At the same time, the asymmetric structure can be easily extended to most codec-based image restoration models, and has high innovation and application value.

[0079] Embodiment 2:

[0080] Based on the multi-scene image restoration method provided in the above embodiment 1, the present invention further provides a multi-scene image restoration device that can be used to implement the above method, such as Figure 8 , which is a schematic diagram of the device architecture of an embodiment of the present invention. The multi-scenario image restoration device of this embodiment includes one or more processors 11 and a memory 12. Figure 8 A processor 11 is taken as an example.

[0081] The processor 11 and the memory 12 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0082] The memory 12 is a non-volatile computer-readable storage medium for a multi-scene image restoration method, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as the multi-scene image restoration method in Example 1. The processor 11 executes various functional applications and data processing of the multi-scene image restoration device by running the non-volatile software programs, instructions and modules stored in the memory 12, that is, the multi-scene image restoration method in Example 1 is implemented.

[0083] The memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 12 may optionally include a memory remotely arranged relative to the processor 11, and these remote memories may be connected to the processor 11 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0084] The program instructions / modules are stored in the memory 12. When executed by one or more processors 11, the multi-scene image restoration method in the above embodiment 1 is executed. For example, the above described Figure 2 The steps shown.

[0085] An embodiment of the present invention further provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, such as Figure 8 A processor 11 in the embodiment may enable the one or more processors to execute the multi-scenario image restoration method in embodiment 1, for example, to execute the above-described Figure 2 The steps shown can also be implemented Figure 1 , Figures 3 to 7 The various modules and units described.

[0086] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.

[0087] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-scene image restoration method, characterized in that: The neural network model used in image restoration includes encoder, decoder A and decoder B, including: Input the image sample into the encoder, and use the encoder to extract multi-scale deep features of the image sample; The multi-scale deep features are input into decoder A and decoder B respectively, decoder A is used to restore the noisy image, and decoder B is used to restore the noise-free image; A loss function is obtained based on the noisy image and the noise-free image, and a neural network model is trained based on the loss function; The trained encoder is used to extract the multi-scale deep features of the image to be repaired, and the multi-scale deep features are input into the decoder B to obtain the repaired clean image.

2. The multi-scene image restoration method according to claim 1, characterized in that: The encoder includes Stem convolution, large kernel convolution, Transformer block and downsampling convolution, and the encoder is used to extract multi-scale depth features of image samples, specifically including: Get the tensor of the image sample and input the tensor into the Stem convolution, large kernel convolution, Transformer block and downsampling convolution respectively; Use Stem convolution to extract shallow features of image samples, use large kernel convolution to extract middle features of image samples, and use 2 sets of serially connected Transformer blocks and downsampling convolution to extract deep features of different dimensions of image samples respectively; The shallow features, mid-level features and deep features are represented as feature tensors respectively, and the collection of all feature tensors is regarded as the multi-scale deep features of the image samples.

3. The multi-scene image restoration method according to claim 2, characterized in that: Each of the Transformer blocks is composed of two basic Transformer modules connected in series, specifically including: Each basic Transformer module is connected in the form of residuals by layer regularization and multi-head self-attention, and then connected in series with a 3×3 convolution kernel activation function residual.

4. The multi-scene image restoration method according to claim 2, characterized in that: The downsampling convolution includes at least one convolution kernel, specifically including: The number of convolution kernels in the downsampling convolution is greater than the stride, and the output channel dimension of the downsampling convolution is equal to twice the input channel dimension.

5. The multi-scene image restoration method according to claim 1, characterized in that: The inputting of the multi-scale depth features into decoder A and decoder B respectively specifically includes: The feature tensor of each scale and dimension in the multi-scale deep features is obtained, and the obtained feature tensors are input into the corresponding modules of decoder A and decoder B respectively in a skip connection manner.

6. The multi-scene image restoration method according to claim 1, characterized in that: The decoder A includes four transposed convolution modules with the same structure, and the use of the decoder A to restore the noisy image specifically includes: The output result of the corresponding layer of the encoder and the output result of the upper decoder are obtained, and the obtained output results are input into the corresponding convolution of each transposed convolution module, so as to obtain the restored noisy image through the corresponding transposed convolution module.

7. The multi-scene image restoration method according to claim 6, characterized in that: Each transposed convolution module of the decoder A includes: batch normalization, first convolution, activation function, first transposed convolution, second convolution and second transposed convolution in sequence, and the obtained output result is input into the corresponding convolution of each transposed convolution module, specifically including: The output result of the corresponding layer of the encoder is input into the second convolution, and the output result of the decoder of the previous layer is input into batch normalization and the second transposed convolution.

8. The multi-scene image restoration method according to claim 1, characterized in that: The loss function is constructed as follows: Among them, DA i is the image output by decoder A, DB i The image output by decoder B, Dmg i is the i-th image sample, Gt i is the clean image corresponding to the i-th image sample, N is the total number of image samples, and L1(·) indicates that the mean absolute error loss function is used for calculation.

9. A multi-scene image restoration device, characterized in that: The method comprises at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions executable by the at least one processor, and the instructions, after being executed by the processor, are used to complete the multi-scene image restoration method described in any one of claims 1 to 8.

10. A non-volatile computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which are executed by one or more processors to complete the multi-scene image restoration method as described in any one of claims 1-8.

Images