A multi-scene-oriented image inpainting method and device

Abstract

The present application relates to the field of image inpainting, and particularly relates to a multi-scene-oriented image inpainting method and device.The present application: the neural network model used in image inpainting comprises an encoder, a decoder A and a decoder B;an image sample is input into the encoder, and the encoder is used to extract multi-scale deep features of the image sample;the multi-scale deep features are input into the decoder A and the decoder B respectively, the decoder A is used to restore a noisy image, and the decoder B is used to restore a non-noisy image;loss functions are obtained based on the noisy image and the non-noisy image, the neural network model is trained based on the loss functions;the trained encoder is used to extract multi-scale deep features of an image to be inpainted, and the multi-scale deep features are input into the decoder B to obtain a repaired clean image.The present application can enhance the image quality for various degradation factors, and restore a high-quality image.

Description

Technical Field

[0001] This invention relates to the field of image restoration, and in particular to an image restoration method and apparatus for multiple scenarios. Background Technology

[0002] With the rapid development and widespread adoption of artificial intelligence and the Internet of Things (IoT) technologies, visual tasks in outdoor scenarios have received extensive attention from both industry and academia. Outdoor scenarios, especially those with adverse weather conditions such as rain and fog, and low light levels, can introduce degrading factors like raindrop interference, snowflake interference, or low-light interference, significantly impacting image quality and resulting in a large number of low-quality images being captured. Due to weather noise and insufficient lighting, using low-quality images for visual tasks such as image classification or target recognition often leads to significant deviations in results.

[0003] Therefore, how to overcome the shortcomings of existing technologies and solve the problem of low image quality caused by scene factors is a problem to be solved in this technical field. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention solves the problem of low image quality caused by scene factors.

[0005] The embodiments of the present invention adopt the following technical solutions:

[0006] In a first aspect, the present invention provides an image restoration method for multiple scenarios, specifically comprising: a neural network model used for image restoration including an encoder, a decoder A, and a decoder B, comprising: inputting image samples into the encoder and using the encoder to extract multi-scale depth features of the image samples; inputting the multi-scale depth features into decoder A and decoder B respectively, using decoder A to restore the noisy image, and using decoder B to restore the noise-free image; obtaining a loss function based on the noisy image and the noise-free image, and training the neural network model based on the loss function; using the trained encoder to extract multi-scale depth features of the image to be restored, and inputting the multi-scale depth features into decoder B to obtain a restored clean image.

[0007] Preferably, the encoder includes Stem convolution, large kernel convolution, Transformer block, and downsampling convolution. The step of using the encoder to extract multi-scale depth features of image samples specifically includes: obtaining tensors of the image samples; inputting the tensors into Stem convolution, large kernel convolution, Transformer block, and downsampling convolution respectively; extracting shallow features of the image samples using Stem convolution, extracting mid-level features of the image samples using large kernel convolution, and extracting deep features of different dimensions of the image samples using two sets of concatenated Transformer blocks and downsampling convolution respectively; representing the shallow, mid-level, and deep features as feature tensors respectively, and using the set of all feature tensors as the multi-scale depth features of the image samples.

[0008] Preferably, each Transformer block consists of two basic Transformer modules connected in series, specifically: each basic Transformer module is connected by layer regularization and multi-head self-attention in the form of residuals, and then connected in series with a 3×3 convolutional kernel activation function residual.

[0009] Preferably, the downsampling convolution includes at least one convolution kernel, specifically including: the number of convolution kernels in the downsampling convolution is greater than the stride, and the output channel dimension of the downsampling convolution is equal to twice the input channel dimension.

[0010] Preferably, the step of inputting the multi-scale deep features into decoder A and decoder B respectively includes: obtaining the feature tensors of each scale and dimension in the multi-scale deep features, and inputting the obtained feature tensors into the corresponding modules of decoder A and decoder B respectively in a skip connection manner.

[0011] Preferably, the decoder A includes four transposed convolutional modules with identical structures. The process of using decoder A to recover a noisy image specifically includes: obtaining the output results of the corresponding layer of the encoder and the output results of the upper-level decoder, and inputting the obtained output results into the corresponding convolution of each transposed convolutional module so that the recovered noisy image can be obtained through the corresponding transposed convolutional module.

[0012] Preferably, each transposed convolutional module of the decoder A includes, in sequence: batch normalization, first convolution, activation function, first transposed convolution, second convolution, and second transposed convolution. The step of inputting the obtained output results into the corresponding convolution of each transposed convolutional module specifically includes: inputting the output results of the corresponding layer of the encoder into the second convolution, and inputting the output results of the previous layer decoder into the batch normalization and second transposed convolution.

[0013] Preferably, the loss function is constructed as follows:

[0014]

[0015] Among them, DA i DB is the image output by decoder A. i Dmg is the image output by decoder B. i For the i-th image sample, Gt i Let L1 be the clean image corresponding to the i-th image sample, N be the total number of image samples, and L1(·) represent the calculation using the mean absolute error loss function.

[0016] In a second aspect, the present invention provides an image restoration apparatus for multiple scenarios, specifically comprising at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, the memory stores instructions that can be executed by the at least one processor, and the instructions, after being executed by the processor, are used to complete the image restoration method for multiple scenarios in the first aspect.

[0017] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors to perform the method described in the first aspect.

[0018] Fourthly, a chip is provided, comprising: a processor and an interface for calling and running a computer program stored in memory, performing the method as described in the first aspect.

[0019] Fifthly, a computer program product containing instructions is provided that, when executed on a computer or processor, causes the computer or processor to perform the method as described in the first aspect.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: by using an asymmetric codec structure, deep feature learning of low-quality images and high-quality image restoration based on multiple constraints can be achieved, which can enhance image quality for various degradation factors and restore high-quality images better. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0022] Figure 1 This is a flowchart of an image restoration method for multiple scenarios provided in an embodiment of the present invention;

[0023] Figure 2Flowchart of another image restoration method for multiple scenarios provided in this embodiment of the invention

[0024] Figure 3 This is a schematic diagram of the encoder structure in the method provided in the embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of the Transformer block structure of the decoder in the method provided in the embodiments of the present invention;

[0026] Figure 5 This is a schematic diagram illustrating the connection relationship between the encoder and decoder A in the method provided in this embodiment of the invention;

[0027] Figure 6 This is a schematic diagram illustrating the connection relationship between the encoder and decoder B in the method provided in this embodiment of the invention;

[0028] Figure 7 This is a schematic diagram of the transposed convolution module structure of encoder A in the method provided in this embodiment of the invention;

[0029] Figure 8 This is a schematic diagram of an image restoration device for multiple scenarios provided in an embodiment of the present invention;

[0030] The reference numerals in the attached figures are as follows:

[0031] 11: Processor; 12: Memory. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0033] This invention is an architecture of a specific functional system. Therefore, the specific embodiments mainly describe the functional logic relationship of each structural module, and do not limit the specific software and hardware implementation methods.

[0034] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0035] Example 1:

[0036] To address the issue of low image quality caused by scene factors, this embodiment proposes an end-to-end solution that can efficiently repair images for multiple weather scenes, transforming the aforementioned low-quality images into high-quality images.

[0037] The method provided in this embodiment uses a neural network model for image inpainting. As shown in the figure, the neural network model used for image inpainting includes an encoder, decoder A, and decoder B.

[0038] like Figure 2 As shown, the specific steps of the image restoration method for multiple scenarios provided in this embodiment of the invention are as follows:

[0039] Step 101: Input the image samples into the encoder and use the encoder to extract multi-scale depth features of the image samples.

[0040] To perform image restoration using a neural network model, the model must first be trained using image samples, with each image sample serving as a potential image to be restored. Simultaneously, to optimize the neural network model, it is also necessary to obtain the corresponding clean real images from the image samples.

[0041] During training, image samples are first input into the encoder, which includes convolution and Transformer modules to extract multi-scale depth features from the image samples.

[0042] Step 102: Input the multi-scale depth features into decoder A and decoder B respectively, use decoder A to recover the noisy image, and use decoder B to recover the noise-free image.

[0043] After the encoder obtains multi-scale depth features, these features are input into the decoder. Two decoders then perform the image restoration task in parallel to complete the image inpainting. In practice, the two decoders have the same architecture, consisting of Transformer blocks and deconvolution. The two decoders run in parallel: decoder A is used to restore the high-quality image with noise, and decoder B is used to restore the clean image without noise.

[0044] Step 103: Obtain the loss function based on the noisy image and the noise-free image, and train the neural network model based on the loss function.

[0045] When training a neural network model, optimization using a loss function is necessary. In this embodiment, two decoders are used to obtain two types of output images. Therefore, it is also necessary to obtain the overall loss function of the neural network model based on the two types of output images and the clean images corresponding to the image samples, and then optimize the neural network model based on the overall loss function.

[0046] Step 104: Use the trained encoder to extract multi-scale depth features of the image to be repaired, and input the multi-scale depth features into decoder B to obtain the repaired clean image.

[0047] After training the neural network model, the trained model can be used to perform image restoration through model inference. During the model inference stage, only encoder + decoder B needs to be run, which does not increase the computational power required for model inference compared to existing technologies, allowing for image restoration work to be completed with relatively low computational consumption. Figure 1 In the diagram, the dashed line represents the neural network model framework used in the model inference stage.

[0048] After steps 101-104 provided in this embodiment, the low-quality image repair caused by scene factors can be completed, and a corresponding clean image can be obtained.

[0049] In practical implementation, the neural network model for repair can be established using the following methods, or the structure and parameters of the model can be adjusted according to actual needs. It is understood that the following specific implementation methods are only used to illustrate the specific implementation process of the method in certain specific scenarios in this embodiment, and are not intended to limit the scope of protection.

[0050] The encoder's network structure is as follows Figure 3 As shown, the encoder includes Stem convolution, big kernel convolution, Transformer block, and downsampling convolution.

[0051] The image to be repaired, input into the encoder, can be represented as a tensor, Dmg∈R. 3×H×W The tensors of the image samples are obtained and then input into Stem convolution, big kernel convolution, Transformer block and downsampling convolution respectively.

[0052] During model training, shallow features of image samples are extracted using Stem convolutions, mid-level features are extracted using large-kernel convolutions, and deep features of different dimensions of image samples are extracted using two sets of cascaded Transformer blocks and downsampling convolutions. During model inference, the same modules in the encoder acquire features at the corresponding scales in the image to be repaired in the same manner.

[0053] (1) The first part of the encoder is the Stem convolution, which consists of two concatenated convolutions. Each convolution has a 3×3 kernel, a stride of 2, and an output channel dimension of 96. This is used to extract shallow features from the input image. The feature dimension of the shallow features is...

[0054] (2) The second part of the encoder is a large kernel convolution. The large kernel convolution has a kernel size of 21, a stride of 2, and an output feature dimension of 256. In practice, large kernel convolution can be used for filtering to extract mid-level features from the image. The feature dimension of the mid-level features is...

[0055] (3) The third part of the encoder consists of a Transformer block and a downsampled convolutional block connected in series. This part is referred to as stage 1. The feature dimension output by this part is...

[0056] (4) The fourth part of the encoder is the same as the third part, consisting of a Transformer block and a downsampled convolutional block concatenated together. This part will be referred to as stage 2. The feature dimension output by this part is...

[0057] After each part of the encoder completes the above process, it can obtain features of different depths and scales of the image sample. The shallow features, medium features and deep features are represented as feature tensors respectively, and the set of all feature tensors is taken as the multi-scale depth features of the image sample.

[0058] In practice, the third and fourth parts of the encoder can be used to complete feature extraction in the following way.

[0059] The Transformer block used in Stage 1 is composed of two basic Transformer modules chained together, and its structure is as follows: Figure 4 The two dashed boxes in the diagram illustrate this. The main body of both basic Transformer modules consists of a multi-head attention mechanism and convolutions. Each basic Transformer module is formed by layer regularization and multi-head self-attention connected in the form of residuals, then concatenated with a convolutional kernel and an activation function residual. The convolutional kernel is a 3×3 kernel with a stride of 1.

[0060] The multi-head attention mechanism calculation formula is defined as follows:

[0061]

[0062] Here, x is the image input to Stage 1, which is input in the form of a feature tensor; Q, K, V are three 1×1 convolutions that output x as three feature tensors with the same dimensions as x; the spatial dimensions of the input and output tensors of the multi-head attention module do not change.

[0063] In downsampling convolutions, the number of kernels is greater than the stride; for example, a convolution with 7 kernels and a stride of 2. The output channel dimension of a downsampling convolution is twice the input channel dimension, thus a convolution with an output channel dimension twice that of the input is considered a downsampling convolution. Because the kernel of a downsampling convolution is larger than the stride, overlapping semantic features can be embedded into adjacent output pixels during the downsampling process, which helps to enhance the information interaction between different pixels.

[0064] Stage 2 is similar in structure to Stage 1, both consisting of basic Transformer modules. In practice, it can be composed of four basic Transformer blocks connected in series.

[0065] During the model training phase, two structurally similar decoders, A and B, are used. These two decoders are responsible for restoring the clean, repaired image and the original image, respectively. Specifically, the output of the corresponding layer of the encoder and the output of the higher-level decoder are obtained. The obtained outputs are then input into the corresponding convolutional layers of each transposed convolutional module to obtain the restored noisy image through the appropriate transposed convolutional module.

[0066] In practical implementation, feature tensors for each scale and dimension of the multi-scale deep features are obtained. These feature tensors are then input into the corresponding modules of decoder A and decoder B using skip connections. Skip connections are a method that directly connects input information from a certain level or branch of the model across several levels to subsequent levels or branches. By skipping intermediate processing levels, low-level and high-level features are directly connected, thereby promoting information flow and fusion. In actual implementation, the connection between the encoder and the two decoders forms a U-shaped network structure, with the encoder at the bottom and the two decoders as the two arms. Figure 5 As shown, this is the connection structure of encoder and decoder A. The first part of the encoder inputs the output to the second part and the fourth layer of decoder A. The second part inputs the output to Stage 1 and the third layer of decoder A. Stage 1 inputs the output to Stage 2 and the third layer of decoder A. Stage 2 inputs the output to the first layer of decoder A. Figure 6 As shown, this is the connection structure of encoder and decoder B, and the connection relationship is similar to that of encoder and decoder A.

[0067] In this embodiment, decoder A is used as an example for illustration. Decoder B can be constructed with reference to the structure of decoder A. Decoder A includes four transposed convolutional modules with identical structures.

[0068] like Figure 7As shown, each layer of decoder A is a transposed convolutional module. Each transposed convolutional module includes, in sequence: batch normalization, first convolution, activation function, first transposed convolution, second convolution, and second transposed convolution. Specifically, the first convolution is a 3x3 convolution with a stride of 1; the first transposed convolution is a 4x4 transposed convolution with a stride of 2; the second convolution is a 3x3 convolution with a stride of 2; and the second transposed convolution is a 4x4 transposed convolution with a stride of 2. During decoding, the output of Stage 2 of the decoder is input into the second transposed convolution in the first transposed convolutional module. The input of each transposed convolutional module in the second to fourth transposed convolutional modules consists of two parts: the output of the previous transposed convolutional module and the output of the corresponding transposed convolutional module in the encoder. Specifically: according to... Figure 5 The correspondence shown indicates that the output of the corresponding layer of the encoder is input into the second convolution, and the output of the previous layer's decoder is input into the batch normalization and second transposed convolution. Due to the encoder's excellent generalization ability and outstanding image feature representation capability, even using a general deconvolution module as the decoder can achieve remarkable image enhancement effects.

[0069] Compared to the traditional single encoder-single decoder architecture, the asymmetric encoder-decoder design provided in this embodiment uses two parallel decoders during the model training phase to perform two image reconstruction tasks. Since both tasks require recovering the overall semantic features of the image, and they differ in the details they recover: decoder A takes the encoder output as input and is responsible for recovering noisy or degraded images; decoder B, also taking its output as input, is responsible for recovering clean, noise-free images. The coupling between the two decoders in terms of coarse-grained features, and the complementarity in terms of fine-grained features, can generate more accurate regularization constraints for the decoders during model training, improving encoder performance and thus enhancing the overall performance of the model.

[0070] During the model training phase, the training data consists of a large number of image pairs (Dmg, Gt), where Dmg ∈ R. 3×H×W Let Gt ∈ R represent the image sample to be repaired. 3×H×W This represents the actual, clean image after restoration. For any input Dmg i The image output by decoder A is denoted as DA. i The image output by decoder B is denoted as DB. i Both have R as their data dimension. 3×H×W .

[0071] In practice, a suitable loss function can be selected for the output of each decoder as needed, and then the loss functions of the two decoders can be combined to obtain the overall loss function of the model. In a specific scenario, the mean absolute error loss function (L1 loss) is used as the loss function for the two decoder branches. The mean absolute error loss function has a stable gradient for any input value, which will not lead to gradient explosion and has a relatively robust solution.

[0072] The overall loss function of the model is constructed as follows:

[0073]

[0074] Among them, DA i DB is the image output by decoder A. i Dmg is the image output by decoder B. i For the i-th image sample, Gt i Let L1 be the clean image corresponding to the i-th image sample, N be the total number of image samples, and L1(·) represent the calculation using the mean absolute error loss function.

[0075] The image restoration method for multiple scenarios provided in this embodiment has the following advantages compared to existing technologies.

[0076] (1) It has strong generalization ability and is suitable for image restoration in various scenarios. The asymmetric encoding and decoding structure proposed in this embodiment has two decoders, which are used to restore high-quality images and restore the original images respectively. The two decoders have the characteristics of deep feature aliasing and shallow feature complementarity at the feature reconstruction level, which helps to improve the generalization ability and detail restoration ability of the model.

[0077] (2) The shared encoding features and coupled decoder structure contribute to high-quality image restoration. The two types of decoders apply regularization constraints to the encoder features from two dimensions, which can improve the encoder's feature learning ability and enable the model to better restore image details.

[0078] (3) The asymmetric codec structure can significantly improve the image restoration performance of the model without increasing computational power consumption. Compared with the traditional codec structure, the asymmetric codec structure proposed in this embodiment only increases the computational complexity during model training, without increasing the computational power consumption of model inference, and has low computational power consumption. At the same time, this asymmetric structure can be easily extended to most codec-based image restoration models, and has high innovation and application value.

[0079] Example 2:

[0080] Based on the multi-scene image restoration method provided in Embodiment 1 above, the present invention also provides a multi-scene image restoration apparatus that can be used to implement the above method, such as... Figure 8 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. This embodiment of the image restoration device for multiple scenarios includes one or more processors 11 and a memory 12. Figure 8 Take a processor 11 as an example.

[0081] Processor 11 and memory 12 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0082] The memory 12 serves as a non-volatile computer-readable storage medium for a multi-scene image restoration method. It can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the multi-scene image restoration method in Embodiment 1. The processor 11 executes various functional applications and data processing of the multi-scene image restoration device by running the non-volatile software programs, instructions, and modules stored in the memory 12, thereby implementing the multi-scene image restoration method of Embodiment 1.

[0083] Memory 12 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 12 may optionally include memory remotely located relative to processor 11, which can be connected to processor 11 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0084] The program instructions / modules are stored in memory 12. When executed by one or more processors 11, they perform the multi-scene image restoration method described in Embodiment 1 above, for example, the method described above. Figure 2 The steps shown.

[0085] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 8 One of the processors 11 can enable the one or more processors to execute the multi-scene image inpainting method of Embodiment 1, for example, to perform the above-described... Figure 2 The steps shown can also be implemented. Figure 1 , Figures 3 to 7 The various modules and units mentioned above.

[0086] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0087] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image restoration method for multiple scenarios, characterized in that, The neural network model used for image inpainting includes an encoder, decoder A, and decoder B, comprising: Image samples are input into the encoder, which extracts multi-scale depth features from the image samples. Multi-scale depth features are input into decoder A and decoder B respectively. Decoder A is used to recover the noisy image, and decoder B is used to recover the noise-free image. Decoder A and decoder B have the same structure. The loss function is obtained based on noisy and noiseless images, and the neural network model is trained based on the loss function. The trained encoder is used to extract multi-scale depth features from the image to be repaired, and the multi-scale depth features are input into decoder B to obtain the repaired clean image. The encoder includes Stem convolution, large kernel convolution, Transformer blocks, and downsampling convolution. The extraction of multi-scale depth features from image samples using the encoder specifically includes: Obtain tensors from image samples and input them into Stem convolution, large kernel convolution, Transformer block, and downsampling convolution, respectively. Use Stem convolution to extract shallow features from image samples, use large kernel convolution to extract mid-level features from image samples, and use two sets of concatenated Transformer blocks and downsampling convolution to extract deep features of image samples in different dimensions. Represent shallow, mid, and deep features as feature tensors, and use the set of all feature tensors as the multi-scale depth features of image samples. The decoder A includes four transposed convolutional modules with identical structures. The process of using decoder A to recover a noisy image specifically includes: obtaining the output results of the corresponding layer of the encoder and the output results of the upper-level decoder, and inputting the obtained output results into the corresponding convolution of each transposed convolutional module so that the recovered noisy image can be obtained through the corresponding transposed convolutional module. Each transposed convolutional module of the decoder A sequentially includes: batch normalization, a first convolution, an activation function, a first transposed convolution, a second convolution, and a second transposed convolution. The step of inputting the obtained output into the corresponding convolution of each transposed convolutional module specifically includes: The output of the corresponding layer of the encoder is input into the second convolution, and the output of the previous layer of the decoder is input into the batch normalization and the second transposed convolution.

2. The image restoration method for multiple scenarios according to claim 1, characterized in that, Each Transformer block consists of two basic Transformer modules connected in series, specifically including: Each basic Transformer module consists of layer regularization and multi-head self-attention connected in the form of residuals, and then concatenated with a 3×3 convolutional kernel activation function residual.

3. The image restoration method for multiple scenarios according to claim 1, characterized in that, The downsampling convolution includes at least one convolution kernel, specifically including: In downsampling convolution, the number of convolutional kernels is greater than the stride, and the output channel dimension of downsampling convolution is twice the input channel dimension.

4. The image restoration method for multiple scenarios according to claim 1, characterized in that, The step of inputting multi-scale depth features into decoder A and decoder B respectively includes: Obtain the feature tensors of each scale and dimension in the multi-scale deep features, and input the obtained feature tensors into the corresponding modules of decoder A and decoder B respectively in a skip connection manner.

5. The image restoration method for multiple scenes according to claim 1, characterized in that, The loss function is constructed as follows: ； in, The image output by decoder A. The image output by decoder B. For the i-th image sample, The clean image corresponding to the i-th image sample. The total number of image samples. This indicates that the mean absolute error loss function is used for calculation.

6. An image restoration device for multiple scenarios, characterized in that: The method includes at least one processor and a memory, which are connected via a data bus. The memory stores instructions that can be executed by the at least one processor. After being executed by the processor, the instructions are used to complete the multi-scene image restoration method according to any one of claims 1-5.

7. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which are executed by one or more processors to perform the multi-scene image restoration method as described in any one of claims 1-5.

Images