Image restoration model construction method, image restoration method, system and equipment

Abstract

The invention discloses an image inpainting model construction method, an image inpainting method, an image inpainting system and image inpainting equipment. The method comprises the following steps: taking a preprocessed image sample as the input of an initial image inpainting model; compressing and reconstructing the preprocessed image sample through a variational auto-encoder to obtain a reconstructed image sample; on the basis of forward propagation, a repaired image sample is obtained through a generative adversarial network according to the preprocessed image sample and the reconstructed image sample; calculating a peak signal-to-noise ratio value and a structural similarity value between the repaired image sample and the preprocessed image sample; and judging whether a preset restoration condition is met, if yes, storing the model parameters to obtain an image restoration model, and if not, performing iterative optimization on the initial image restoration model based on back propagation until the preset restoration condition is met. According to the method, automatic restoration is carried out through the optimized image restoration model, the restoration efficiency is improved, and a good restoration effect can be achieved in the face of large-area damage.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image restoration model construction method, image restoration method, system and device. Background Technology

[0002] Image data is prone to fading, scratches, stains, and damage during long-term preservation, leading to the loss of its aesthetic value and historical significance. Traditional restoration techniques rely on professional restorers using software such as Photoshop to manually repair defects pixel by pixel through methods such as layer repair and color correction. While this technique can precisely restore image details, it demands a high level of experience, artistic perception, and technical proficiency from the operator, and the restoration of a single image is time-consuming and inefficient.

[0003] In recent years, breakthroughs in deep learning technology have provided efficient and automated solutions for photo restoration. However, existing methods still have many limitations: when faced with complex damage (such as multiple scratches and stains, or severe local missing parts), the restoration results may exhibit blurred textures, distorted colors, or logical contradictions (such as misaligned architectural structures); the models lack generalization ability and have poor adaptability to rare damage types or special historical image styles; and the realism of detail restoration still needs improvement, such as the possibility that facial features and the unique yellowish texture of photos may be over-smoothed or lost during the restoration process. Therefore, how to improve the efficiency of image restoration while ensuring the effectiveness of the restoration has become an urgent problem to be solved.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention The main objective of this invention is to provide an image restoration model construction method, image restoration method, system, and device, aiming to solve the technical problem of how to improve image restoration efficiency while ensuring the image restoration effect.

[0005] To achieve the above objectives, the present invention provides an image restoration model construction method, the image restoration model construction method comprising: S1, preprocessing image samples with different degrees of damage; S2, the preprocessed image samples are used as input to the initial image inpainting model, which is constructed based on a combination of variational autoencoder and generative adversarial network; S3, the preprocessed image samples are compressed and reconstructed by the variational autoencoder to obtain reconstructed image samples; S4, Based on forward propagation, the repaired image sample is obtained through the generative adversarial network according to the preprocessed image sample and the reconstructed image sample; S5, calculate the peak signal-to-noise ratio and structural similarity value between the repaired image sample and the preprocessed image sample; S6. Determine whether the preset repair conditions are met based on the peak signal-to-noise ratio and structural similarity values. If so, save the model parameters to obtain the image repair model. Otherwise, iteratively optimize the initial image repair model based on backpropagation until the preset repair conditions are met.

[0006] Optionally, S1 includes: S1.1, Noise is added to image samples with different degrees of damage; S1.2, the noisy image samples are processed by Gaussian filtering using the cv2.GaussianBlur() function; S1.3, determine whether the filtered image samples meet the image enhancement conditions; S1.4 If so, then the filtered image sample is subjected to image enhancement processing using the torch.clamp() function.

[0007] Optionally, S4 includes: S4.1, The preprocessed image sample and the reconstructed image sample are input into the generator of the generative adversarial network, and the repaired image sample is generated through forward propagation.

[0008] Optionally, S5 includes: S5.1, the mean square error is calculated by the discriminator of the generative adversarial network based on the repaired image sample and the preprocessed image sample; S5.2, Based on the mean square error, calculate the peak signal-to-noise ratio (PSNR) and structural similarity values ​​between the repaired image sample and the preprocessed image sample using the peak signal-to-noise ratio (PSNR) formula and the structural similarity formula.

[0009] Optionally, the peak signal-to-noise ratio formula is:

[0010] In the formula, This is the peak signal-to-noise ratio value. The maximum pixel value of the image. This represents the mean square error.

[0011] Optionally, the structural similarity formula is:

[0012]

[0013]

[0014]

[0015]

[0016] In the formula, The structural similarity value. This refers to the pixel information at the i-th position within the preprocessed image sample. To repair the pixel information at the i-th position within the image sample, The mean of the preprocessed image samples. To repair the mean of the image samples, The standard deviation of the preprocessed image samples. To correct the standard deviation of image samples, For covariance, and For parameters.

[0017] Optionally, S6 includes: S6.1, determine whether the peak signal-to-noise ratio and the structural similarity value meet the preset repair conditions; S6.2 If so, save the model parameters to obtain the image restoration model; S6.3 If not, calculate the parameter gradient of the initial image restoration model based on the mean square error, and update the model parameters of the initial image restoration model according to the parameter gradient using gradient descent based on backpropagation. S6.4 Optimize the initial image restoration model according to the updated model parameters, and return to S2, iteratively execute until the preset restoration conditions are met.

[0018] Furthermore, to achieve the above objectives, this invention also proposes an image restoration method. The image restoration method based on the image restoration model constructed using the above-mentioned image restoration model construction method includes: Preprocess the image to be repaired; The preprocessed image is input into the image restoration model to obtain the restored image; Calculate the peak signal-to-noise ratio and structural similarity value between the preprocessed image and the restored image; When the peak signal-to-noise ratio and the structural similarity value meet the preset repair conditions, the repaired image is output through the image repair model.

[0019] Furthermore, to achieve the above objectives, the present invention also proposes an image restoration system, the image restoration system comprising: The processing module is used to preprocess image samples with different degrees of damage; An iterative module is used to take preprocessed image samples as input to an initial image inpainting model, which is constructed based on a combination of variational autoencoder and generative adversarial network. The encoding module is used to compress and reconstruct the preprocessed image samples using the variational autoencoder to obtain reconstructed image samples; The generation module is used to obtain repaired image samples through the generative adversarial network based on the preprocessed image samples and the reconstructed image samples; The optimization module is used to calculate the peak signal-to-noise ratio and structural similarity value between the repaired image sample and the preprocessed image sample; The optimization module is also used to determine whether the preset repair conditions are met based on the peak signal-to-noise ratio and structural similarity values. If so, the model parameters are saved to obtain the image repair model. Otherwise, the initial image repair model is iteratively optimized based on backpropagation until the preset repair conditions are met. The repair module is used to input the image to be repaired into the image repair model and output the repaired image.

[0020] This invention first preprocesses image samples with varying degrees of damage. The preprocessed image samples are then used as input to an initial image restoration model, which is constructed based on a combination of variational autoencoders (VAEs) and generative adversarial networks (GANs). The VAE compresses and reconstructs the preprocessed image samples to obtain reconstructed image samples. Based on forward propagation, the GAN generates restored image samples from the preprocessed and reconstructed image samples. The peak signal-to-noise ratio (PSNR) and structural similarity values ​​between the restored and preprocessed image samples are calculated. Then, based on the PSNR and structural similarity values, it is determined whether preset restoration conditions are met. If so, the model parameters are saved, resulting in the image restoration model. Otherwise, backpropagation is used to iteratively optimize the initial image restoration model until the preset restoration conditions are met. This invention does not require manual intervention and can directly utilize a deep learning model for automatic restoration, reducing labor costs, improving restoration efficiency, and demonstrating good restoration results even with large-area damage. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of the image restoration device in the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the first embodiment of the image restoration model construction method of the present invention; Figure 3 This is a diagram of the forward computation model of the neural network in the first embodiment of the image restoration model construction method of the present invention; Figure 4This is a diagram of the reverse computation model of the neural network in the first embodiment of the image restoration model construction method of the present invention. Figure 5 This is a flowchart of the model optimization process for the first embodiment of the image restoration model construction method of the present invention; Figure 6 This is a photograph with blurriness defects, representing the first embodiment of the image restoration model construction method of the present invention. Figure 7 This is a comparison image before and after denoising in the first embodiment of the image restoration model construction method of the present invention; Figure 8 This is a comparison image before and after contrast enhancement in the first embodiment of the image restoration model construction method of the present invention; Figure 9 This is an image restoration effect diagram of the first embodiment of the image restoration model construction method of the present invention; Figure 10 This is a structural block diagram of the first embodiment of the image restoration system of the present invention.

[0022] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0024] Reference Figure 1 , Figure 1 This is a schematic diagram of the image restoration device structure in the hardware operating environment involved in the embodiments of the present invention.

[0025] like Figure 1As shown, the image restoration device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. Optionally, the memory 1005 may also be a storage system independent of the aforementioned processor 1001.

[0026] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the image restoration device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0027] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and an image restoration program.

[0028] exist Figure 1 In the image restoration device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the image restoration device of the present invention can be set in the image restoration device, and the image restoration device calls the image restoration program stored in the memory 1005 through the processor 1001 and executes the image restoration method provided in the embodiment of the present invention.

[0029] This invention provides a method for constructing an image restoration model, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the image restoration model construction method of the present invention.

[0030] In this embodiment, the image restoration model construction method includes the following steps: Step S1: Preprocess image samples with different degrees of damage.

[0031] It should also be noted that image samples of varying degrees of damage can be multiple blurry or defective photos collected by the user from publicly available data platforms.

[0032] Furthermore, the preprocessing method for image samples with different degrees of damage is as follows: S1.1, add noise to the image samples with different degrees of damage; S1.2, perform Gaussian filtering on the noisy image samples using the cv2.GaussianBlur() function; S1.3, determine whether the filtered image samples meet the image enhancement conditions; S1.4, if yes, perform image enhancement on the filtered image samples using the torch.clamp() function, and use the enhanced image samples as the preprocessed image samples; S1.5, if no, use the filtered image samples as the preprocessed image samples.

[0033] In the specific implementation, image samples with different degrees of damage are subjected to noise addition using a noise addition formula, which is as follows:

[0034] In the formula, It follows a Gaussian distribution with a mean of 0 and a standard deviation of 0.1, i.e. ~ N(0, 0.1 2 ); For image samples; These are image samples after noise has been added.

[0035] Then, the `cv2.GaussianBlur()` function from Python's OpenCV library is used to perform Gaussian filtering on the noisy image samples. The convolution kernel size is set to `kernel_size=(3,3)`, and the Gaussian kernel standard deviation σ=1. The formula for the two-dimensional Gaussian filter is as follows:

[0036] Gaussian filtering for image denoising involves convolving the image with a Gaussian filter, resulting in filtered image samples. for:

[0037] In the formula, m and n are the data dimensions of the image samples after adding noise.

[0038] It should also be noted that manual judgment is required to determine whether image enhancement processing is needed for the filtered image samples. If so, the image sharpness and readability are enhanced by increasing the contrast (torch.clamp(D(x,y) * 1.2, 0, 1)), which multiplies the filtered image samples by 1.2 and limits the pixel values ​​to between 0 and 1. The formula for increasing contrast (i.e., the torch.clamp() function) is as follows:

[0039] in This is the scaling factor. It is calculated by multiplying by... By magnifying the differences in pixel values, the differences in pixel values ​​in areas that originally had low contrast become more apparent. This is an enhanced image sample.

[0040] Step S2: Use the preprocessed image samples as input to the initial image inpainting model, which is constructed based on a combination of variational autoencoder and generative adversarial network.

[0041] It should also be noted that when training the initial image restoration model, the batch size can be set to batch_size=32 and the number of training epochs to num_epochs=50. The input layer dimension is defined as input_dim=256, the hidden layer dimension as hidden_dim=128, and the latent space dimension as latent_dim=64. This embodiment does not impose any limitations, and users can customize and adjust these dimensions according to their needs.

[0042] Step S3: Compress and reconstruct the preprocessed image samples using the variational autoencoder to obtain reconstructed image samples.

[0043] In its implementation, the variational autoencoder (VAE) consists of an encoder and a decoder. The preprocessed image samples are input into the encoder of the variational autoencoder (VAE) to compress the image into a low-dimensional feature representation. Then, the decoder generates a preliminary reconstructed image (i.e., a reconstructed image sample) while preserving the overall structure of the image.

[0044] Step S4: Based on forward propagation, obtain the repaired image sample through the generative adversarial network according to the preprocessed image sample and the reconstructed image sample.

[0045] It should also be noted that, using preprocessed image samples and reconstructed image samples as input, the generator of a generative adversarial network (GAN) learns the details of the restored image to generate the restored image (i.e., the restored image sample).

[0046] In its implementation, a Generative Adversarial Network (GAN) consists of a generator and a discriminator. Preprocessed and reconstructed image samples are input into the generator, which generates repaired image samples through forward propagation. Figure 3 , Figure 3 This is a diagram of the forward computation model of the neural network in the first embodiment of the image restoration model construction method of the present invention. Assuming the input to the hidden layer is X (i.e., the preprocessed image sample I and the reconstructed image sample), the output H of the hidden layer is calculated through weighted summation and an activation function.

[0047] Where σ is the activation function, W1 is the hidden layer weight, and b1 is the hidden layer bias. The output restored image sample Y (i.e., the restored image sample J) of the output layer is:

[0048] In the formula, W2 is the output layer weight and b2 is the output layer bias.

[0049] Step S5: Calculate the peak signal-to-noise ratio and structural similarity value between the repaired image sample and the preprocessed image sample.

[0050] It should also be noted that the discriminator distinguishes between the original image (i.e., the pre-processed image) and the restored image, forcing the generator to continuously optimize the restoration effect. In this process, the model makes full use of the various features and restoration rules learned during the training phase to intelligently restore the damaged parts of the photo and restore the true appearance of the photo.

[0051] Furthermore, the processing method for calculating the peak signal-to-noise ratio (PSNR) and structural similarity values ​​between the repaired image sample and the preprocessed image sample is as follows: S5.1, calculate the mean square error using a discriminator based on the repaired image sample and the preprocessed image sample; S5.2, calculate the PSNR and structural similarity values ​​between the repaired image sample and the preprocessed image sample based on the mean square error using the PSNR formula and the structural similarity formula.

[0052] In this embodiment, the repaired image sample and the preprocessed image sample are input into the discriminator of the generative adversarial network. The mean square error between the repaired image sample and the preprocessed image sample is calculated using the mean square error formula. The mean square error is used as the loss function L to measure the difference between the reconstructed data and the original data. The smaller the deviation, the better the reconstruction effect of the model.

[0053] The formula for mean square error is:

[0054] In the formula, Mean square error, and For data dimensions, For pixel coordinates, These are preprocessed image samples. To repair image samples.

[0055] The formula for peak signal-to-noise ratio is:

[0056] In the formula, This is the peak signal-to-noise ratio value. The maximum pixel value of the image. This represents the mean square error.

[0057] The formula for structural similarity is:

[0058]

[0059]

[0060]

[0061]

[0062] In the formula, The structural similarity value. This refers to the pixel information at the i-th position within the preprocessed image sample. To repair the pixel information at the i-th position within the image sample, The mean of the preprocessed image samples. To repair the mean of the image samples, The standard deviation of the preprocessed image samples. To correct the standard deviation of image samples, For covariance, and For parameters, , .

[0063] Step S6: Determine whether the preset repair conditions are met based on the peak signal-to-noise ratio and structural similarity values. If so, save the model parameters to obtain the image repair model. Otherwise, iteratively optimize the initial image repair model based on backpropagation until the preset repair conditions are met.

[0064] Further, refer to Figure 5 , Figure 5The flowchart for the model optimization of the first embodiment of the image restoration model construction method of the present invention is as follows: S6.1, determine whether the peak signal-to-noise ratio and the structural similarity value meet the preset restoration conditions; S6.2, if yes, save the model parameters to obtain the image restoration model; S6.3, if no, calculate the parameter gradient of the initial image restoration model based on the mean square error, and iteratively update the model parameters of the initial image restoration model according to the parameter gradient through gradient descent based on backpropagation; S6.4, optimize the initial image restoration model according to the updated model parameters, and return to step S2 for iterative execution until the preset restoration conditions are met.

[0065] In the specific implementation, refer to Figure 4 , Figure 4 This is a diagram of the backpropagation model of the neural network in the first embodiment of the image restoration model construction method of the present invention. During backpropagation, the gradient of the loss function with respect to the output layer weights W2 and bias b2 is calculated according to the chain rule.

[0066] Next, calculate the gradient of the loss function with respect to the hidden layer output H:

[0067] Next, calculate the gradient with respect to the hidden layer weights W1 and bias b1:

[0068] in" "" indicates element-wise multiplication, and σ' is the derivative of the activation function σ. Backpropagation, by calculating the gradient, can accurately identify the parts of the model parameters that need adjustment, thus optimizing the model in the direction of reducing the loss function value.

[0069] Then, the model parameters are updated using gradient descent, with the learning rate η = 0.0001. The update formula is as follows:

[0070] It should also be noted that PSNR mainly measures the difference between the restored image and the original image at the pixel level. The higher the value, the closer the restored image is to the original image at the pixel level, and the better the restoration effect. SSIM takes into account the brightness, contrast and structural information of the image, and can better reflect the human eye's perception of image similarity. The closer its value is to 1, the more similar the restored image is to the original image in terms of structure and visual effect.

[0071] Meeting the preset restoration conditions for peak signal-to-noise ratio (PSNR) and structural similarity value means that the PSNR is greater than or equal to a preset PSNR threshold, and the structural similarity value is greater than a preset structural similarity threshold. These preset PSNR and structural similarity thresholds can be user-defined, for example, a PSNR threshold of 20dB and a structural similarity threshold of 0.8. When both PSNR and structural similarity values ​​meet the preset restoration conditions, the restored image is close to the original image in terms of pixel fidelity and visual perception, and is considered a valid restoration. The model training is then complete, and the model at this point is saved as the image restoration model.

[0072] In this embodiment, reference Figure 6-9 , Figure 6 This is a photograph with blurriness defects, representing the first embodiment of the image restoration model construction method of the present invention. Figure 7 This is a comparison image before and after denoising in the first embodiment of the image restoration model construction method of the present invention; Figure 8 This is a comparison image before and after contrast enhancement in the first embodiment of the image restoration model construction method of the present invention; Figure 9 This is an image restoration effect diagram of the first embodiment of the image restoration model construction method of the present invention; the image to be restored ( Figure 6 The original image (with blurry defects) is preprocessed, and the preprocessed image is input into the image restoration model to obtain the restored image; the peak signal-to-noise ratio (PSNR) and structural similarity value between the preprocessed image and the restored image are calculated; when the PSNR and structural similarity values ​​meet the preset restoration conditions, the restored image is output through the image restoration model.

[0073] It should also be noted that, for the image to be repaired ( Figure 6 The process involves preprocessing the original image (which has blurry defects): adding noise to the image to be repaired; applying Gaussian filtering to the noisy image using the cv2.GaussianBlur() function; determining whether the filtered image meets the image enhancement conditions; if so, applying image enhancement to the filtered image using the torch.clamp() function and using the enhanced image as the preprocessed image; otherwise, using the filtered image as the preprocessed image.

[0074] The preprocessed image is input into the trained image inpainting model. The preprocessed image is compressed and reconstructed by a variational autoencoder to obtain a reconstructed image sample. Based on forward propagation, the inpainted image is obtained by a generative adversarial network using the preprocessed image and the reconstructed image. The peak signal-to-noise ratio (PSNR) and structural similarity value between the inpainted image and the preprocessed image are calculated. When the PSNR and structural similarity values ​​meet the preset inpainting conditions, the inpainted image is output by the image inpainting model.

[0075] It should also be noted that when the peak signal-to-noise ratio (PSNR) and structural similarity value do not meet the preset restoration conditions, the image restoration model needs to be re-optimized based on the PSNR and structural similarity values ​​using backpropagation.

[0076] In this embodiment, image samples with different degrees of damage are first preprocessed. The preprocessed image samples are then used as input to an initial image restoration model, which is constructed based on a combination of variational autoencoders and generative adversarial networks (GANs). The variational autoencoder compresses and reconstructs the preprocessed image samples to obtain reconstructed image samples. Based on forward propagation, the GAN is used to obtain restored image samples from the preprocessed and reconstructed image samples. The peak signal-to-noise ratio (PSNR) and structural similarity values ​​between the restored and preprocessed image samples are calculated. Then, based on the PSNR and structural similarity values, it is determined whether the preset restoration conditions are met. If so, the model parameters are saved, and the image restoration model is obtained. Otherwise, based on backpropagation, the initial image restoration model is iteratively optimized until the preset restoration conditions are met. This embodiment does not require manual intervention and can directly utilize a deep learning model for automatic restoration, reducing labor costs, improving restoration efficiency, and demonstrating good restoration results even with large-area damage.

[0077] Reference Figure 10 , Figure 10 This is a structural block diagram of the first embodiment of the image restoration system of the present invention.

[0078] like Figure 10 As shown, the image restoration system proposed in this embodiment of the invention includes: Processing module 1010 is used to preprocess image samples with different degrees of damage; The iterative module 1020 is used to take the preprocessed image samples as input to the initial image inpainting model, which is constructed based on a combination of variational autoencoder and generative adversarial network. Encoding module 1030 is used to compress and reconstruct the preprocessed image samples using the variational autoencoder to obtain reconstructed image samples; The generation module 1040 is used to obtain repaired image samples through the generative adversarial network based on the preprocessed image samples and the reconstructed image samples; Optimization module 1050 is used to calculate the peak signal-to-noise ratio and structural similarity value between the repaired image sample and the preprocessed image sample; The optimization module 1050 is further configured to determine whether the preset repair conditions are met based on the peak signal-to-noise ratio and structural similarity values. If so, the model parameters are saved to obtain the image repair model. Otherwise, the initial image repair model is iteratively optimized based on backpropagation until the preset repair conditions are met. The repair module 1060 is used to input the image to be repaired into the image repair model and output the repaired image.

[0079] Other embodiments or specific implementations of the image restoration system of the present invention can be found in the above-described method embodiments, and will not be repeated here.

[0080] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0081] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0083] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for constructing an image restoration model, characterized in that, Includes the following steps: S1, preprocessing image samples with different degrees of damage; S2, the preprocessed image samples are used as input to the initial image inpainting model, which is constructed based on a combination of variational autoencoder and generative adversarial network; S3, the preprocessed image samples are compressed and reconstructed by the variational autoencoder to obtain reconstructed image samples; S4, Based on forward propagation, the repaired image sample is obtained through the generative adversarial network according to the preprocessed image sample and the reconstructed image sample; S5, calculate the peak signal-to-noise ratio and structural similarity value between the repaired image sample and the preprocessed image sample; S6. Determine whether the preset repair conditions are met based on the peak signal-to-noise ratio and structural similarity values. If so, save the model parameters to obtain the image repair model. Otherwise, iteratively optimize the initial image repair model based on backpropagation until the preset repair conditions are met.

2. The method as described in claim 1, characterized in that, S1 includes: S1.1, Noise is added to image samples with different degrees of damage; S1.2, the noisy image samples are processed by Gaussian filtering using the cv2.GaussianBlur() function; S1.3, determine whether the filtered image samples meet the image enhancement conditions; S1.4 If so, then the filtered image sample is subjected to image enhancement processing using the torch.clamp() function.

3. The method as described in claim 1, characterized in that, The S4 includes: S4.1, The preprocessed image sample and the reconstructed image sample are input into the generator of the generative adversarial network, and the repaired image sample is generated through forward propagation.

4. The method as described in claim 1, characterized in that, The S5 includes: S5.1, the mean square error is calculated by the discriminator of the generative adversarial network based on the repaired image sample and the preprocessed image sample; S5.2, Based on the mean square error, calculate the peak signal-to-noise ratio (PSNR) and structural similarity values ​​between the repaired image sample and the preprocessed image sample using the peak signal-to-noise ratio (PSNR) formula and the structural similarity formula.

5. The method as described in claim 4, characterized in that, The peak signal-to-noise ratio formula is: In the formula, This is the peak signal-to-noise ratio value. The maximum pixel value of the image. This represents the mean square error.

6. The method as described in claim 4, characterized in that, The structural similarity formula is: In the formula, The structural similarity value. This refers to the pixel information at the i-th position within the preprocessed image sample. To repair the pixel information at the i-th position within the image sample, The mean of the preprocessed image samples. To repair the mean of the image samples, The standard deviation of the preprocessed image samples. To correct the standard deviation of image samples, For covariance, and For parameters.

7. The method as described in claim 4, characterized in that, The S6 includes: S6.1, determine whether the peak signal-to-noise ratio and the structural similarity value meet the preset repair conditions; S6.2 If so, save the model parameters to obtain the image restoration model; S6.3 If not, calculate the parameter gradient of the initial image restoration model based on the mean square error, and update the model parameters of the initial image restoration model according to the parameter gradient using gradient descent based on backpropagation. S6.4 Optimize the initial image restoration model according to the updated model parameters, and return to S2, iteratively execute until the preset restoration conditions are met.

8. An image restoration method, characterized in that, The image restoration method based on the image restoration model construction method of claim 1 includes: Preprocess the image to be repaired; The preprocessed image is input into the image restoration model to obtain the restored image; Calculate the peak signal-to-noise ratio and structural similarity value between the preprocessed image and the restored image; When the peak signal-to-noise ratio and the structural similarity value meet the preset repair conditions, the repaired image is output through the image repair model.

9. An image restoration system, characterized in that, The image restoration system includes: The processing module is used to preprocess image samples with different degrees of damage; An iterative module is used to take preprocessed image samples as input to an initial image inpainting model, which is constructed based on a combination of variational autoencoder and generative adversarial network. The encoding module is used to compress and reconstruct the preprocessed image samples using the variational autoencoder to obtain reconstructed image samples; The generation module is used to obtain repaired image samples through the generative adversarial network based on the preprocessed image samples and the reconstructed image samples; The optimization module is used to calculate the peak signal-to-noise ratio and structural similarity value between the repaired image sample and the preprocessed image sample; The optimization module is also used to determine whether the preset repair conditions are met based on the peak signal-to-noise ratio and structural similarity values. If so, the model parameters are saved to obtain the image repair model. Otherwise, the initial image repair model is iteratively optimized based on backpropagation until the preset repair conditions are met. The repair module is used to input the image to be repaired into the image repair model and output the repaired image.

10. An image restoration device, characterized in that, The device includes: a memory, a processor, and an image restoration program stored in the memory and executable on the processor, the image restoration program being configured to implement the steps of the image restoration method as claimed in claim 8.

Images