Method for comprehensive repair of scratch old photo

Abstract

The application discloses a method for comprehensive repair of scratches in old photos, wherein a deep learning model network is trained, scratches and faces in the photos are detected through different detectors, and the scratches and faces are repaired through different technical solutions; even if a portrait photo with a damaged face, a small target face detection algorithm and a face generation recovery network can also have a good repair effect.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of low-level computer vision, and particularly relates to a method for comprehensive repair of scratchy old photos. BACKGROUND

[0002] With the development of intelligent devices and the popularization of computer resources, it is simple and efficient to convert traditional paper photos into digital format. Compared with paper photos, the advantages of digital archives are easy to edit, review and spread. Due to the passage of time, paper photos are mostly damaged or creased, and after scanning, the photos mostly have the problems of blurred quality and rough structure, and the problems are particularly serious for portrait photos. For the problem of comprehensive repair of scratchy old photos, it can be divided into traditional methods and deep learning-based methods. The traditional repair method mainly manually marks the repair area, and the algorithm calculates the filling pixels according to the edge pixels of the repair area, and finally iteratively fills to achieve the purpose of repair, but this method lacks deep understanding of the image content, and the repaired image still has the problems of blurring and discontinuous structure. In recent years, deep learning technology has developed rapidly and is widely used in image processing tasks. At present, the old photo repair method based on deep learning mostly focuses on single degradation information repair or still has the problems of blurred quality and texture misplacement. The image restoration method based on diffusion model can control the structure information of the damaged image from a global perspective to complete the repair of the damaged image. In portrait photos, due to the characteristics of incomplete content and low resolution of degraded old photos, the detection and restoration of the human face also face certain challenges, and most of the existing old photo restoration technical solutions do not have targeted operation and processing for portrait photos. After enhancing the face and quality, the details of the old photo can be more sharp and clear. Finally, the colorization operation of the old photo can make the repair effect more vivid and the visual effect better. Most of the existing old photo scratch repair technical solutions have the problem of distorted repair effect, especially in portrait photos. For single portrait scratches and group photo scratches, the visual effect is not good after repair. The face detection efficiency in the existing photo repair system is low and cannot be applied to group photos or old photos with small face regions. In addition, when the old photo is repaired and colored, due to the low clarity and few texture details, the outline color may overflow. SUMMARY

[0003] To solve the above technical problems, the present application provides a method for comprehensive repair of scratchy old photos, which trains a deep learning model network, detects scratches and faces in the photo through different detectors, and repairs them through different technical solutions. Through the small target face detection algorithm and the face generation and restoration network, even the portrait photo with damaged face can have a good repair effect.

[0004] To achieve the above object, the application provides a method for comprehensive repair of scratchy old photos, comprising:

[0005] S1, a diffusion model is established, the scratchy old photo to be repaired is added with noise based on a noise adding mode of the diffusion model, a complete Gaussian noise image is obtained, and a scratch denoising space is established according to the scratchy old photo to be repaired;

[0006] S2, a noise prediction network is established, noise prediction is performed on the complete Gaussian noise image and the scratch denoising space, corresponding pure noise images are obtained respectively, and scores of the pure noise images are calculated;

[0007] S3, the scores of the pure noise images are weighted and calculated to obtain a state image, reverse iterative denoising processing is performed on the state image based on a denoising mode of the diffusion model, and a scratch-free old photo is obtained;

[0008] S4, an image super-resolution network is established, a convolutional neural network is used to extract a shallow feature map of the scratch-free old photo, and the shallow feature map is directly input into a reconstruction module through a long skip connection for retaining low-frequency information;

[0009] S5, the shallow feature map is input into a deep feature extraction module, deep feature modeling is performed on the shallow feature map to obtain a deep feature map for restoring lost high-frequency information;

[0010] S6, the low-frequency information is directly input into the reconstruction module through the long skip connection to help the deep feature extraction module focus on high-frequency information, a sub-pixel convolutional layer is used to up-sample the low-frequency information and the high-frequency information, and the scratch-free old photo is reconstructed by gathering the up-sampled low-frequency information and high-frequency information, and a clear scratch-free old photo is obtained through 4 times super-resolution;

[0011] S7, a face detection network is established, the clear scratch-free old photo is input into the face detection network for face detection in the photo, and a face local area cutout image is obtained;

[0012] S8, a face recovery network is established, the face local area cutout image is processed separately for face recovery, and a repaired face image is obtained;

[0013] S9, the repaired face is histogram matched with a face detection position to obtain a clear scratch-free photo with a complete face;

[0014] S10, an image coloring network is established, the clear scratch-free photo with the complete face is colored to obtain a scratch-free color photo with a clear face.

[0015] Further, the method for obtaining a complete Gaussian noise map is:

[0016] dx=θ t (η-x)dt+ξ t dw

[0017] wherein x is a state map at time t, θ t (η-x) and ξ t are drift and dispersion equations respectively, η is a scratch old photo, dw is Gaussian noise, θ t and ξ t are positive parameters related to time.

[0018] Further, the method for establishing a scratch denoising space based on the scratch old photo to be repaired comprises:

[0019] Based on an odd Gaussian convolution kernel, the scratch old photo to be repaired is subjected to Gaussian blur processing to obtain a brightness information feature map that retains the brightness information of the picture and eliminates the brightness information of the scratch;

[0020] A scratch detection network is established and trained to detect the scratch of the scratch old photo to be repaired to obtain a scratch normal distribution;

[0021] Based on the brightness information feature map and the scratch normal distribution, the scratch denoising space is established.

[0022] Further, the method for establishing and training the scratch detection network comprises:

[0023] Based on real scratch old photos and synthetic scratch simulation old photos, a scratch old photo data set is established;

[0024] The scratch area of each old photo in the scratch old photo data set is labeled, and the scratch old photo and the labeled photo to be segmented are one-to-one corresponding, an image processing network is trained, and a scratch detection result is obtained;

[0025] Based on the scratch detection result, the cross-entropy loss minimization is used to detect the difference between the scratch and the actual scratch;

[0026] Based on the difference, a target function of the scratch detection network is obtained, and the establishment and training of the scratch detection network are completed.

[0027] Further, the method for obtaining the scratch normal distribution comprises:

[0028] Based on the scratch detection network, the scratch of the scratch old photo to be repaired is detected to obtain a binary graph mask with scratch position information;

[0029] The binary mask is cropped with the scratch old photo to be repaired to obtain a feature map with only scratch information;

[0030] The feature map with only scratch information is encoded to obtain scratch normal distribution of the scratch old photo to be repaired.

[0031] Further, the method for obtaining the scratch-free old photo comprises:

[0032] The scores of each pure noise map are weighted to obtain a final score calculation result;

[0033] Based on the final score calculation result, a next stage state map is obtained;

[0034] Based on the denoising mode of the diffusion model, the state map is reversely iteratively denoised to exclude scratch information and generate global semantic information, thereby obtaining the scratch-free old photo.

[0035] Further, the method for extracting the shallow feature map of the scratch-free old photo using a convolutional neural network comprises:

[0036] F0=H SF (I LQ )

[0037] Wherein, F0 is a shallow feature map; I LQ is a scratch-free old photo; H SF (.) is a convolutional neural network.

[0038] Further, the method for obtaining a deep feature map by modeling the shallow feature map comprises:

[0039] F DF =H DF (F0)

[0040] Wherein, F DF is a deep feature map; H DF (.) is a deep feature extraction module; F0 is a shallow feature map.

[0041] Further, the method for obtaining the clear scratch-free old photo comprises:

[0042] I RHQ =H REC (F0+F DF )

[0043] Wherein, I RHQ is a clear scratch-free old photo; H REC (.) is a reconstruction module; F DF is a deep feature map; F0 is a shallow feature map.

[0044] The technical effect of the present application: the present application discloses a method for comprehensive repair of scratch old photos, which trains a deep learning model network, detects scratches and faces in the photos through different detectors, and repairs them through different technical solutions; through a small target face detection algorithm and a face generation recovery network, even a portrait photo with a damaged face can have a good repair effect. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application and their description together with the drawings serve to explain the application. In the drawings:

[0046] Figure 1 A flowchart of the method for comprehensive repair of scratch old photos according to an embodiment of the present application is shown in the figure.

[0047] Figure 2 A flowchart of the method for removing scratches from old photos according to an embodiment of the present application is shown in the figure.

[0048] Figure 3 A network diagram of noise prediction according to an embodiment of the present application is shown in the figure.

[0049] Figure 4 A flowchart of the image super-resolution method according to an embodiment of the present application is shown in the figure.

[0050] Figure 5 A face enhancement structure diagram according to an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0051] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0052] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0053] As shown in the figure, a method for comprehensive repair of scratch old photos is provided in the embodiment, which comprises: Figure 1

[0054] S1, a diffusion model is established, and based on the noise adding mode of the diffusion model, a scratch old photo to be repaired is added with noise to obtain a complete Gaussian noise image; and a scratch denoising space is established;

[0055] The specific method for establishing the diffusion model is:

[0056] ​The signal is added by the noise of the stochastic differential equation, that is, Brown motion, which is a continuous Gaussian process. It injects stable and random noise into the system at each continuous moment, and changes the discrete noise adding and denoising process into a continuous process. Where w represents Brown motion (or Wiener process), θ t (η-x) controls the "flow direction" of the entire SDE, ξ t controls the diffusion degree or noise adding speed of the SDE. Gaussian noise is sampled at each step, and the variance gradually increases to finally reach a completely Gaussian noise graph. That is, Gaussian noise is injected into the old photo with scratches to be repaired at each continuous moment to obtain a completely Gaussian noise graph, and the formula is as follows:

[0057] dx=θ t (η-x)dt+ξ t dw,

[0058] Where x is the state graph at time t, θ t (η-x) and ξ t are the drift and dispersion equations respectively, η is the old photo with scratches, dw is Gaussian noise, θ t and ξ t are positive parameters related to time; in this process, the overall time T is set to 100, and the noise is changed from a discrete process to a continuous process by referring to the diffusion method of the SDE stochastic differential equation.

[0059] The completely Gaussian noise graph and the scratch denoising space are denoised by using the reverse iterative denoising method, and the formula is as follows:

[0060]

[0061] Where, is the probability density logarithmic gradient function; x is the state graph at time t (that is, the completely Gaussian noise graph or the state graph at time t of the scratch denoising space)

[0062] Based on the noise adding method of the stochastic differential equation and the reverse iterative denoising method, the discrete noise adding and denoising process is converted into a continuous noise adding and denoising process, and the diffusion model is established.

[0063] Based on the old photo with scratches to be repaired, a scratch denoising space is established. The space construction is to predict the noise in the Gaussian blurred photo by the distribution of the scratches, inputting t and the Gaussian blurred photo and x(t) into the noise prediction network for noise prediction, because in this way the brightness information of the original picture can be retained while the scratch information can be excluded.

[0064] Further embodiments are that in step S1, the method for establishing the scratch denoising space is:

[0065] As shown in Figure 2 Based on the odd Gaussian convolution kernel, the pixel value center point weighted average value of the old photo to be repaired with scratches is calculated, the Gaussian blur processing is completed, and a brightness information feature map is obtained through this process, which retains the brightness information of the picture and eliminates the brightness information features of high-frequency information such as scratches;

[0066] The scratch detection network is established and trained, the scratch detection is performed on the old photo to be repaired with scratches, and the normal distribution of the scratches is obtained.

[0067] Based on the brightness information feature map and the normal distribution of the scratches, the scratch denoising space is established.

[0068] Further embodiments are that the method for obtaining the brightness information feature map is:

[0069] Based on the odd Gaussian convolution kernel, the center point weighted average value of the old photo to be repaired with scratches is calculated, the Gaussian blur processing is completed, and the brightness information feature map is obtained.

[0070] The scratch detection and segmentation network uses the image processing network Unet which is widely used in application scenarios. Through fine-tuning and retraining of the network, it is used in the scratch detection and segmentation scenario. Unet is used to extract the scratch features of the picture through multiple downsampling and upsampling operations.

[0071] Further embodiments are that the method for establishing and training the scratch detection network is:

[0072] Based on the real old photo with scratches and the synthetic scratch simulation old photo, the scratch old photo data set is established;

[0073] The scratch area of each old photo in the scratch old photo data set is labeled, and the scratch old photo and the labeled photo to be segmented are one-to-one corresponding. The image processing network Unet is trained to obtain the scratch detection result;

[0074] Based on the scratch detection result, the cross-entropy loss minimization is used to detect the difference between the scratch mask and the actual scratch mask y;

[0075] Based on the difference, the objective function of the scratch detection network is obtained, and the establishment and training of the scratch detection network are completed.

[0076] Further embodiments are that the method for obtaining the normal distribution of the scratches is:

[0077] Based on the scratch detection network, the scratch detection is performed on the old photo to be repaired with scratches, and a binary graph mask with scratch position information is obtained;

[0078] The binary graph mask and the old photo to be repaired with scratches are subjected to cutting operation to obtain a feature map with only scratch information.

[0079] The encoder is used to encode the feature map with only scratch information to obtain the normal distribution of scratches of the old photo to be repaired. Specifically, a picture with only scratch content can be obtained through a simple cutting operation, and a set of means and a set of variances can be obtained by encoding the picture, and the means and variances correspond to a Gaussian distribution respectively.

[0080] S2: Establishing a noise prediction network, predicting noise for the complete Gaussian noise map and the scratch denoising space to obtain corresponding pure noise maps respectively, and calculating scores of the pure noise maps. Specifically, the output in S1 is subjected to reverse iterative denoising processing, and only the unknown is The probability density logarithmic gradient function is a score, a noise prediction network based on U-net is trained, the network is trained in a maximum likelihood estimation manner to make more accurate prediction of the noise in the current state. In the test generation, the noise in the current state is predicted by the network to obtain a pure noise map, and finally the score is calculated. The network diagram is shown in Figure 3 .

[0081] S3: Weighted calculation of scores of the pure noise maps to obtain a state map; wherein the score is a probability density logarithmic gradient function; based on the denoising mode of the diffusion model, the state map is subjected to reverse iterative denoising processing to obtain a clean old photo without scratches.

[0082] Further embodiments are that in step S3, the method for obtaining the clean old photo without scratches is:

[0083] The scores of the pure noise maps are weighted and calculated to obtain a final score calculation result;

[0084] Based on the final score calculation result, a next stage state map is obtained;

[0085] Based on the denoising mode of the diffusion model, the state map is subjected to reverse iterative denoising processing to exclude scratch information and generate global semantic information to obtain the clean old photo without scratches.

[0086] In this embodiment, the number of normal distributions in S2 is set to 3, which generates three scores in the Gaussian state space. In addition to the score predicted by the scratch noise map, four scores are combined to obtain the final score. In S3, the calculation mode is generated in reverse, and the next state map is obtained according to the final score.

[0087] According to the calculated score, x(t) can obtain the state diagram x(t-1) of the next stage. Through this iterative iteration, this process can be regarded as the reverse recovery of S1, but in the reverse recovery process, the information of the scratch is excluded and the global semantic information is generated. The predicted noise and scratch are regarded as abnormal information, and the noise is taken away. After removing the noise, the noise is filled according to the reverse formula to calculate x(t-1). It can be simply understood that in a vector space, the denoising process goes to the target image that wants to achieve. From T to gradually restore to X, the scratch is gradually removed and new content is generated, and finally a clear and scratch-free photo is obtained, that is, the repaired photo.

[0088] S4: Establish an image super-resolution network, use a convolutional neural network to extract the shallow feature map of the scratch-free old photo, and directly transmit it to the reconstruction module through a long skip connection to retain the low-frequency information.

[0089] The image super-resolution process is shown in Figure 4 Further embodiments are based on the repaired scratch-free old photo LQ, Then a convolution module H SF (.) is used to extract shallow features F0∈R H，W，C

[0090] F0=H SF (I LQ ),

[0091] S5: The shallow feature map is sent to a deep feature extraction module to model the deep features of the shallow features and obtain a deep feature map for restoring the lost high-frequency information.

[0092] Further embodiments are based on the obtained shallow features F0, which are further extracted using a deep feature extraction module HDF. The deep feature extraction module is composed of K residual Swin Transformer blocks (RSTB) and a 3x3 convolution.

[0093] F DF =H DF (F0),

[0094] The output F1, F2, F K of each RSTB, and the output deep feature F DK , where represents the i-th RSTB module, H CONV represents the final convolution layer. The convolution layer can introduce the inductive bias of convolution into the Transformer-based network, laying a foundation for the fusion of subsequent shallow and deep features.

[0095]

[0096] F DF =H CONV (F K ),

[0097] S6: By long jump connection, the low-frequency information is directly transmitted to the reconstruction module, helping the deep feature extraction module focus on high-frequency information, using sub-pixel convolution layer to up-sample the low-frequency information and the high-frequency information, and reconstructing the old photo without scratches by gathering the up-sampled low-frequency information and the high-frequency information, and performing 4 times super-resolution to obtain a clear old photo without scratches.

[0098] Further implementation is to reconstruct a high-quality picture I DK by fusing the shallow feature F0 and the deep feature F RHQ , wherein H REC is a reconstruction module.

[0099] I RHQ =H REC (F0+F DF ),

[0100] The shallow feature F0 mainly contains low-frequency information, while the deep feature focuses on restoring the lost high-frequency information. Specifically, a long distance connection is adopted to directly transmit the low-frequency information to the reconstruction module, which can help the deep feature extraction module focus on high-frequency information and stabilize the training. The features are up-sampled by sub-pixel convolution layer to realize reconstruction.

[0101] S7: Establishing a face detection network, inputting the clear old photo without scratches into the face detection network for face detection in the photo, and obtaining a face local area cutout;

[0102] For this face detection problem, three aspects are mainly developed: scale invariance, detection accuracy and the role of context reasoning. That is, different detectors are trained according to the faces in different scale ranges, at the same time, the super-resolution of the image is realized by S4, so that the face feature points are more prominent, greatly improving the accuracy of face detection, in addition, the range of face context is changed to improve the accuracy of face recognition.

[0103] S8: Establishing a face recovery network, processing the face local area cutout separately for face recovery, and obtaining a repaired face image;

[0104] The recovery quality of the face is most sensitive to human perception, so a network for face enhancement (GPEN) is used. The GPEN repair algorithm uses GAN as an image prior design to refine the face area of the old photo.

[0105] Face recovery network Figure 5 As shown, the overall architecture follows the U-net framework, employing a CNN encoder to learn how to map the input blurred image x into the latent space Z. Then, image super-resolution is performed using a GAN prior network. A further implementation involves first pre-training the GAN prior network according to the StyleGAN training strategy, and then embedding the pre-trained GAN model into GPEN. To fine-tune the GPEN model, three loss functions are used: adversarial loss L... A Content loss L C Feature matching loss L F :

[0106]

[0107] Among them, X and This indicates a clear image versus a degraded image.

[0108]

[0109] Where T is the total number of intermediate layers used for feature extraction, and D i (X) is the feature extracted at the i-th layer.

[0110] The FFHQ dataset was used to train a GAN prior network, and the following degradation model was used to synthesize LQ images from sharp images in FFHQ.

[0111]

[0112] Where I is the input image, K is the blur kernel, and n σ It is Gaussian noise with a standard deviation σ, I d This is a degraded image. A further implementation involves that, for each image, the blur kernel K is randomly selected from a set of blur models, including Gaussian blur and motion blur with different kernel sizes. Additive Gaussian noise n σ It involves sampling from a normal distribution channel by channel.

[0113] By using these severely degraded images to fine-tune the model, the encoder part of GPEN can learn to generate appropriate latent codes and noise inputs to the GAN prior decoder network.

[0114] The trained GAN prior network is embedded into a U-shaped DNN as a prior decoder, and then a set of synthetic low neural networks is used to fine-tune the GAN prior embedding DNN, at this time, the latent code z is replaced by the fully connected layer of the DNN encoder, and the noise input of the GAN network is replaced by the output of each layer of the encoder, which will control the reconstruction of the global face structure, the local face details and the face image background. At the same time, in order to leave space for the feature map extracted by the encoder, additional noise input is provided for each GAN block.

[0115] S9: Histogram matching of the repaired face and face detection position is performed to obtain a clear and unscratched photo with a complete face.

[0116] S10: An image coloring network is established, and a technology named DeOldify is used. Since the coloring effect of the original model is too thick, after retraining the photo through the processing of S4-S8 and adjusting the parameters, the coloring effect will be more visually realistic, and finally a color photo without scratches and with a clear face is obtained.

[0117] Further implementation is that, for the coloring problem of the old photo face, after retraining by fine-tuning the training parameters, the image coloring effect will be more natural, and based on the S4 super-resolution step, the outline of the colored picture is clearer, and the color overflow phenomenon does not occur. At the same time, based on the S8 face recovery network, the face details are greatly improved, and then coloring is performed, so that the color filling is more detailed and the texture is clearer.

[0118] According to the above steps, the overall logic and process of the scheme are as shown in Figure 1 After inputting the old photo, the steps S1-S3 are performed to obtain a scratch-free old photo, the scratch-free old photo is taken as input, the steps S4-S6 are performed to obtain a clear scratch-free old photo, and then the steps S7 and S8 are performed to separately recover the face part of the old photo. After performing the face recovery, S9 is performed to obtain a clear, unscratched and complete face photo, and finally S10 is performed to color the clear, unscratched and complete face photo.

[0119] The application discloses a method for comprehensively repairing a scratchy old photo, which trains a deep learning model network, detects scratches and faces in the photo through different detectors, and repairs the scratches and faces through different technical solutions.

[0120] The above merely provides the preferred embodiments of the present application, and the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for comprehensive restoration of scratched old photographs, characterized in that, include: S1. Establish a diffusion model, and based on the noise addition method of the diffusion model, add noise to the old photo with scratches to be repaired to obtain a complete Gaussian noise map. A scratch denoising space is created based on the old photo with scratches to be repaired; The method for establishing a scratch denoising space based on the old photo with scratches to be repaired includes: Based on an odd number of Gaussian convolution kernels, the old photo with scratches to be repaired is subjected to Gaussian blurring to obtain a brightness information feature map that retains the brightness information of the image and eliminates the scratches. A scratch detection network was established and trained to perform scratch detection on the old photos with scratches to be repaired, and the normal distribution of scratches was obtained. Based on the brightness information feature map and the scratch normal distribution, the scratch denoising space is established; The method for obtaining the normal distribution of the scratches includes: Based on the scratch detection network, scratch detection is performed on the old photo with scratches to be repaired to obtain a binary image mask with scratch location information; The binary image mask is cropped with the old photo of the scratches to be repaired to obtain a feature map containing only scratch information. Encode the feature map containing only scratch information to obtain the normal distribution of scratches in the old photo of the scratches to be repaired; S2. Establish a noise prediction network to predict noise in the complete Gaussian noise map and the scratch denoising space, obtain the corresponding pure noise maps respectively, and calculate the score of each pure noise map. S3. Calculate the scores of each of the pure noise maps by weighting to obtain a state map; based on the denoising method of the diffusion model, perform reverse iterative denoising processing on the state map to obtain a scratch-free old photo; the score is the logarithmic gradient function of the probability density. S4. Establish an image super-resolution network, use a convolutional neural network to extract the shallow feature map of the scratch-free old photo, and directly input the shallow feature map into the reconstruction module through long skip connections to retain low-frequency information. S5. Input the shallow feature map into the deep feature extraction module to perform deep feature modeling on the shallow feature map and obtain a deep feature map for recovering lost high-frequency information; S6. Through long skip connections, the low-frequency information is directly input into the reconstruction module to help the deep feature extraction module focus on the high-frequency information. Sub-pixel convolutional layers are used to upsample the low-frequency information and the high-frequency information. The upsampled low-frequency information and high-frequency information are aggregated to reconstruct the scratch-free old photo. A 4x super-resolution is performed to obtain a clear scratch-free old photo. S7. Establish a face detection network, input the clear, scratch-free old photo into the face detection network to detect faces in the photo, and obtain a cropped image of the local face region; S8. Establish a face restoration network, process the cropped image of the local face region separately to restore the face, and obtain the restored face image; S9. Perform histogram matching between the repaired face and the face detection position to obtain a clear, scratch-free photo with a complete face. S10. Establish an image coloring network to color a clear, scratch-free photo with a complete face, and obtain a color photo with a clear face and no scratches.

2. The method for comprehensive restoration of scratched old photographs as described in claim 1, characterized in that, The method for obtaining a complete Gaussian noise map is as follows: in, This is the state diagram at time t. and These are the drift and dispersion terms, respectively. For scratched old photos, It's Gaussian noise. and It is a positive parameter that is related to time.

3. The method for comprehensive restoration of scratched old photographs as described in claim 1, characterized in that, Methods for building and training scratch detection networks include: A dataset of old scratched photos was established based on real old scratched photos and synthetic old scratched photos. The scratched areas of each old photo in the scratched old photo dataset are labeled, and the scratched old photos and the labeled photos to be segmented are matched one by one. The image processing network is trained to obtain the scratch detection results. Based on the scratch detection results, the difference between the detected scratch and the actual scratch is minimized using cross-entropy loss; Based on the difference, the objective function of the scratch detection network is obtained, and the establishment and training of the scratch detection network are completed.

4. The method for comprehensive restoration of scratched old photographs as described in claim 1, characterized in that, The methods for obtaining the scratch-free old photographs include: The scores of each of the pure noise maps are weighted and calculated to obtain the final score result; Based on the final score calculation result, the state diagram for the next stage is obtained; Based on the denoising method of the diffusion model, the state diagram is subjected to reverse iterative denoising processing to remove scratch information and generate semantic information that conforms to the global context, thereby obtaining the scratch-free old photo.

5. The method for comprehensive restoration of scratched old photographs as described in claim 1, characterized in that, The method for extracting shallow feature maps of the scratch-free old photo using a convolutional neural network is as follows: in, This is a shallow feature map; These are old photos without scratches; It is a convolutional neural network.

6. The method for comprehensive restoration of scratched old photographs as described in claim 1, characterized in that, The method for obtaining a deep feature map by performing deep feature modeling on the shallow feature map is as follows: in, For deep feature maps; This is a deep feature extraction module; This is a shallow feature map.

7. The method for comprehensive restoration of scratched old photographs as described in claim 1, characterized in that, The method for obtaining the aforementioned clear, scratch-free old photograph is as follows: in, For clear, scratch-free old photos; To rebuild the module; For deep feature maps; This is a shallow feature map.

Images