Two-stage damaged face image restoration method based on generative adversarial network

Abstract

The invention relates to the technical field of computer vision and digital image processing, and discloses a two-stage damaged face image restoration method based on a generative adversarial network, and the method comprises the following steps: inputting a damaged face image into a reconstruction network, the input feature map is processed through at least one self-adaptive harmonic purification convolution module built in the reconstruction network, and a purified feature map is generated; based on the purified feature map, the reconstruction network outputs a low-frequency basic image and a final degradation feature map; inputting the low-frequency basic image and the final degradation feature map into a detail generator to generate a high-frequency detail map; and fusing the low-frequency basic image and the high-frequency detail image to obtain a final restored image. According to the method, the adaptive harmonic purification convolution module is arranged in the reconstruction network, frequency domain analysis can be carried out on local image features in the initial stage of feature extraction, and the harmonic attention mask is adaptively generated according to the content.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and digital image processing, in particular to a two-stage damaged face image restoration method based on a generative adversarial network. BACKGROUND

[0002] Face images are key carriers for information transmission and identity recognition. Restoring low-quality or damaged face images, i.e., face image restoration, has important application value in the fields of security monitoring, historical image restoration, mobile device imaging enhancement, etc. However, due to the complexity of image degradation process, restoring a corresponding high-quality clear image from a low-quality observation image is a typical inverse problem with high uncertainty.

[0003] With the development of deep learning technology, methods based on convolutional neural networks, especially generative adversarial networks (GAN), have become the mainstream technology in the field of face image restoration. Such methods can generate images containing rich details and high realism by learning a large amount of data.

[0004] Although the existing technology has made certain progress, it still faces inherent technical bottlenecks when dealing with real-world damaged face images with complex unknown degradation. Existing end-to-end restoration models usually perform both the tasks of removing degradation and generating details in a single network, which leads to the network often being trapped in the contradiction between fidelity and realism during optimization. Excessive attention to pixel-level fidelity, i.e., similarity to the original high-quality image, will make the restoration result tend to be smooth and lose high-frequency texture details; while excessive pursuit of the realism of the generated image, it is easy to introduce artificial traces that do not conform to the original identity features, and even lead to distortion of identity information. In addition, the degradation mode of real-world images is mixed and diverse, such as blur, noise, compression artifacts, etc. In the existing network structure, these degradation information will be encoded into the network's feature map together with the inherent content information of the image (such as identity, structure). With the deepening of network layers, these mixed degradation features will be transmitted and diffused in the entire feature space, continuously interfering with subsequent feature learning and image reconstruction processes, making it difficult for the network to accurately separate and remove the degradation, thereby limiting the upper limit of the quality of the final restored image. SUMMARY

[0005] In view of the deficiencies of the prior art, the present application provides a two-stage damaged face image restoration method based on a generative adversarial network, which solves the problem that the existing face image restoration technology cannot simultaneously generate high-fidelity and realistic results when dealing with images containing unknown and mixed degradation.

[0006] To achieve the above object, the present application provides a two-stage damaged face image restoration method based on a generative adversarial network, which uses an innovative convolution operator to realize feature purification and low-frequency basic reconstruction in the first stage, and generates conditioned high-frequency details according to the analysis results of the first stage in the second stage, so as to obtain a repair result with high fidelity and high realism.

[0007] The present application provides a two-stage damaged face image restoration method based on a generative adversarial network, which includes the following steps: The feature purification step: input a damaged face image into a reconstruction network. The reconstruction network processes the input feature map in the network through at least one adaptive harmonic purification convolution module built-in to generate a purified feature map.

[0008] The basic image and degradation feature generation step: based on the purified feature map, the reconstruction network finally outputs a low-frequency basic image and a final degradation feature map through its network structure. The low-frequency basic image retains the main structure and identity information of the damaged face image, and the final degradation feature map represents the degradation mode of the input image.

[0009] The detail generation step: the low-frequency basic image and the final degradation feature map are input into a detail generator as conditions. The detail generator generates a high-frequency detail map according to the condition input.

[0010] The image fusion step: fuse the low-frequency basic image and the high-frequency detail map to obtain the final repair image.

[0011] Preferably, the adaptive harmonic purification convolution module processing the input feature map includes the following steps: Harmonic decomposition: apply two-dimensional discrete cosine transform to a local feature block in the input feature map to obtain a harmonic coefficient matrix representing the frequency domain information of the feature block; Mask generation: generate a harmonic attention mask with the same dimension as the harmonic coefficient matrix through a neural network according to the content of the harmonic coefficient matrix; Frequency domain purification: perform a preset operation on the harmonic attention mask and the harmonic coefficient matrix to adjust the frequency components of the harmonic coefficient matrix to obtain a purified harmonic coefficient matrix; Spatial reconstruction: apply inverse two-dimensional discrete cosine transform to the purified harmonic coefficient matrix to reconstruct it from the frequency domain to the spatial domain to obtain a purified feature block, and generate the purified feature map based on the purified feature block.

[0012] In one embodiment, the frequency domain purification is defined by the following equation: ; In the equation, is a harmonic attention mask; is a harmonic coefficient matrix; is a harmonic coefficient matrix; represents element-wise multiplication.

[0013] Preferably, the method further comprises a degradation-aware routing step, which comprises: generating a degradation feature map based on the harmonic attention mask generated by the adaptive harmonic purification convolution module in the mask generation step; and concatenating the degradation feature map and the purified feature map in the channel dimension to form an enhanced feature map containing content information and degradation information, and taking the enhanced feature map as the input of the next layer of the reconstruction network.

[0014] In one embodiment, in the base image and degradation feature generation step, the training of the reconstruction network is supervised by a joint loss function, which includes a pixel loss for constraining pixel consistency, a perception loss for constraining perceptual similarity, and an identity loss for constraining identity consistency.

[0015] Further, the joint loss function is defined by the following equation: ; In the equation, the joint loss function is ; is a pixel loss; is a perception loss; is an identity loss; , , is a preset weight coefficient.

[0016] Preferably, in the detail generation step, the input of the detail generator further includes a random noise vector, which is used to introduce diversity when generating the high-frequency detail map.

[0017] In one embodiment, in the detail generation step, the training of the detail generator is adversarial training with a discriminator under the condition of fixing the parameters of the reconstruction network.

[0018] Preferably, the reconstruction network has an encoder-decoder structure for down-sampling encoding and up-sampling decoding of the input feature.

[0019] In one embodiment, the image fusion step is specifically pixel-wise addition of the low-frequency base image and the high-frequency detail map.

[0020] The application provides a two-stage damaged face image restoration method based on a generative adversarial network. 1、The adaptive harmonic purification convolution module is arranged in the reconstruction network, the local image features are subjected to frequency domain analysis in the initial stage of feature extraction, and the harmonic attention mask is adaptively generated according to the content, the mechanism can specifically inhibit the specific frequency components related to image degradation, while retaining the structural frequency components related to identity features, so that the low-frequency basic image output in the first stage has higher fidelity, and a more reliable basis is provided for subsequent detail generation.

[0021] 2、The harmonic attention mask generated by the adaptive harmonic purification convolution module is converted into a degradation feature map by arranging the degradation perception routing step, and is input into the detail generator as a condition, the design enables the detail generator to obtain the degradation mode information of the original image, so that conditional and non-uniform detail generation is performed, which ensures that the generated high-frequency details have higher consistency with the local content and context of the image, avoids generating uniform or mismatched textures in different degradation areas, and improves the naturalness of the final repaired image.

[0022] 3、The two-stage decoupling architecture is adopted, the face restoration task is divided into low-frequency basic reconstruction in the first stage and high-frequency detail generation in the second stage, the joint loss function containing pixel loss, perception loss and identity loss is used for supervision in the first stage, and the focus is on restoring high-fidelity content, and the second stage adopts adversarial training, and focuses on generating high-fidelity details, and the decoupling optimization of the conflicting targets enables the method to effectively retain the original identity features while significantly improving the texture fidelity of the image, and finally obtain a repaired result with high fidelity and high fidelity. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 It is a method flowchart of the application; Figure 2 It is a reconstruction network structure diagram of the application; Figure 3 It is an internal structure diagram of the adaptive harmonic purification convolution module of the application; Figure 4 It is a degradation perception routing mechanism diagram of the application; Figure 5 It is a first-stage training flowchart of the application; Figure 6 It is a detail generator structure and condition input diagram of the application; Figure 7 It is a second-stage adversarial training flowchart of the application; Figure 8 Fig. 1 is a schematic diagram of an image fusion process according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0025] Reference is made to the accompanying drawings Figure 1 , Figure 1 Fig. 1 is a schematic diagram of an image fusion process according to an embodiment of the present application. The present application provides a two-stage damaged face image restoration method based on a generative adversarial network. The method can specifically include a feature purification step, a basic image and degradation feature generation step, a detail generation step, and an image fusion step.

[0026] In a specific embodiment, the execution flow of the method is as follows: First, a damaged face image is provided as original input data to a reconstruction network. In the feature purification step, the reconstruction network uses at least one adaptive harmonic purification convolution module set inside the network to process the feature map flowing through the network, generating a purified feature map. This processing process suppresses the degradation information existing in the features at the source of feature extraction.

[0027] Subsequently, in the basic image and degradation feature generation step, the reconstruction network, based on the purified feature map obtained by processing, outputs two results at the end of the processing flow through the overall network structure of the reconstruction network: a low-frequency basic image and a final degradation feature map. The low-frequency basic image retains the main structure and identity information of the input image, while the final degradation feature map represents the degradation mode existing in the input image.

[0028] Next, in the detail generation step, the low-frequency basic image and the final degradation feature map generated in the previous stage are provided as conditional inputs to a detail generator. The detail generator generates a high-frequency detail map through its internal network layers according to the two conditional inputs received.

[0029] Finally, in the image fusion step, the low-frequency basic image and the high-frequency detail map are fused to synthesize and output a final restoration image. The final restoration image integrates the faithful content generated in the first stage and the realistic details generated in the second stage.

[0030] Reference is made to the accompanying drawings Figure 2 , Figure 2is a schematic diagram of a reconstruction network structure according to an embodiment of the present application. In the feature purification step and the base image and degraded feature generation step, the reconstruction network used is a neural network with a symmetric encoder-decoder structure. The network receives a damaged face image as input and processes the image to generate a low-frequency base image and a final degraded feature map.

[0031] In a specific embodiment, the encoder-decoder structure consists of an encoder path, a decoder path, and a bottleneck layer connecting the two.

[0032] The encoder path consists of a plurality of encoder stages connected in sequence. Each encoder stage contains at least one adaptive harmonic purification convolution module and ends with a down-sampling operation. During the data flow through the encoder path, the spatial resolution of the feature map is gradually reduced by the down-sampling operation, while the number of channels of the feature map is correspondingly increased.

[0033] The decoder path also consists of a plurality of decoder stages connected in sequence. Each decoder stage starts with an up-sampling operation, followed by at least one adaptive harmonic purification convolution module. During the data flow through the decoder path, the spatial resolution of the feature map is gradually restored by the up-sampling operation, while the number of channels of the feature map is correspondingly reduced.

[0034] To supplement the detail information in the process of restoring the spatial resolution in the decoder path, the reconstruction network is provided with a skip connection. Each skip connection splices the feature map output by a specific stage in the encoder path with the feature map after the up-sampling operation in the decoder path, which has the same spatial resolution in the channel dimension. The spliced feature map is then input to the subsequent processing module of the decoder stage. This structure transmits the high-resolution feature information retained in the encoder path to the decoder path.

[0035] Finally, the feature map output by the last stage of the decoder path is processed by one or more final convolution layers to generate a low-frequency base image and a final degraded feature map, respectively Referring to the accompanying Figure 3 , Figure 3 is an internal structure and data flow schematic diagram of an adaptive harmonic purification convolution module according to an embodiment of the present application. The adaptive harmonic purification convolution module is a basic unit for performing feature purification in the reconstruction network, which replaces the standard convolution layer in the prior art.

[0036] In a specific embodiment, the adaptive harmonic purification convolution module receives an input feature map and processes it according to the following steps: First step, harmonic decomposition. For any local feature block in the input feature map, the dimension of the local feature block is: ; wherein, is the kernel size of the module; is the number of channels of the input feature map.

[0037] A two-dimensional discrete cosine transform (DCT) is applied to the local feature block to convert it from the spatial domain to the frequency domain, obtaining a harmonic coefficient matrix. Each element of the harmonic coefficient matrix corresponds to a specific frequency component.

[0038] Second step, mask generation. This step is performed by a mask generation network. First, a global average pooling (GAP) operation is applied to the harmonic coefficient matrix obtained in the previous step to extract a feature vector representing the global frequency domain distribution of the matrix. Then, the feature vector is input into the mask generation network, which is composed of at least one fully connected layer. The output of the network is then passed through a Sigmoid activation function to finally generate a harmonic attention mask with the same dimensions as the harmonic coefficient matrix. The values of each element in the mask are in the interval of 0 to 1.

[0039] Third step, frequency domain purification. The harmonic attention mask generated in the previous step is multiplied element by element with the harmonic coefficient matrix obtained in the first step. This operation uses the harmonic attention mask to weight and adjust each frequency component of the harmonic coefficient matrix to suppress frequency components related to image degradation and retain frequency components related to image content structure, thereby obtaining a purified harmonic coefficient matrix. The process is as follows: ; wherein, is the harmonic attention mask; is the harmonic coefficient matrix; is the harmonic coefficient matrix; represents element-wise multiplication.

[0040] Fourth step, spatial reconstruction. The purified harmonic coefficient matrix obtained in the previous step is applied to the inverse two-dimensional discrete cosine transform (IDCT) to reconstruct it from the frequency domain back to the spatial domain, obtaining a purified feature block. After completing the above steps, the adaptive harmonic purification convolution module performs a standard convolution operation on the purified feature block to extract features. By performing the above process on all positions of the input feature map, a purified feature map is finally generated.

[0041] Referring to the accompanying Figure 4 , Figure 4 is a schematic diagram of a degradation-aware routing mechanism according to an embodiment of the present application. The degradation-aware routing mechanism is set inside the reconstruction network and is used to utilize the harmonic attention mask generated by the adaptive harmonic purification convolution module to pass the degradation information of the image during the forward propagation of the network.

[0042] In one specific embodiment, the implementation process of the mechanism is as follows: First, when an adaptive harmonic purification convolution module completes the processing of an input feature map and outputs a purified feature map, the complete harmonic attention mask generated in the mask generation step inside the module, which has the same spatial dimensions and channel number as the input feature map, is also obtained. The complete harmonic attention mask records the frequency domain suppression information of the input feature map at each spatial position and each channel.

[0043] Then, the degradation-aware routing mechanism receives the complete harmonic attention mask and generates a degradation feature map through a pooling operation. In one specific embodiment, the pooling operation is implemented through a 1x1 convolution layer. The 1x1 convolution layer receives the complete harmonic attention mask as input and reduces the channel number from the channel number of the input feature map to a preset smaller value (for example, 1 channel), thereby generating a degradation feature map. The degradation feature map is consistent with the purified feature map in spatial dimensions.

[0044] Subsequently, the purified feature map output by the adaptive harmonic purification convolution module is spliced with the degradation feature map generated in the previous step in the channel dimension. The splicing operation generates an enhanced feature map. The enhanced feature map contains both feature information representing image content and feature information representing image degradation patterns in its channel dimension.

[0045] Finally, the enhanced feature map is input into the next convolution module or the next network stage in the reconstruction network. By repeatedly executing this degradation-aware routing mechanism at multiple stages of the reconstruction network, the network can refer to the degradation information at the current scale at each step of feature extraction and image reconstruction, thereby performing adaptive processing.

[0046] Referring to the accompanying drawings Figure 5 , Figure 5 is a schematic diagram of the first stage training process according to one embodiment of the present application. In the basic image and degradation feature generation step, the training process of the reconstruction network is completed by minimizing a preset joint loss function, and the process aims to make the low-frequency basic image output by the network consistent with the original high-quality face image in terms of pixels, perceptual features, and identity features.

[0047] In one specific embodiment, the joint loss function is composed of the weighted sum of three independent loss terms: a pixel loss term, a perceptual loss term, and an identity loss term. The joint loss function is defined by the following formula: ; In the formula, the joint loss function is ; is a pixel loss; is a perceptual loss; is an identity loss; , , is a preset weight coefficient.

[0048] The pixel loss is used to calculate the pixel-level difference between the low-frequency base image output by the reconstruction network and the corresponding original high-quality face image. In an embodiment, the loss is calculated using the L1 distance, which constrains the network to generate an image consistent with the target in overall structure and color by penalizing the sum of absolute differences between the two images at corresponding pixel positions.

[0049] The perceptual loss is used to calculate the similarity between the two images in the deep feature space. The calculation process includes: inputting the low-frequency base image and the original high-quality face image into a pre-trained feature extraction network (e.g., a VGG network) on a large image dataset, and extracting feature maps from multiple intermediate layers of the network. Then, the L2 distance (Euclidean distance) between the two sets of corresponding feature maps is calculated and summed as the final perceptual loss. This loss constrains the network to generate an image with similar semantic and style features to the target image.

[0050] The identity loss is used to maintain the identity consistency of the face image before and after repair. The calculation process includes: inputting the low-frequency base image and the original high-quality face image into a pre-trained face recognition network (e.g., an ArcFace network) to extract respective identity feature vectors. Then, the cosine similarity between the two identity feature vectors is calculated. The identity loss is defined as 1 minus the cosine similarity. This loss is specifically used to penalize any generated results that cause changes in identity information.

[0051] During training, the gradients of the joint loss function with respect to the parameters of the reconstruction network are calculated using the backpropagation algorithm, and an optimizer (e.g., the Adam optimizer) is used to update the network parameters according to the gradients until the network converges.

[0052] Referring to the accompanying drawings Figure 6 , Figure 6 is a structure and condition input diagram of a detail generator according to an embodiment of the present application. The detail generator is used to perform the detail generation step, which receives the low-frequency base image and the final degraded feature map from the first stage, as well as a random noise vector as input, and generates a high-frequency detail map based on these inputs.

[0053] In one specific embodiment, the main structure of the detail generator is composed of multiple sequentially connected residual blocks. The low-frequency base image is fed into the first residual block as the initial input, and the output of each residual block is used as the input of the next residual block in sequence.

[0054] To enable the final degraded feature map and the random noise vector to control the generation process of high-frequency details, this embodiment employs a conditional modulation mechanism. The final degraded feature map and the random noise vector are first input into a mapping network. In one embodiment, the mapping network is composed of multiple fully connected layers, which are used to convert the two conditional inputs into a set of modulation parameters.

[0055] Inside each residual block of the detail generator, the intermediate feature map being processed by the residual block is subjected to a normalization operation, such as instance normalization. Subsequently, the modulation parameters output from the mapping network are applied to the normalized feature map to perform an affine transformation. This operation adaptively adjusts the feature responses at each position in the network according to the input degraded information and random noise, thereby guiding the generation of detail textures that match the local degradation patterns and randomness.

[0056] After processing by all residual blocks, the final feature map is processed by a terminal convolutional layer and passed through an activation function (e.g., a Tanh function) to generate a high-frequency detail map. The numerical range of the high-frequency detail map is limited to a pre-set interval (e.g., 1 to 1), and its content represents the detail and texture information that should be added to the low-frequency base image.

[0057] Referring to the accompanying drawings Figure 7 , Figure 7 is a schematic diagram of the second-stage adversarial training process according to one embodiment of the present application. In the detail generation step, the training process of the detail generator is completed through adversarial training with a discriminator network under the condition that all parameters of the reconstruction network are fixed. This process aims to make the high-frequency detail map generated by the detail generator indistinguishable from real image details in terms of texture and structure.

[0058] In one specific embodiment, the discriminator is a convolutional neural network that receives an input pair and outputs a scalar value representing the probability that the input pair is real. The input pair is formed by concatenating a low-frequency base image and a high-frequency detail map in the channel dimension.

[0059] In each training iteration, the adversarial training process includes the following operations: First, real sample pairs and fake sample pairs are generated. A real sample pair is composed of a low-frequency base image output by the reconstruction network, and a real high-frequency detail map obtained by subtracting the low-frequency base image from the corresponding original high-quality face image pixel by pixel. A fake sample pair is composed of the same low-frequency base image, and a fake high-frequency detail map generated by the detail generator.

[0060] Second, the network parameters of the discriminator are updated. The real sample pairs and fake sample pairs are input into the discriminator respectively. The loss function of the discriminator is calculated, which aims to maximize the probability of the discriminator outputting a high score for the real sample pairs, and minimize the probability of the discriminator outputting a high score for the fake sample pairs. According to the gradient calculated by the loss function, the parameters of the discriminator are updated once.

[0061] Third, the network parameters of the detail generator are updated. Under the condition of keeping the parameters of the discriminator fixed, the fake sample pairs are input into the discriminator again. The loss function of the detail generator is calculated, which aims to maximize the probability of the discriminator outputting a high score for the fake sample pairs. This loss function (i.e. adversarial loss) drives the detail generator to adjust its parameters to generate high-frequency detail maps that can make the discriminator misjudge as real. According to the gradient calculated by the loss function, the parameters of the detail generator are updated once.

[0062] By alternately performing the above updating steps of the discriminator and the detail generator, the detail generator is made to have the ability to generate high-frequency detail maps with high realism until it converges.

[0063] Referring to the accompanying drawings Figure 8 , Figure 8 is a schematic diagram of an image fusion process according to an embodiment of the present application. The image fusion step is the last step of the method of the present application, and its function is to merge the low-frequency base image generated in the first stage and the high-frequency detail map generated in the second stage to form the final repaired image.

[0064] In a specific embodiment, the fusion operation is realized by pixel-by-pixel addition of the low-frequency base image and the high-frequency detail map. The low-frequency base image provides the main structure, color and identity information of the repair result, while the high-frequency detail map provides the texture details needed to enhance the realism. The addition operation superimposes the detail information onto the base image.

[0065] The process formula is: ; In the formula, is the final repaired image; is the low-frequency base image; is the high-frequency detail map.

[0066] The addition operation is performed independently for each color channel at each spatial position (x, y) of the image.

[0067] After the addition operation, the pixel values of the high-frequency detail map can be positive or negative, and the pixel values of the added result may exceed the preset display range (for example, for an 8-bit image, the range is [0, 255]). Therefore, the embodiment further includes a clipping processing step. This step checks each pixel value in , sets all pixel values less than 0 to 0, and sets all pixel values greater than 255 to 255, to ensure that the pixel values of the final output image are within the valid display range.

[0068] Referring to the accompanying Figure 1 The accompanying Figure 8 , the above embodiment fully demonstrates a two-stage damaged face image restoration method based on a generative adversarial network. The method divides the complex restoration task into two independent but interrelated stages through a decoupled architecture.

[0069] In the first stage, a reconstruction network with an encoder-decoder structure is used to process the input damaged face image. The core of the network is an internally integrated adaptive harmonic purification convolution module, which adjusts the features in the frequency domain by adaptive weighting, suppresses the degradation information while preserving the content structure, and generates a high-fidelity low-frequency base image. At the same time, this stage extracts and transmits the degradation mode information of the image through a degradation perception routing mechanism to form the final degradation feature map. The training of this stage is supervised by a joint loss function containing pixel, perception and identity losses to ensure the consistency of the output low-frequency base image with the original high-quality image in multiple dimensions.

[0070] In the second stage, a detail generator receives the low-frequency base image output by the first stage and the final degradation feature map as conditions, and combines a random noise vector to generate a high-frequency detail map. The training process of this stage uses an adversarial training mechanism to play a game with a discriminator network, so that the detail generator focuses on generating texture details with high realism.

[0071] Finally, by performing a pixel-by-pixel addition fusion operation on the high-fidelity low-frequency base image output by the first stage and the high-realism high-frequency detail map output by the second stage, the final repaired image is obtained. Through the cooperative work of the above steps, the embodiment realizes the effective restoration of the damaged face image.

Claims

1. A two-stage method for restoring damaged face images based on generative adversarial networks, characterized in that, Includes the following steps: S1. Input a damaged face image into a reconstruction network, and process the input feature map through at least one adaptive harmonic sanitization convolution module built into the reconstruction network to generate a sanitized feature map. S2. Based on the purified feature map, the reconstruction network outputs a low-frequency base image and a final degraded feature map; S3. Input the low-frequency base image and the final degradation feature map as conditions into a detail generator to generate a high-frequency detail map according to the condition input; S4. The low-frequency base image and the high-frequency detail image are fused to obtain the final repaired image.

2. The two-stage damaged face image restoration method based on generative adversarial networks according to claim 1, characterized in that, The steps of the adaptive harmonic sanitization convolution module in processing the input feature map include: Harmonic decomposition: Apply a two-dimensional discrete cosine transform to a local feature block in the input feature map to obtain a harmonic coefficient matrix; Mask generation: A harmonic attention mask is generated based on the contents of the harmonic coefficient matrix using a neural network; Frequency domain purification: The harmonic attention mask is operated on with the harmonic coefficient matrix to obtain a purified harmonic coefficient matrix; Spatial reconstruction: Apply inverse two-dimensional discrete cosine transform to the purified harmonic coefficient matrix to obtain a purified feature block, and generate the purified feature map based on the purified feature block.

3. The two-stage damaged face image restoration method based on generative adversarial networks according to claim 2, characterized in that, The frequency domain cleanup specifically involves applying the harmonic attention mask. With the harmonic coefficient matrix By performing element-wise multiplication, the purified harmonic coefficient matrix is ​​obtained. The process is described by the following formula: ； In the formula, For harmonic attention mask; This is the harmonic coefficient matrix; This is the harmonic coefficient matrix; This represents element-wise multiplication.

4. The two-stage damaged face image restoration method based on generative adversarial networks according to claim 1, characterized in that, The method further includes a degradation-aware routing step, which includes: generating a degradation feature map based on the harmonic attention mask generated by the adaptive harmonic sanitization convolution module in mask generation; and concatenating the degradation feature map with the sanitized feature map in the channel dimension to form an enhanced feature map, which is then used as the input to the next layer of the reconstructed network.

5. The two-stage damaged face image restoration method based on generative adversarial networks according to claim 1, characterized in that, In the basic image and degradation feature generation step, the training of the reconstruction network is supervised by a joint loss function that includes pixel loss, perceptual loss, and identity loss.

6. The two-stage damaged face image restoration method based on generative adversarial networks according to claim 5, characterized in that, The joint loss function Limited by the following formula: ； In the formula, the joint loss function is: ; For pixel loss; To perceive loss; Loss of identity; , , These are the preset weighting coefficients.

7. The two-stage damaged face image restoration method based on generative adversarial networks according to claim 1, characterized in that, In the detail generation step, the input to the detail generator also includes a random noise vector, which is used to improve the diversity of the high-frequency detail map generation.

8. The two-stage damaged face image restoration method based on generative adversarial networks according to claim 1, characterized in that, In the detail generation step, the detail generator is trained adversarially against a discriminator while the parameters of the reconstructed network are fixed.

9. The two-stage damaged face image restoration method based on generative adversarial networks according to claim 1, characterized in that, The reconstruction network has an encoder-decoder structure.

10. The two-stage damaged face image restoration method based on generative adversarial networks according to claim 1, characterized in that, The image fusion step specifically involves adding the low-frequency base image and the high-frequency detail image pixel by pixel.

Images