A face image super-resolution reconstruction method, system, device and medium

Abstract

The application discloses a face image super-resolution reconstruction method, system, device and medium, and relates to the technical field of computer vision.The method is based on a target face image super-resolution reconstruction model to realize image reconstruction.The method steps include: shallow feature extraction on a low-resolution face image to be reconstructed to obtain a first shallow feature image; deep feature extraction on the first shallow feature image to obtain a first deep feature image; and determination of a target reconstruction image according to the first shallow feature image and the first deep feature image.The method solves the problems that the existing face image super-resolution reconstruction method based on local image blocks and global face statistics cannot achieve consistent global structure and loses local details, resulting in poor reconstruction effect, and the existing face image super-resolution reconstruction method based on structure priori cannot generate a super-resolution face image without distortion.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method, system, device, and medium for super-resolution reconstruction of human face images. Background Technology

[0002] In recent years, with the rapid development of deep learning technology, super-resolution image reconstruction methods for face images have been extensively studied. Currently, existing face image super-resolution reconstruction methods can be roughly divided into three categories: face image super-resolution reconstruction methods based on local image patches (referred to as local methods), face image super-resolution reconstruction methods based on global face statistics (referred to as global methods), and face image super-resolution reconstruction methods based on both local image patches and global face statistics (referred to as hybrid methods).

[0003] Local methods first crop the face image into uniformly sized image patches, then restore the texture details of these local patches, and finally fuse the patches to obtain the reconstructed super-resolution face image. However, local methods neglect the global structural information of the face image, leading to inconsistencies in facial contours. Since face images are highly structured objects, using structured information for face image reconstruction is a good strategy, especially using stable global face structured information, which is essential for downstream tasks such as face detection, localization, and recognition. However, while global methods can better preserve the global structural information of the face image, their expressive power is insufficient due to the very high dimensionality of face images. They cannot accurately represent such high-dimensional face data, resulting in super-resolution face images reconstructed by these methods often resembling an "average face" and lacking crucial facial feature details. Hybrid methods combine the above two types of methods to reconstruct global contour information and local detail information, but due to their extremely high complexity and computational cost, researchers tend to use either local or global methods alone.

[0004] Convolutional Neural Networks (CNNs) are widely used in face image reconstruction due to their powerful local modeling capabilities. CNN-based face image super-resolution reconstruction methods can be divided into two main categories: CNN-based local methods and CNN-based global methods. Currently, there are two mainstream CNN-based global methods: global image-based face image super-resolution reconstruction and structure prior-based face image super-resolution reconstruction. CNN-based local methods divide the face image into blocks, using CNNs to infer missing high-frequency details within these blocks. However, this block-division operation destroys global face structure information, inevitably leading to blurred reconstruction results. Global image-based face image super-resolution reconstruction inputs the entire face image into a CNN to obtain global structure information. However, due to the limited receptive field of CNNs, they cannot handle long-range dependencies, making this method unsuitable for extracting global information from the input face image. Face image super-resolution reconstruction methods based on structural priors first determine facial structure information using the location information of facial components and additional facial prior information. Then, a face super-resolution reconstruction model captures global structural information based on the facial structure information. The face prior information and global structural information are then fused, and finally, a decoder is used to obtain the super-resolution face image. However, since the additional facial prior information requires an additional face detection model, this not only increases the cost and difficulty of training and inference for such methods, but may also lead to difficulties in fitting the face super-resolution reconstruction model and distortion of the generated super-resolution face image. Summary of the Invention

[0005] The technical problem this invention aims to solve is that existing face image super-resolution reconstruction methods based on local image patches and global face statistics suffer from inconsistent global structure and loss of local details, resulting in poor reconstruction effects. Furthermore, existing face image super-resolution reconstruction methods based on structural priors also exhibit distortion in the generated super-resolution face images. To address these problems, this invention provides a face image super-resolution reconstruction method, system, device, and medium.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0007] A face image super-resolution reconstruction method includes a target face image super-resolution reconstruction model, wherein the target face image super-resolution reconstruction model includes a first shallow feature extractor and a first deep feature extractor connected in sequence; the method includes the following steps:

[0008] Step S1: Obtain the low-resolution face image to be reconstructed, and input the low-resolution face image into the target face image super-resolution reconstruction model;

[0009] Step S2: Perform shallow feature extraction on the low-resolution face image using the first shallow feature extractor to obtain a first shallow feature image;

[0010] Step S3: Extract deep features from the first shallow feature image using the first deep feature extractor to obtain the first deep feature image;

[0011] Step S4: Determine the target reconstructed image corresponding to the low-resolution face image to be reconstructed based on the first shallow feature image and the first deep feature image.

[0012] The beneficial effects of this invention are as follows: The method of this invention utilizes a pre-trained target face image super-resolution reconstruction model to sequentially extract shallow features, extract deep features, and reconstruct the target reconstructed image from the low-resolution face image to be reconstructed. This method has low computational cost and low complexity. By reconstructing face images using the method of this invention, high-resolution face images can be efficiently reconstructed, that is, low-resolution face images can be reconstructed into target reconstructed images, which facilitates image observation. Compared with existing face image super-resolution reconstruction methods, the method of this invention preserves the consistency of the global face structure and the fidelity of local detail restoration, while also preserving the high-frequency features of the face, thus avoiding distortion in the reconstructed image.

[0013] Based on the above technical solution, the present invention can be further improved as follows.

[0014] Furthermore, the target face image super-resolution reconstruction model is trained based on a pre-built initial face image super-resolution reconstruction network. The training process of the target face image super-resolution reconstruction model includes:

[0015] Step A1: Obtain multiple raw high-resolution face images, and the multiple raw high-resolution face images form a training set;

[0016] Step A2: Randomly initialize the model parameters of the initial face image super-resolution reconstruction network, and number each original high-resolution face image contained in the training set;

[0017] Step A3: Input each of the original high-resolution face images into the initial face image super-resolution reconstruction network, and perform image reconstruction on each of the original high-resolution face images through the initial face image super-resolution reconstruction network to obtain the first target reconstruction image corresponding to each of the original high-resolution face images. Calculate the first loss function value based on each of the original high-resolution face images and the first target reconstruction image corresponding to each of the original high-resolution face images.

[0018] Step A4: Based on the first loss function value, optimize the model parameters of the initial face image super-resolution reconstruction network using the backpropagation algorithm to obtain the optimized face super-resolution reconstruction model;

[0019] Step A5: Input each of the original high-resolution face images into the initial face image super-resolution reconstruction network, load the optimized face super-resolution reconstruction model through the initial face image super-resolution reconstruction network to obtain the face image super-resolution reconstruction network, and perform image reconstruction on each of the original high-resolution face images through the face image super-resolution reconstruction network to obtain the second target reconstruction image corresponding to each of the original high-resolution face images. Calculate the second loss function value based on each of the original high-resolution face images and the second target reconstruction image corresponding to each of the original high-resolution face images.

[0020] Step A6: Based on the second loss function value, optimize the model parameters of the face image super-resolution reconstruction network using the backpropagation algorithm to obtain the optimized face super-resolution reconstruction model;

[0021] Step A7: If the second loss function converges, the optimized face super-resolution reconstruction model when the second loss function converges is determined as the best face super-resolution reconstruction model. The best face super-resolution reconstruction model is loaded through the initial face image super-resolution reconstruction network to obtain the target face image super-resolution reconstruction model. If the second loss function does not converge, repeat steps A5 to A6 until the value of the second loss function converges.

[0022] The beneficial effects of adopting the above-mentioned further scheme are: face reconstruction using the above method can efficiently obtain high-resolution face images, while maintaining the consistency of facial structure and preserving local texture details, thereby obtaining better reconstruction results; and by performing secondary reconstruction on the first target reconstructed image obtained after the initial reconstruction, the present invention can further improve the reconstruction effect of low-resolution face images, thereby improving the reconstruction performance of the target face image super-resolution reconstruction model.

[0023] Furthermore, the first deep feature extractor includes multiple transmission connection blocks connected in sequence; step S3 specifically includes:

[0024] For each of the transport connection blocks in the first deep feature extractor, the transport connection block performs deep feature extraction on the image input to the transport connection block to obtain the first-level fusion feature image corresponding to the transport connection block. In each pair of adjacent transport connection blocks, the first-level fusion feature image corresponding to the transport connection block that is earlier is used as the input to the transport connection block that is later.

[0025] The first-level fusion feature image corresponding to the last transport connection block in the first deep feature extractor is subjected to convolution processing to obtain the first deep feature image;

[0026] Each of the transmission connection blocks includes a global feature extractor, a local feature extractor, and an aggregation module. The global feature extractor and the local feature extractor are connected in parallel and then connected to the aggregation module. The global feature extractor is used to extract global feature information of the image, the local feature extractor is used to extract local feature information of the image, and the aggregation module is used to fuse the global feature information and local feature information of the image.

[0027] For each transport connection block in the first deep feature extractor, the transport connection block performs deep feature extraction on the image input to the transport connection block to obtain a first-level fused feature image corresponding to the transport connection block, specifically including:

[0028] For the image of the input transmission connection block, the global feature extractor performs global feature extraction on the image to obtain the first global feature image corresponding to the image;

[0029] For the image of the input transmission connection block, the local feature extractor performs local feature extraction on the image to obtain the first local feature image corresponding to the image;

[0030] For the image input to the transmission connection block, the aggregation module adjusts the channel dimension of the first global feature image corresponding to the image to obtain the third global feature image corresponding to the image;

[0031] For the image of the input transmission connection block, the aggregation module adjusts the planar dimension of the first local feature image corresponding to the image to obtain the third local feature image corresponding to the image;

[0032] For the image of the input transmission connection block, the third global feature image and the third local feature image corresponding to the image are concatenated along the channel dimension to obtain the first fused feature image corresponding to the image;

[0033] For the input image of the transport connection block, a first-level fusion feature image corresponding to the transport connection block is obtained based on the global features and local representations in the first fusion feature image;

[0034] The global feature extractor includes multiple Transformer modules with the same structure, which are connected in sequence. Each Transformer module includes a spatial convolutional self-attention layer and a fully connected feedforward layer.

[0035] For the image input to the transport connection block, the global feature extractor performs global feature extraction on the image to obtain a first global feature image corresponding to the image, specifically including:

[0036] For each Transformer module, the Transformer module performs block processing on the image input to the Transformer module to obtain multiple image blocks;

[0037] For each Transformer module, spatial information of each image block is obtained through the spatial convolutional self-attention layer to obtain the first feature map corresponding to the Transformer module;

[0038] For each Transformer module, a non-linear transformation is performed on each image block through the fully connected feedforward layer to obtain the second feature map corresponding to the Transformer module;

[0039] For each Transformer module, the first feature map and the second feature map corresponding to the Transformer module are added element by element to obtain the third feature map corresponding to the Transformer module. In each pair of adjacent Transformer modules, the third feature map corresponding to the Transformer module that is earlier is used as the input of the Transformer module that is later.

[0040] The third feature map corresponding to the last Transformer module in the global feature extractor is convolved to obtain the first global feature image;

[0041] For each Transformer module, the spatial information of each image block is obtained through the spatial convolutional self-attention layer to obtain the first feature map corresponding to the Transformer module, specifically including:

[0042] Each dimension of each image block is embedded and encoded to obtain multiple encoded image blocks;

[0043] For each coded image block, the spatial information of the coded image block is obtained by pre-constructing a first convolutional neural network, a second convolutional neural network, and a third convolutional neural network, to obtain the first spatial image, the second spatial image, and the third spatial image corresponding to the coded image block;

[0044] For each coded image block, the first spatial image, the second spatial image and the third spatial image corresponding to the coded image block are flattened respectively to obtain the query weight matrix corresponding to the first spatial image, the key weight matrix corresponding to the second spatial image and the value weight matrix corresponding to the third spatial image.

[0045] For each coded image block, the query weight matrix and key weight matrix corresponding to the coded image block are flattened to obtain flattened query weight matrix and flattened key weight matrix. The flattened query weight matrix is ​​reshaped to obtain the target query weight matrix corresponding to the coded image block. The flattened key weight matrix is ​​reshaped to obtain the target key weight matrix corresponding to the coded image block.

[0046] For each encoded image block, a similarity matrix is ​​calculated based on the target query weight matrix and the target key weight matrix corresponding to the encoded image block. The similarity matrix corresponding to the encoded image block is then multiplied by the value weight matrix to obtain the aggregated feature image corresponding to the encoded image block.

[0047] The aggregated feature images corresponding to each encoded image block are folded to obtain the first combined feature image corresponding to the image input to the Transformer module;

[0048] The image input to the Transformer module is added element by element to the first combined feature image corresponding to the image input to the Transformer module to obtain the second combined feature image. The second combined feature image is then normalized to obtain the first feature map corresponding to the Transformer module.

[0049] For each Transformer module, a non-linear transformation is performed on each image patch through the fully connected feedforward layer to obtain the second feature map corresponding to the Transformer module, specifically including:

[0050] For each image block, a nonlinear transformation is performed on the image block through the fully connected feedforward layer to obtain the fourth feature map corresponding to the image block;

[0051] The fourth feature maps corresponding to each of the image blocks are aggregated to obtain the second feature map corresponding to the Transformer module.

[0052] The local feature extractor includes multiple CNN modules with identical structures, and each of the CNN modules is connected in sequence.

[0053] For the image input to the transport connection block, the local feature extractor performs local feature extraction on the image to obtain a first local feature image corresponding to the image, specifically including:

[0054] For each CNN module, local feature extraction is performed on the image input to the CNN module to obtain the fifth feature map corresponding to the CNN module. In each pair of adjacent CNN modules, the fifth feature map corresponding to the CNN module that is earlier is used as the input of the CNN module that is later.

[0055] The fifth feature map corresponding to the last CNN module in the transmission connection block is convolved to obtain the first local feature image corresponding to the image.

[0056] Each of the CNN modules includes multiple convolutional layers F with the same structure, and the convolutional layers F are connected in sequence;

[0057] For each CNN module, local feature extraction is performed on the image input to the CNN module to obtain the fifth feature map corresponding to the CNN module, specifically including:

[0058] For the image input to the CNN module, the image is convolved by the convolutional layer F in the CNN module to extract local feature information of the image, and obtain the local feature image A corresponding to the convolutional layer F. In each of two adjacent convolutional layers F, the local feature image A corresponding to the convolutional layer F that is earlier is used as the input of the convolutional layer F that is later.

[0059] A non-linear transformation is performed on the local feature image A corresponding to the last convolutional layer F in the CNN module to obtain the local feature image B corresponding to the convolutional layer F.

[0060] The local feature image A and the local feature image B corresponding to the last convolutional layer F in the CNN module are added element by element to obtain the fifth feature map corresponding to the CNN module.

[0061] For the image input to the transport connection block, the aggregation module adjusts the channel dimension of the first global feature image corresponding to the image to obtain the third global feature image corresponding to the image, specifically including:

[0062] By adjusting the channel dimension of the first global feature image corresponding to the image through a pre-constructed convolutional layer P, the channel dimension of the first global feature image corresponding to the image is made equal to the channel dimension of the first local feature image, thereby obtaining the third global feature image corresponding to the image.

[0063] For the image input to the transmission connection block, the aggregation module adjusts the planar dimension of the first local feature image corresponding to the image to obtain the third local feature image corresponding to the image, specifically including:

[0064] The planar dimension of the first local feature image corresponding to the image is adjusted by a pre-constructed convolutional layer Q, so that the planar dimension of the first local feature image corresponding to the image is equal to the planar dimension of the first global feature image, thereby obtaining the third local feature image corresponding to the image.

[0065] The beneficial effects of adopting the above-mentioned further scheme are as follows: global and local features in the image are obtained in parallel through global and local feature extractors. Specifically, the Transformer module is used to obtain global facial structure information, and the CNN module is used to obtain local facial detail information. This can simultaneously preserve both global and local facial structure information. Then, the outputs of the global and local feature extractors are aggregated through the aggregation module, thereby maintaining the consistency of the global facial structure and the fidelity of local detail restoration. This avoids the problems of local detail loss caused by using the global feature extractor alone and the inconsistency of the global structure caused by using the local feature extractor alone.

[0066] Further, in step A3, the initial face image super-resolution reconstruction network includes a first shallow feature extractor, a first deep feature extractor, a second shallow feature extractor, and a second deep feature extractor connected in sequence; the initial face image super-resolution reconstruction network is used to reconstruct each of the original high-resolution face images to obtain a first target reconstructed image corresponding to each of the original high-resolution face images, specifically including:

[0067] Step A3.1: For each of the original high-resolution face images, convert the original high-resolution face image into a low-resolution face image corresponding to the original high-resolution face image;

[0068] Step A3.2: For each of the original high-resolution face images, shallow feature extraction is performed on the low-resolution face image corresponding to the original high-resolution face image using the first shallow feature extractor to obtain the first shallow feature image corresponding to the original high-resolution face image.

[0069] Step A3.3: For each of the original high-resolution face images, perform deep feature extraction on the first shallow feature image corresponding to the original high-resolution face image using the first deep feature extractor to obtain the first deep feature image corresponding to the original high-resolution face image;

[0070] Step A3.4: For each of the original high-resolution face images, determine the first initial reconstructed image corresponding to the original high-resolution face image based on the first shallow feature image and the first deep feature image corresponding to the original high-resolution face image;

[0071] Step A3.5: For each of the original high-resolution face images, the downsampling module is used to perform downsampling processing on the first initial reconstructed image corresponding to the original high-resolution face image to obtain the downsampled feature image corresponding to the original high-resolution face image;

[0072] Step A3.6: For each of the original high-resolution face images, shallow feature extraction is performed on the downsampled feature image corresponding to the original high-resolution face image using the second shallow feature extractor to obtain the second shallow feature image corresponding to the original high-resolution face image;

[0073] Step A3.7: For each of the original high-resolution face images, perform deep feature extraction on the second shallow feature image corresponding to the original high-resolution face image using the second deep feature extractor to obtain the second deep feature image corresponding to the original high-resolution face image;

[0074] Step A3.8: For each of the original high-resolution face images, determine the first target reconstruction image corresponding to the original high-resolution face image based on the second shallow feature image and the second deep feature image corresponding to the original high-resolution face image.

[0075] The beneficial effects of adopting the above-mentioned further scheme are: the obtained first target reconstructed image is used for identity feature extraction, and the first loss function value and the second loss function value are calculated, which can improve the reconstruction effect of the face image.

[0076] Furthermore, step A3.8 specifically includes:

[0077] The second deep feature image corresponding to the low-resolution face image to be reconstructed is upsampled by a pre-constructed convolutional layer L to obtain the upsampled feature image corresponding to the low-resolution face image to be reconstructed.

[0078] By adjusting the channel dimension of the upsampled feature image corresponding to the low-resolution face image to be reconstructed through a pre-constructed convolutional layer M, the channel dimension of the upsampled feature image corresponding to the low-resolution face image to be reconstructed is made equal to the channel dimension of the second shallow feature image, thereby obtaining the third deep feature image corresponding to the low-resolution face image to be reconstructed.

[0079] The planar dimension of the second shallow feature image corresponding to the low-resolution face image to be reconstructed is adjusted by a pre-constructed convolutional layer N, so that the planar dimension of the second shallow feature image corresponding to the low-resolution face image to be reconstructed is equal to the planar dimension of the upsampled feature image, thereby obtaining the third shallow feature image corresponding to the low-resolution face image to be reconstructed.

[0080] The third shallow feature image and the third deep feature image corresponding to the low-resolution face image to be reconstructed are stitched together along the channel dimension to obtain the target fusion feature image corresponding to the low-resolution face image to be reconstructed.

[0081] By learning the global features and local representations in the target fusion feature image corresponding to the low-resolution face image to be reconstructed through a pre-constructed convolutional layer S, the first target reconstruction image corresponding to the low-resolution face image to be reconstructed is obtained.

[0082] The beneficial effect of adopting the above-mentioned further scheme is that the low-resolution first target reconstruction image obtained by adopting the above-mentioned dual-loop structure can be used to calculate the loss value with the low-resolution face image initially fed into the network, which can better improve the reconstruction effect of the model and enhance the reconstruction performance of the model.

[0083] Further, in step A3, the first loss function value is calculated based on each of the original high-resolution face images and the corresponding first target reconstructed image, specifically including:

[0084] For each of the original high-resolution face images, identity features are extracted from the original high-resolution face images to obtain the first identity feature image corresponding to the original high-resolution face image;

[0085] For each of the original high-resolution face images, identity features are extracted from the first initial reconstructed image corresponding to the original high-resolution face image to obtain the second identity feature image corresponding to the original high-resolution face image;

[0086] The first loss function value is calculated based on each of the original high-resolution face images and the corresponding low-resolution face images, the first initial reconstructed image, the first target reconstructed image, the first identity feature image, and the second identity feature image.

[0087] The beneficial effects of adopting the above-mentioned further scheme are as follows: Identity features are an inherent internal attribute feature of face images. The identity attributes of the same face image remain basically unchanged under different resolutions. This method extracts identity features from the original high-resolution face image and its corresponding first target reconstructed image, thereby minimizing the feature angle difference between the original high-resolution face image and its corresponding first target reconstructed image in the angle space. This makes the first target reconstructed image corresponding to the original high-resolution face image highly robust to feature perturbations, thereby constraining the generation of the initial reconstructed image and the target reconstructed image corresponding to the low-resolution face image to be reconstructed, and ensuring that the initial reconstructed image and the target reconstructed image have the same identity features as the low-resolution face image to be reconstructed.

[0088] Further, based on each of the original high-resolution face images and the corresponding low-resolution face images, the first initial reconstructed image, the first target reconstructed image, the first identity feature image, and the second identity feature image, the first loss function value is calculated, specifically including:

[0089] Based on each of the original high-resolution face images and the corresponding first initial reconstructed image, a reconstruction loss value is determined. This reconstruction loss value is determined using a first formula, which is:

[0090]

[0091] Among them, L rec The variable i represents the image index of the original high-resolution face image in the training set, N represents the total number of images in the training set, and x represents the reconstruction loss value. i This represents the first initial reconstructed image corresponding to the original high-resolution face image with index i in the training set, where y i This represents the original high-resolution face image with index i in the training set, where ||·||1 represents the 1-norm;

[0092] Based on the low-resolution face image corresponding to each of the original high-resolution face images and the reconstructed image of the first target, a dual-cycle loss value is determined. This dual-cycle loss value is determined using a second formula, which is:

[0093]

[0094] Among them, L dual The variable i represents the image index of the original high-resolution face image in the training set, N represents the total number of images in the training set, and p represents the double-loop loss value. i q represents the second identity feature image corresponding to the original high-resolution face image with index i in the training set. i Let i represent the low-resolution face image corresponding to the original high-resolution face image with index i in the training set, and ||·||2 represent the 2-norm;

[0095] Based on the first and second identity feature images corresponding to each of the original high-resolution face images, an identity loss value is determined. This identity loss value is determined using a third formula, which is:

[0096]

[0097] Among them, L idi The identity loss value is represented by variable i, where i represents the image index of the original high-resolution face image in the training set, N represents the total number of images in the training set, and S represents the identity loss value. i H represents the first identity feature image corresponding to the original high-resolution face image with index i in the training set. i This represents the second identity feature image corresponding to the original high-resolution face image with index i in the training set, where ||·||2 represents the 2-norm;

[0098] Based on the reconstruction loss value, the double-loop loss value, and the identity loss value, the first loss function value is determined. This first loss function value is determined using a fourth formula, which is:

[0099] L ml =L rec +L dual +αL idt

[0100] Among them, L ml This represents the value of the first loss function, where α is a variable used to control the weight of the identity loss value.

[0101] The beneficial effects of adopting the above-mentioned further scheme are as follows: the calculation method of the first loss function value is similar to the calculation method of the second loss function value. The first loss function value and the second loss function value are calculated based on the reconstruction loss value, the double loop loss value and the identity loss value to determine the optimal face super-resolution reconstruction model. Based on the optimal face super-resolution reconstruction model, the target face image super-resolution reconstruction model is determined to achieve the reconstruction of high-resolution face images.

[0102] To address the technical problems existing in the prior art, the present invention also provides a face image super-resolution reconstruction system, comprising:

[0103] An image input module is used to acquire a low-resolution face image to be reconstructed and input the low-resolution face image into the target face image super-resolution reconstruction model.

[0104] A shallow feature extraction module is used to perform shallow feature extraction on the low-resolution face image through the first shallow feature extractor to obtain a first shallow feature image;

[0105] The deep feature extraction module is used to extract deep features from the first shallow feature image using the first deep feature extractor to obtain the first deep feature image;

[0106] The reconstruction module is used to determine the target reconstructed image corresponding to the low-resolution face image to be reconstructed based on the first shallow feature image and the first deep feature image.

[0107] To address the technical problems existing in the prior art, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the face image super-resolution reconstruction method as described above.

[0108] To address the technical problems existing in the prior art, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the face image super-resolution reconstruction method as described above.

[0109] It should be noted that, in this invention, the images obtained after feature extraction all refer to feature maps. Attached Figure Description

[0110] Figure 1 This is a flowchart illustrating the face image super-resolution reconstruction method of the present invention;

[0111] Figure 2 This is a schematic diagram illustrating the implementation principle of the face image super-resolution reconstruction method in this invention;

[0112] Figure 3 This is a schematic diagram of the transmission connection block in this invention;

[0113] Figure 4 This is a schematic diagram illustrating the implementation principle of the spatial convolutional self-attention layer in this invention;

[0114] Figure 5Figure A shows the results of super-resolution reconstruction of image data in the FFHQ test dataset using the method of the present invention and existing face image super-resolution reconstruction methods;

[0115] Figure 6 Figure B shows the results of super-resolution reconstruction of image data in the FFHQ test dataset using the method of the present invention and existing face image super-resolution reconstruction methods;

[0116] Figure 7 This is a schematic diagram of the structure of the face image super-resolution reconstruction system in this invention;

[0117] Figure 8 This is a schematic diagram of the electronic device in this invention. Detailed Implementation

[0118] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0119] Example 1

[0120] To address the technical problems existing in the prior art, this embodiment provides a face image super-resolution reconstruction method (IET-FSR for short), such as... Figure 1 and Figure 2 As shown, the method includes:

[0121] A face image super-resolution reconstruction method includes a target face image super-resolution reconstruction model, wherein the target face image super-resolution reconstruction model includes a first shallow feature extractor and a first deep feature extractor connected in sequence, and the method includes the following steps:

[0122] Step S1: Obtain the low-resolution face image to be reconstructed, and input the low-resolution face image into the target face image super-resolution reconstruction model;

[0123] Step S2: Perform shallow feature extraction on the low-resolution face image using the first shallow feature extractor to obtain a first shallow feature image;

[0124] Step S3: Extract deep features from the first shallow feature image using the first deep feature extractor to obtain the first deep feature image;

[0125] Step S4: Determine the target reconstructed image corresponding to the low-resolution face image to be reconstructed based on the first shallow feature image and the first deep feature image.

[0126] The target face image super-resolution reconstruction model is trained based on a pre-constructed initial face image super-resolution reconstruction network. The training process of the target face image super-resolution reconstruction model includes:

[0127] Step A1: Obtain multiple raw high-resolution face images, and the multiple raw high-resolution face images form a training set;

[0128] Step A2: Randomly initialize the model parameters of the initial face image super-resolution reconstruction network, and number each original high-resolution face image contained in the training set;

[0129] Step A3: Input each of the original high-resolution face images into the initial face image super-resolution reconstruction network, and perform image reconstruction on each of the original high-resolution face images through the initial face image super-resolution reconstruction network to obtain the first target reconstruction image corresponding to each of the original high-resolution face images. Calculate the first loss function value based on each of the original high-resolution face images and the first target reconstruction image corresponding to each of the original high-resolution face images.

[0130] Step A4: Based on the first loss function value, optimize the model parameters of the initial face image super-resolution reconstruction network using the backpropagation algorithm to obtain the optimized face super-resolution reconstruction model;

[0131] Step A5: Input each of the original high-resolution face images into the initial face image super-resolution reconstruction network, load the optimized face super-resolution reconstruction model through the initial face image super-resolution reconstruction network to obtain the face image super-resolution reconstruction network, and perform image reconstruction on each of the original high-resolution face images through the face image super-resolution reconstruction network to obtain the second target reconstruction image corresponding to each of the original high-resolution face images. Calculate the second loss function value based on each of the original high-resolution face images and the second target reconstruction image corresponding to each of the original high-resolution face images.

[0132] Step A6: Based on the second loss function value, optimize the model parameters of the face image super-resolution reconstruction network using the backpropagation algorithm to obtain the optimized face super-resolution reconstruction model;

[0133] Step A7: If the second loss function converges, the optimized face super-resolution reconstruction model when the second loss function converges is determined as the best face super-resolution reconstruction model. The best face super-resolution reconstruction model is loaded through the initial face image super-resolution reconstruction network to obtain the target face image super-resolution reconstruction model. If the second loss function does not converge, repeat steps A5 to A6 until the value of the second loss function converges.

[0134] In step A3, the initial face image super-resolution reconstruction network includes a first shallow feature extractor, a first deep feature extractor, a second shallow feature extractor, and a second deep feature extractor connected in sequence. The initial face image super-resolution reconstruction network reconstructs each of the original high-resolution face images to obtain a first target reconstructed image corresponding to each of the original high-resolution face images. Specifically, this includes:

[0135] Step A3.1: For each of the original high-resolution face images, convert the original high-resolution face image into a low-resolution face image corresponding to the original high-resolution face image;

[0136] Step A3.2: For each of the original high-resolution face images, shallow feature extraction is performed on the low-resolution face image corresponding to the original high-resolution face image using the first shallow feature extractor to obtain the first shallow feature image corresponding to the original high-resolution face image.

[0137] Step A3.3: For each of the original high-resolution face images, perform deep feature extraction on the first shallow feature image corresponding to the original high-resolution face image using the first deep feature extractor to obtain the first deep feature image corresponding to the original high-resolution face image;

[0138] Step A3.4: For each of the original high-resolution face images, determine the first initial reconstructed image corresponding to the original high-resolution face image based on the first shallow feature image and the first deep feature image corresponding to the original high-resolution face image;

[0139] Step A3.5: For each of the original high-resolution face images, the downsampling module is used to perform downsampling processing on the first initial reconstructed image corresponding to the original high-resolution face image to obtain the downsampled feature image corresponding to the original high-resolution face image;

[0140] Step A3.6: For each of the original high-resolution face images, shallow feature extraction is performed on the downsampled feature image corresponding to the original high-resolution face image using the second shallow feature extractor to obtain the second shallow feature image corresponding to the original high-resolution face image;

[0141] Step A3.7: For each of the original high-resolution face images, perform deep feature extraction on the second shallow feature image corresponding to the original high-resolution face image using the second deep feature extractor to obtain the second deep feature image corresponding to the original high-resolution face image;

[0142] Step A3.8: For each of the original high-resolution face images, determine the first target reconstruction image corresponding to the original high-resolution face image based on the second shallow feature image and the second deep feature image corresponding to the original high-resolution face image.

[0143] In this embodiment, the downsampling module includes a 3×3 convolutional layer K.

[0144] In step A3, the first loss function value is calculated based on each of the original high-resolution face images and the first target reconstructed image corresponding to each of the original high-resolution face images, specifically including:

[0145] For each of the original high-resolution face images, identity features are extracted from the original high-resolution face images to obtain the first identity feature image corresponding to the original high-resolution face image;

[0146] For each of the original high-resolution face images, identity features are extracted from the first initial reconstructed image corresponding to the original high-resolution face image to obtain the second identity feature image corresponding to the original high-resolution face image;

[0147] The first loss function value is calculated based on each of the original high-resolution face images and the corresponding low-resolution face images, the first initial reconstructed image, the first target reconstructed image, the first identity feature image, and the second identity feature image.

[0148] Specifically, the extraction of identity features from the original high-resolution face image includes:

[0149] The original high-resolution face image is mapped to Euclidean space using a face recognition method to obtain a feature vector with a dimension of 128; in this embodiment, the face recognition method used is FaceNet.

[0150] The feature vector is subjected to L2 regularization to obtain the first identity feature image corresponding to the original high-resolution face image.

[0151] In this invention, the method for extracting identity features from the original high-resolution face image is similar to the method for extracting identity features from the first initial reconstructed image corresponding to the original high-resolution face image, and the similarities will not be repeated here.

[0152] Based on each of the original high-resolution face images and the corresponding low-resolution face images, the first initial reconstructed image, the first target reconstructed image, the first identity feature image, and the second identity feature image, the first loss function value is calculated, specifically including:

[0153] Based on each of the original high-resolution face images (i.e. Figure 2 The HR in the original high-resolution face image and the first initial reconstructed image corresponding to each of the original high-resolution face images (i.e., Figure 2 The reconstruction loss value is determined by the first formula, which is: (SR in the original text).

[0154]

[0155] Among them, L rec The variable i represents the image index of the original high-resolution face image in the training set, N represents the total number of images in the training set, and x represents the reconstruction loss value. i This represents the first initial reconstructed image corresponding to the original high-resolution face image with index i in the training set, where y i This represents the original high-resolution face image with index i in the training set, where ||·||1 represents the 1-norm;

[0156] Based on the low-resolution face images corresponding to each of the original high-resolution face images (i.e. Figure 2 LR) and the first target reconstructed image (i.e. Figure 2 The LR' in the formula is used to determine the double-cycle loss value, which is determined by a second formula, namely:

[0157]

[0158] Among them, L dual The variable i represents the image index of the original high-resolution face image in the training set, N represents the total number of images in the training set, and p represents the double-loop loss value. i q represents the second identity feature image corresponding to the original high-resolution face image with index i in the training set. iLet i represent the low-resolution face image corresponding to the original high-resolution face image with index i in the training set, and ||·||2 represent the 2-norm;

[0159] Based on the first and second identity feature images corresponding to each of the original high-resolution face images, an identity loss value is determined. This identity loss value is determined using a third formula, which is:

[0160]

[0161] Among them, L idt The identity loss value is represented by variable i, where i represents the image index of the original high-resolution face image in the training set, N represents the total number of images in the training set, and S represents the identity loss value. i H represents the first identity feature image corresponding to the original high-resolution face image with index i in the training set. i This represents the second identity feature image corresponding to the original high-resolution face image with index i in the training set, where ||·||2 represents the 2-norm;

[0162] Based on the reconstruction loss value, the double-loop loss value, and the identity loss value, the first loss function value is determined. This first loss function value is determined using a fourth formula, which is:

[0163] L ml =L rec +L dual +αL idt

[0164] Among them, L ml The first loss function value is represented by α, which is a variable used to control the weight of the identity loss value. In this embodiment, the value of α is 0.5.

[0165] In this invention, the method for calculating the first loss function value based on each of the original high-resolution face images and the first target reconstruction image corresponding to each of the original high-resolution face images is similar to the method for calculating the second loss function value based on each of the original high-resolution face images and the second target reconstruction image corresponding to each of the original high-resolution face images, and the similarities will not be repeated here.

[0166] Specifically, step A3.1 includes:

[0167] The original high-resolution face image is input into a pre-constructed convolutional layer A. The convolutional layer A downsamples the original high-resolution face image to obtain a low-resolution face image corresponding to the original high-resolution face image. In this embodiment, the size of the convolutional layer A is 3×3.

[0168] The structure of the first shallow feature extractor is the same as that of the second shallow feature extractor. Step S2 specifically includes:

[0169] The first shallow feature extractor extracts features from the low-resolution face image to obtain a first shallow feature image corresponding to the low-resolution face image to be reconstructed. The first shallow feature image contains rich facial structure information. In this embodiment, the first shallow feature extractor includes a 3×3 convolutional layer B.

[0170] In this invention, the implementation method of step S2 is similar to the implementation method of step A3.2, which uses the first shallow feature extractor to perform shallow feature extraction on the low-resolution face image corresponding to the original high-resolution face image to obtain the first shallow feature image corresponding to the original high-resolution face image, and the implementation method of step A3.6, which uses the second shallow feature extractor to perform shallow feature extraction on the downsampled feature image corresponding to the original high-resolution face image to obtain the second shallow feature image corresponding to the original high-resolution face image. The similarities will not be repeated here.

[0171] The first deep feature extractor has the same structure as the second deep feature extractor, and both the first and second deep feature extractors include multiple transfer connection blocks (TCBs) connected in sequence; step S3 specifically includes:

[0172] For each TCB in the first deep feature extractor, the TCB performs deep feature extraction on the image input to the TCB to obtain the first-level fusion feature image corresponding to the TCB. In each pair of adjacent TCBs, the first-level fusion feature image corresponding to the TCB in front is used as the input to the TCB in the back.

[0173] The first deep feature image is obtained by convolving the first-level fused feature image corresponding to the last TCB in the first deep feature extractor with a pre-constructed convolutional layer C; in this embodiment, the size of the convolutional layer C is 3×3.

[0174] like Figure 3 As shown, each TCB includes a global feature extractor, a local feature extractor, and an aggregation module. The global feature extractor and the local feature extractor are connected in parallel and then connected to the aggregation module. The global feature extractor is used to extract global feature information of the image, the local feature extractor is used to extract local feature information of the image, and the aggregation module is used to fuse the global feature information and local feature information of the image.

[0175] For each TCB in the first deep feature extractor, the TCB performs deep feature extraction on the image input to the TCB to obtain a first-level fused feature image corresponding to the TCB, specifically including:

[0176] For the input image of the TCB, the global feature extractor performs global feature extraction on the image to obtain the first global feature image corresponding to the image;

[0177] For the input image of the TCB, the local feature extractor performs local feature extraction on the image to obtain the first local feature image corresponding to the image;

[0178] For the input TCB image, the aggregation module adjusts the channel dimension of the first global feature image corresponding to the image to obtain the third global feature image corresponding to the image;

[0179] For the input TCB image, the aggregation module adjusts the planar dimension of the first local feature image corresponding to the image to obtain the third local feature image corresponding to the image;

[0180] For the input TCB image, the third global feature image and the third local feature image corresponding to the image are concatenated along the channel dimension to obtain the first fused feature image corresponding to the image;

[0181] For the input image of the TCB, the global features and local representations in the first fused feature image are learned through a pre-constructed convolutional layer H to obtain the first-level fused feature image corresponding to the TCB; in this embodiment, the size of the convolutional layer H is 1×1;

[0182] The global feature extractor includes multiple structurally identical Transformer modules, which are connected sequentially. Each Transformer module includes a spatial convolutional self-attention layer and a fully connected feedforward layer. The spatial convolutional self-attention layer includes an activation function layer. The spatial convolutional self-attention layer is used to acquire the spatial information of the image input to the Transformer module to obtain a first global feature image. The fully connected feedforward layer is used to perform a non-linear transformation on the image input to the Transformer module to obtain a third global feature image. In this embodiment, each TCB contains 10 Transformer modules.

[0183] For the image input to the TCB, the global feature extractor performs global feature extraction on the image to obtain the first global feature image corresponding to the image, specifically including:

[0184] For each Transformer module, the Transformer module performs block processing on the image input to the Transformer module to obtain multiple image blocks;

[0185] For each Transformer module, spatial information of each image block is obtained through the spatial convolutional self-attention layer to obtain the first feature map corresponding to the Transformer module;

[0186] For each Transformer module, a non-linear transformation is performed on each image block through the fully connected feedforward layer to obtain the second feature map corresponding to the Transformer module;

[0187] For each Transformer module, the first feature map and the second feature map corresponding to the Transformer module are added element by element to obtain the third feature map corresponding to the Transformer module. In each pair of adjacent Transformer modules, the third feature map corresponding to the Transformer module that is earlier is used as the input of the Transformer module that is later.

[0188] The first global feature image is obtained by convolving the third feature map corresponding to the last Transformer module in the global feature extractor with a pre-constructed convolutional layer D; in this embodiment, the size of the convolutional layer D is 1×1.

[0189] For each Transformer module, spatial information of each image patch is obtained through the spatial convolutional self-attention layer to obtain the first feature map corresponding to the Transformer module, such as... Figure 4 As shown, it specifically includes:

[0190] Embedding encoding for each dimension in each of the image blocks (i.e. Figure 3 (embedding location information in the image) to obtain multiple encoded image blocks;

[0191] For each coded image block, the spatial information of the coded image block is obtained through a pre-constructed first convolutional neural network, a second convolutional neural network, and a third convolutional neural network, to obtain the first spatial image, the second spatial image, and the third spatial image corresponding to the coded image block; in this embodiment, the convolution kernel size of the first convolutional neural network, the second convolutional neural network, and the third convolutional neural network is 3×3, the stride is 1, and the padding is 1;

[0192] For each coded image block, the first spatial image, the second spatial image and the third spatial image corresponding to the coded image block are flattened respectively to obtain the query weight matrix corresponding to the first spatial image, the key weight matrix corresponding to the second spatial image and the value weight matrix corresponding to the third spatial image.

[0193] For each coded image block, the query weight matrix and key weight matrix corresponding to the coded image block are flattened to obtain flattened query weight matrix and flattened key weight matrix. The flattened query weight matrix is ​​reshaped to obtain the target query weight matrix corresponding to the coded image block. The flattened key weight matrix is ​​reshaped to obtain the target key weight matrix corresponding to the coded image block.

[0194] For each encoded image block, a similarity matrix is ​​calculated based on the target query weight matrix and the target key weight matrix corresponding to the encoded image block. The similarity matrix corresponding to the encoded image block is then multiplied by the value weight matrix to obtain the aggregated feature image corresponding to the encoded image block.

[0195] The aggregated feature images corresponding to each encoded image block are folded to obtain the first combined feature image corresponding to the image input to the Transformer module;

[0196] The image input to the Transformer module is added element-wise to the first combined feature image corresponding to the image input to the Transformer module to obtain a second combined feature image. The second combined feature image is then normalized using activation function A to obtain the first feature map corresponding to the Transformer module. In this embodiment, the activation function A is the ReLU function.

[0197] For each Transformer module, a non-linear transformation is performed on each image patch through the fully connected feedforward layer to obtain the second feature map corresponding to the Transformer module, specifically including:

[0198] For each image block, a nonlinear transformation is performed on the image block through the fully connected feedforward layer to obtain the fourth feature map corresponding to the image block;

[0199] The fourth feature maps corresponding to each of the image blocks are aggregated to obtain the second feature map corresponding to the Transformer module.

[0200] The local feature extractor includes multiple CNN modules with identical structures, which are connected sequentially; in this embodiment, each TCB contains 3 CNN modules.

[0201] For the image input to the TCB, the local feature extractor performs local feature extraction on the image to obtain the first local feature image corresponding to the image, specifically including:

[0202] For each CNN module, local feature extraction is performed on the image input to the CNN module to obtain the fifth feature map corresponding to the CNN module. In each pair of adjacent CNN modules, the fifth feature map corresponding to the CNN module that is earlier is used as the input of the CNN module that is later.

[0203] The fifth feature map corresponding to the last CNN module in the TCB is convolved by a pre-constructed convolutional layer E to obtain the first local feature image corresponding to the image; in this embodiment, the size of the convolutional layer E is 3×3.

[0204] Each of the CNN modules includes multiple convolutional layers F with the same structure, and the convolutional layers F are connected in sequence; in this embodiment, the size of the convolutional layer F is 3×3;

[0205] For each CNN module, local feature extraction is performed on the image input to the CNN module to obtain the fifth feature map corresponding to the CNN module, specifically including:

[0206] For the image input to the CNN module, the image is convolved by the convolutional layer F in the CNN module to extract local feature information of the image, and obtain the local feature image A corresponding to the convolutional layer F. In each of two adjacent convolutional layers F, the local feature image A corresponding to the convolutional layer F that is earlier is used as the input of the convolutional layer F that is later.

[0207] A non-linear transformation is performed on the local feature image A corresponding to the last convolutional layer F in the CNN module to obtain the local feature image B corresponding to the convolutional layer F.

[0208] The local feature image A and the local feature image B corresponding to the last convolutional layer F in the CNN module are added element by element to obtain the fifth feature map corresponding to the CNN module.

[0209] For the input image of the TCB, the aggregation module adjusts the channel dimension of the first global feature image corresponding to the image to obtain the third global feature image corresponding to the image, specifically including:

[0210] The channel dimension of the first global feature image corresponding to the image is adjusted by a pre-constructed convolutional layer P, so that the channel dimension of the first global feature image corresponding to the image is equal to the channel dimension of the first local feature image, thereby obtaining the third global feature image corresponding to the image; in this embodiment, the size of the convolutional layer P is 1×1;

[0211] For the input image of the TCB, the aggregation module adjusts the planar dimension of the first local feature image corresponding to the image to obtain the third local feature image corresponding to the image, specifically including:

[0212] The planar dimension of the first local feature image corresponding to the image is adjusted by a pre-constructed convolutional layer Q, so that the planar dimension of the first local feature image corresponding to the image is equal to the planar dimension of the first global feature image, thereby obtaining the third local feature image corresponding to the image; in this embodiment, the size of the convolutional layer Q is 3×3.

[0213] In this invention, the implementation method of step S3 is similar to the implementation method of step A3.3, which uses the first deep feature extractor to extract deep features from the first shallow feature image corresponding to the original high-resolution face image to obtain the first deep feature image corresponding to the original high-resolution face image, and the implementation method of step A3.7, which uses the second deep feature extractor to extract deep features from the second shallow feature image corresponding to the original high-resolution face image to obtain the second deep feature image corresponding to the original high-resolution face image. The similarities will not be repeated here.

[0214] Step S4 is implemented through an image reconstruction module, which includes an upsampling module. In this embodiment, the upsampling module includes a 3×3 convolutional layer J. Step S4 specifically includes:

[0215] The first shallow feature image and the first deep feature image corresponding to the low-resolution face image to be reconstructed are added element by element to obtain the sixth feature image.

[0216] The upsampling module performs upsampling processing on the sixth feature map to obtain the initial reconstructed image corresponding to the low-resolution face image to be reconstructed.

[0217] In this invention, the method for implementing step S4 is similar to the method for implementing step A3.4, and the similarities will not be repeated here.

[0218] Specifically, step A3.8 includes:

[0219] The second deep feature image corresponding to the original high-resolution face image is upsampled by a pre-constructed convolutional layer L to obtain the upsampled feature image corresponding to the original high-resolution face image; in this embodiment, the size of the convolutional layer L is 1×1.

[0220] The channel dimension of the upsampled feature image corresponding to the original high-resolution face image is adjusted by a pre-constructed convolutional layer M, so that the channel dimension of the upsampled feature image corresponding to the original high-resolution face image is equal to the channel dimension of the second shallow feature image, thus obtaining the third deep feature image corresponding to the original high-resolution face image; in this embodiment, the size of the convolutional layer M is 1×1.

[0221] The planar dimension of the second shallow feature image corresponding to the original high-resolution face image is adjusted by a pre-constructed convolutional layer N, so that the planar dimension of the second shallow feature image corresponding to the original high-resolution face image is equal to the planar dimension of the upsampled feature image, thus obtaining the third shallow feature image corresponding to the original high-resolution face image; in this embodiment, the size of the convolutional layer N is 3×3.

[0222] The third shallow feature image and the third deep feature image corresponding to the original high-resolution face image are stitched together along the channel dimension to obtain the target fusion feature image corresponding to the original high-resolution face image.

[0223] By learning the global features and local representations in the target fusion feature image corresponding to the original high-resolution face image through a pre-constructed convolutional layer S, a first target reconstruction image corresponding to the original high-resolution face image is obtained. In this embodiment, the size of the convolutional layer S is 1×1.

[0224] In this invention, the network structure of the initial face image super-resolution reconstruction network is the same as that of the face image super-resolution reconstruction network. The method of reconstructing each of the original high-resolution face images through the initial face image super-resolution reconstruction network to obtain the first target reconstructed image corresponding to each of the original high-resolution face images is similar to the method of reconstructing each of the original high-resolution face images through the face image super-resolution reconstruction network to obtain the second target reconstructed image corresponding to each of the original high-resolution face images. The similarities will not be described again.

[0225] Simulation Experiment

[0226] This experiment was run on two Nvidia RTX 2080Ti GPUs in a PyTorch environment, using the ADAM optimizer with parameters β1 = 0.9, β2 = 0.999, and ∈ = 10. -8 Where β1 represents the exponential decay rate, β2 represents the exponential decay rate, and ∈ represents a constant. Two publicly available face image datasets—FFHQ and CelebA—are used. 3800 face images are selected from the FFHQ dataset as the training set, 100 as the test set, and 100 as the validation set from the CelebA dataset. The images selected from the FFHQ dataset are resized to 256×256, and the images selected from the CelebA dataset are resized to 216×176. The scaling factors chosen for the experiments are ×4 and ×8. During the experiments, all training samples were augmented by randomly rotating them by 90°, 180°, and 270° and flipping them horizontally to increase the complexity of the training samples and enhance the robustness of the method. The number of Transformer modules is 10, the number of CNN modules is 3, the number of training iterations is 130,000, and the batch size (epoch) is 16.

[0227] The face reconstruction results are evaluated using four metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Learned Perceptual Patch Similarity (LPIPS), and Mean Perceptual Score (MPS). PSNR and SSIM are standard evaluation metrics widely used in downstream visual tasks. LPIPS and MPS are perceptual metrics that measure image perceptual quality; a lower LPIPS value indicates higher perceptual similarity. MPS is calculated based on SSIM and LPIPS using the formula: MPS = 0.5 × (SSIM + (1 - LPIPS)).

[0228] To verify the practicality of this method, it is compared with existing methods, including face image super-resolution algorithms, general image super-resolution algorithms, and Transformer-based image processing algorithms. Face image super-resolution algorithms include DIDnet (Dual-Identity Constraint Dual-loop network), SPARNet (Spatial Attention Residual Network), and SISN (split-attention in split-attention network). General image super-resolution algorithms include SRFBN (super-resolution feedback network) and NLSN (Non-Local Sparse Attention). The Transformer-based image processing algorithm is IPT.

[0229] Table 1 shows the different parameters and their corresponding values ​​for different methods on the FFHQ dataset. The table lists the experimental results of different methods on the FFHQ test dataset under scaling factors of ×4 and ×8. As can be seen from Table 1, the data of this method is better than the aforementioned existing methods on all four evaluation metrics. CNN-based methods (such as SRBPN and NLSN) have better reconstruction results than Transformer-based methods (such as IPT). IPT did not achieve satisfactory results. The main reason for this result is that the Transformer framework lacks local representation capabilities, which leads to the loss of local texture information. CNN-based methods, due to the lack of identity attribute features, lose high-frequency information of face images, thus making their reconstruction results worse than this method.

[0230] Table 1 - Different parameters and their corresponding values ​​for different methods on the FFHQ dataset.

[0231]

[0232] Figure 5 and Figure 6 The images show the results of super-resolution reconstruction of images in the FFHQ test dataset using different methods at scaling factors of ×4 and ×8, respectively. The bounding boxes in the images represent key regions of the images. Figure 5 It can be seen that the reconstruction results of IPT are somewhat blurry, but it can still maintain the consistency of facial structure very well. For example, in Figure 5In the results image in the third row, the reconstruction results of SRFBN, SPARNet, and NLSN caused severe distortion in the eye region, while the IPT result maintained better structural information, indicating that the Transformer module can obtain global structural information. According to Figure 5 and Figure 6 As can be seen, compared with existing methods, this method can achieve excellent reconstruction results by aggregating the global representation capabilities of the Transformer module and the local representation capabilities of the CNN module, while maintaining the consistency of facial structure and preserving local texture details under the constraint of identity attributes.

[0233] Example 2

[0234] Based on the same principle as the aforementioned face image super-resolution reconstruction method, this embodiment provides a face image super-resolution reconstruction system, such as... Figure 7 As shown, it includes:

[0235] An image input module is used to acquire a low-resolution face image to be reconstructed and input the low-resolution face image into the target face image super-resolution reconstruction model.

[0236] A shallow feature extraction module is used to perform shallow feature extraction on the low-resolution face image through the first shallow feature extractor to obtain a first shallow feature image;

[0237] The deep feature extraction module is used to extract deep features from the first shallow feature image using the first deep feature extractor to obtain the first deep feature image;

[0238] The reconstruction module is used to determine the target reconstructed image corresponding to the low-resolution face image to be reconstructed based on the first shallow feature image and the first deep feature image.

[0239] Example 3

[0240] To address the technical problems existing in the prior art, this embodiment provides an electronic device, such as... Figure 8 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the face image super-resolution reconstruction method as described in Embodiment 1.

[0241] Example 4

[0242] To address the technical problems existing in the prior art, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the face image super-resolution reconstruction method as described in Embodiment 1.

[0243] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for super-resolution reconstruction of facial images, characterized in that, The method includes a target face image super-resolution reconstruction model, which comprises a first shallow feature extractor and a first deep feature extractor connected in sequence; the method includes the following steps: Step S1: Obtain the low-resolution face image to be reconstructed, and input the low-resolution face image into the target face image super-resolution reconstruction model; Step S2: Perform shallow feature extraction on the low-resolution face image using the first shallow feature extractor to obtain a first shallow feature image; Step S3: Extract deep features from the first shallow feature image using the first deep feature extractor to obtain the first deep feature image; Step S4: Determine the target reconstructed image corresponding to the low-resolution face image to be reconstructed based on the first shallow feature image and the first deep feature image; The target face image super-resolution reconstruction model is trained based on a pre-constructed initial face image super-resolution reconstruction network. The training process of the target face image super-resolution reconstruction model includes: Step A1: Obtain multiple raw high-resolution face images, and the multiple raw high-resolution face images form a training set; Step A2: Randomly initialize the model parameters of the initial face image super-resolution reconstruction network, and number each original high-resolution face image contained in the training set; Step A3: Input each of the original high-resolution face images into the initial face image super-resolution reconstruction network, and perform image reconstruction on each of the original high-resolution face images through the initial face image super-resolution reconstruction network to obtain the first target reconstruction image corresponding to each of the original high-resolution face images. Calculate the first loss function value based on each of the original high-resolution face images and the first target reconstruction image corresponding to each of the original high-resolution face images. Step A4: Based on the first loss function value, optimize the model parameters of the initial face image super-resolution reconstruction network using the backpropagation algorithm to obtain the optimized face super-resolution reconstruction model; Step A5: Input each of the original high-resolution face images into the initial face image super-resolution reconstruction network, load the optimized face super-resolution reconstruction model through the initial face image super-resolution reconstruction network to obtain the face image super-resolution reconstruction network, and perform image reconstruction on each of the original high-resolution face images through the face image super-resolution reconstruction network to obtain the second target reconstruction image corresponding to each of the original high-resolution face images. Calculate the second loss function value based on each of the original high-resolution face images and the second target reconstruction image corresponding to each of the original high-resolution face images. Step A6: Based on the second loss function value, optimize the model parameters of the face image super-resolution reconstruction network using the backpropagation algorithm to obtain the optimized face super-resolution reconstruction model; Step A7: If the second loss function converges, the optimized face super-resolution reconstruction model when the second loss function converges is determined as the best face super-resolution reconstruction model. The best face super-resolution reconstruction model is loaded through the initial face image super-resolution reconstruction network to obtain the target face image super-resolution reconstruction model. If the second loss function does not converge, repeat steps A5 to A6 until the value of the second loss function converges. The first deep feature extractor includes multiple transport connection blocks connected in sequence; step S3 specifically includes: For each of the transport connection blocks in the first deep feature extractor, the transport connection block performs deep feature extraction on the image input to the transport connection block to obtain the first-level fusion feature image corresponding to the transport connection block. In each pair of adjacent transport connection blocks, the first-level fusion feature image corresponding to the transport connection block that is earlier is used as the input to the transport connection block that is later. The first-level fusion feature image corresponding to the last transport connection block in the first deep feature extractor is subjected to convolution processing to obtain the first deep feature image; Each of the transmission connection blocks includes a global feature extractor, a local feature extractor, and an aggregation module. The global feature extractor and the local feature extractor are connected in parallel and then connected to the aggregation module. The global feature extractor is used to extract global feature information of the image, the local feature extractor is used to extract local feature information of the image, and the aggregation module is used to fuse the global feature information and local feature information of the image. The global feature extractor includes multiple Transformer modules with the same structure, which are connected in sequence. Each Transformer module includes a spatial convolutional self-attention layer and a fully connected feedforward layer. The local feature extractor includes multiple CNN modules with identical structures, and each of the CNN modules is connected in sequence. In step A3, the first loss function value is calculated based on each of the original high-resolution face images and the first target reconstructed image corresponding to each of the original high-resolution face images, specifically including: For each of the original high-resolution face images, identity features are extracted from the original high-resolution face images to obtain the first identity feature image corresponding to the original high-resolution face image; For each of the original high-resolution face images, identity features are extracted from the first initial reconstructed image corresponding to the original high-resolution face image to obtain the second identity feature image corresponding to the original high-resolution face image; The first loss function value is calculated based on each of the original high-resolution face images and the corresponding low-resolution face images, the first initial reconstructed image, the first target reconstructed image, the first identity feature image, and the second identity feature image. Based on each of the original high-resolution face images and the corresponding low-resolution face images, the first initial reconstructed image, the first target reconstructed image, the first identity feature image, and the second identity feature image, the first loss function value is calculated, specifically including: Based on each of the original high-resolution face images and the corresponding first initial reconstructed image, a reconstruction loss value is determined. This reconstruction loss value is determined using a first formula, which is: in, Represents the reconstruction loss value, variable This indicates the image number of the original high-resolution face image in the training set. This represents the total number of images in the training set. The training set index is indicated as The first initial reconstructed image corresponding to the original high-resolution face image, the The training set index is indicated as The original high-resolution face image, Represents the 1-norm; Based on the low-resolution face image corresponding to each of the original high-resolution face images and the reconstructed image of the first target, a dual-cycle loss value is determined. This dual-cycle loss value is determined using a second formula, which is: in, Represents the double-loop loss value, variable This indicates the image number of the original high-resolution face image in the training set. This represents the total number of images in the training set. The training set index is indicated as The second identity feature image corresponding to the original high-resolution face image. The training set index is indicated as The low-resolution face image corresponding to the original high-resolution face image. Represents the 2-norm; Based on the first and second identity feature images corresponding to each of the original high-resolution face images, an identity loss value is determined. This identity loss value is determined using a third formula, which is: in, Represents the identity loss value, variable This indicates the image number of the original high-resolution face image in the training set. This represents the total number of images in the training set. The training set index is indicated as The first identity feature image corresponding to the original high-resolution face image. The training set index is indicated as The second identity feature image corresponding to the original high-resolution face image. Represents the 2-norm; Based on the reconstruction loss value, the double-loop loss value, and the identity loss value, the first loss function value is determined. This first loss function value is determined using a fourth formula, which is: in, This represents the value of the first loss function. is a variable used to control the weight of the identity loss value.

2. The method according to claim 1, characterized in that, For each transport connection block in the first deep feature extractor, the transport connection block performs deep feature extraction on the image input to the transport connection block to obtain a first-level fused feature image corresponding to the transport connection block, specifically including: For the image of the input transmission connection block, the global feature extractor performs global feature extraction on the image to obtain the first global feature image corresponding to the image; For the image of the input transmission connection block, the local feature extractor performs local feature extraction on the image to obtain the first local feature image corresponding to the image; For the image input to the transmission connection block, the aggregation module adjusts the channel dimension of the first global feature image corresponding to the image to obtain the third global feature image corresponding to the image; For the image of the input transmission connection block, the aggregation module adjusts the planar dimension of the first local feature image corresponding to the image to obtain the third local feature image corresponding to the image; For the image of the input transmission connection block, the third global feature image and the third local feature image corresponding to the image are concatenated along the channel dimension to obtain the first fused feature image corresponding to the image; For the input image of the transport connection block, a first-level fusion feature image corresponding to the transport connection block is obtained based on the global features and local representations in the first fusion feature image; For the image input to the transport connection block, the global feature extractor performs global feature extraction on the image to obtain a first global feature image corresponding to the image, specifically including: For each Transformer module, the Transformer module performs block processing on the image input to the Transformer module to obtain multiple image blocks; For each Transformer module, spatial information of each image block is obtained through the spatial convolutional self-attention layer to obtain the first feature map corresponding to the Transformer module; For each Transformer module, a non-linear transformation is performed on each image block through the fully connected feedforward layer to obtain the second feature map corresponding to the Transformer module; For each Transformer module, the first feature map and the second feature map corresponding to the Transformer module are added element by element to obtain the third feature map corresponding to the Transformer module. In each pair of adjacent Transformer modules, the third feature map corresponding to the Transformer module that is earlier is used as the input of the Transformer module that is later. The third feature map corresponding to the last Transformer module in the global feature extractor is convolved to obtain the first global feature image; For each Transformer module, the spatial information of each image block is obtained through the spatial convolutional self-attention layer to obtain the first feature map corresponding to the Transformer module, specifically including: Each dimension of each image block is embedded and encoded to obtain multiple encoded image blocks; For each coded image block, the spatial information of the coded image block is obtained by pre-constructing a first convolutional neural network, a second convolutional neural network, and a third convolutional neural network, to obtain the first spatial image, the second spatial image, and the third spatial image corresponding to the coded image block; For each coded image block, the first spatial image, the second spatial image and the third spatial image corresponding to the coded image block are flattened respectively to obtain the query weight matrix corresponding to the first spatial image, the key weight matrix corresponding to the second spatial image and the value weight matrix corresponding to the third spatial image. For each coded image block, the query weight matrix and key weight matrix corresponding to the coded image block are flattened to obtain flattened query weight matrix and flattened key weight matrix. The flattened query weight matrix is ​​reshaped to obtain the target query weight matrix corresponding to the coded image block. The flattened key weight matrix is ​​reshaped to obtain the target key weight matrix corresponding to the coded image block. For each encoded image block, a similarity matrix is ​​calculated based on the target query weight matrix and the target key weight matrix corresponding to the encoded image block. The similarity matrix corresponding to the encoded image block is then multiplied by the value weight matrix to obtain the aggregated feature image corresponding to the encoded image block. The aggregated feature images corresponding to each encoded image block are folded to obtain the first combined feature image corresponding to the image input to the Transformer module; The image input to the Transformer module is added element by element to the first combined feature image corresponding to the image input to the Transformer module to obtain the second combined feature image. The second combined feature image is then normalized to obtain the first feature map corresponding to the Transformer module. For each Transformer module, a non-linear transformation is performed on each image patch through the fully connected feedforward layer to obtain the second feature map corresponding to the Transformer module, specifically including: For each image block, a nonlinear transformation is performed on the image block through the fully connected feedforward layer to obtain the fourth feature map corresponding to the image block; The fourth feature maps corresponding to each of the image blocks are aggregated to obtain the second feature map corresponding to the Transformer module. For the image input to the transport connection block, the local feature extractor performs local feature extraction on the image to obtain a first local feature image corresponding to the image, specifically including: For each CNN module, local feature extraction is performed on the image input to the CNN module to obtain the fifth feature map corresponding to the CNN module. In each pair of adjacent CNN modules, the fifth feature map corresponding to the CNN module that is earlier is used as the input of the CNN module that is later. The fifth feature map corresponding to the last CNN module in the transmission connection block is convolved to obtain the first local feature image corresponding to the image. Each of the CNN modules includes multiple convolutional layers F with the same structure, and the convolutional layers F are connected in sequence; For each CNN module, local feature extraction is performed on the image input to the CNN module to obtain the fifth feature map corresponding to the CNN module, specifically including: For the image input to the CNN module, the image is convolved by the convolutional layer F in the CNN module to extract local feature information of the image, and obtain the local feature image A corresponding to the convolutional layer F. In each of two adjacent convolutional layers F, the local feature image A corresponding to the convolutional layer F that is earlier is used as the input of the convolutional layer F that is later. A non-linear transformation is performed on the local feature image A corresponding to the last convolutional layer F in the CNN module to obtain the local feature image B corresponding to the convolutional layer F. The local feature image A and the local feature image B corresponding to the last convolutional layer F in the CNN module are added element by element to obtain the fifth feature map corresponding to the CNN module. For the image input to the transport connection block, the aggregation module adjusts the channel dimension of the first global feature image corresponding to the image to obtain the third global feature image corresponding to the image, specifically including: By adjusting the channel dimension of the first global feature image corresponding to the image through a pre-constructed convolutional layer P, the channel dimension of the first global feature image corresponding to the image is made equal to the channel dimension of the first local feature image, thereby obtaining the third global feature image corresponding to the image. For the image input to the transmission connection block, the aggregation module adjusts the planar dimension of the first local feature image corresponding to the image to obtain the third local feature image corresponding to the image, specifically including: The planar dimension of the first local feature image corresponding to the image is adjusted by a pre-constructed convolutional layer Q, so that the planar dimension of the first local feature image corresponding to the image is equal to the planar dimension of the first global feature image, thereby obtaining the third local feature image corresponding to the image.

3. The method according to claim 1, characterized in that, In step A3, the initial face image super-resolution reconstruction network includes a first shallow feature extractor, a first deep feature extractor, a second shallow feature extractor, and a second deep feature extractor connected in sequence. The initial face image super-resolution reconstruction network is used to reconstruct each of the original high-resolution face images to obtain a first target reconstructed image corresponding to each of the original high-resolution face images, specifically including: Step A3.1: For each of the original high-resolution face images, convert the original high-resolution face image into a low-resolution face image corresponding to the original high-resolution face image; Step A3.2: For each of the original high-resolution face images, shallow feature extraction is performed on the low-resolution face image corresponding to the original high-resolution face image using the first shallow feature extractor to obtain the first shallow feature image corresponding to the original high-resolution face image. Step A3.3: For each of the original high-resolution face images, perform deep feature extraction on the first shallow feature image corresponding to the original high-resolution face image using the first deep feature extractor to obtain the first deep feature image corresponding to the original high-resolution face image; Step A3.4: For each of the original high-resolution face images, determine the first initial reconstructed image corresponding to the original high-resolution face image based on the first shallow feature image and the first deep feature image corresponding to the original high-resolution face image; Step A3.5: For each of the original high-resolution face images, the first initial reconstructed image corresponding to the original high-resolution face image is downsampled using the downsampling module to obtain the downsampled feature image corresponding to the original high-resolution face image; Step A3.6: For each of the original high-resolution face images, shallow feature extraction is performed on the downsampled feature image corresponding to the original high-resolution face image using the second shallow feature extractor to obtain the second shallow feature image corresponding to the original high-resolution face image; Step A3.7: For each of the original high-resolution face images, perform deep feature extraction on the second shallow feature image corresponding to the original high-resolution face image using the second deep feature extractor to obtain the second deep feature image corresponding to the original high-resolution face image; Step A3.8: For each of the original high-resolution face images, determine the first target reconstruction image corresponding to the original high-resolution face image based on the second shallow feature image and the second deep feature image corresponding to the original high-resolution face image.

4. The method according to claim 3, characterized in that, Step A3.8 specifically includes: The second deep feature image corresponding to the original high-resolution face image is upsampled by a pre-constructed convolutional layer L to obtain the upsampled feature image corresponding to the original high-resolution face image. The channel dimension of the upsampled feature image corresponding to the original high-resolution face image is adjusted by pre-constructing a convolutional layer M, so that the channel dimension of the upsampled feature image corresponding to the original high-resolution face image is equal to the channel dimension of the second shallow feature image, thus obtaining the third deep feature image corresponding to the original high-resolution face image. The planar dimension of the second shallow feature image corresponding to the original high-resolution face image is adjusted by pre-constructing convolutional layer N, so that the planar dimension of the second shallow feature image corresponding to the original high-resolution face image is equal to the planar dimension of the upsampled feature image, thus obtaining the third shallow feature image corresponding to the original high-resolution face image. The third shallow feature image and the third deep feature image corresponding to the original high-resolution face image are stitched together along the channel dimension to obtain the target fusion feature image corresponding to the original high-resolution face image. By learning the global features and local representations in the target fusion feature image corresponding to the original high-resolution face image through a pre-constructed convolutional layer S, the first target reconstruction image corresponding to the original high-resolution face image is obtained.

5. A face image super-resolution reconstruction system, characterized in that, include: The image input module is used to acquire a low-resolution face image to be reconstructed and input the low-resolution face image into the target face image super-resolution reconstruction model. The shallow feature extraction module is used to perform shallow feature extraction on the low-resolution face image through the first shallow feature extractor to obtain the first shallow feature image; The deep feature extraction module is used to extract deep features from the first shallow feature image using a first deep feature extractor to obtain a first deep feature image; The reconstruction module is used to determine the target reconstructed image corresponding to the low-resolution face image to be reconstructed based on the first shallow feature image and the first deep feature image. The target face image super-resolution reconstruction model is trained based on a pre-constructed initial face image super-resolution reconstruction network. The training process of the target face image super-resolution reconstruction model includes: Step A1: Obtain multiple raw high-resolution face images, and the multiple raw high-resolution face images form a training set; Step A2: Randomly initialize the model parameters of the initial face image super-resolution reconstruction network, and number each original high-resolution face image contained in the training set; Step A3: Input each of the original high-resolution face images into the initial face image super-resolution reconstruction network, and perform image reconstruction on each of the original high-resolution face images through the initial face image super-resolution reconstruction network to obtain the first target reconstruction image corresponding to each of the original high-resolution face images. Calculate the first loss function value based on each of the original high-resolution face images and the first target reconstruction image corresponding to each of the original high-resolution face images. Step A4: Based on the first loss function value, optimize the model parameters of the initial face image super-resolution reconstruction network using the backpropagation algorithm to obtain the optimized face super-resolution reconstruction model; Step A5: Input each of the original high-resolution face images into the initial face image super-resolution reconstruction network, load the optimized face super-resolution reconstruction model through the initial face image super-resolution reconstruction network to obtain the face image super-resolution reconstruction network, and perform image reconstruction on each of the original high-resolution face images through the face image super-resolution reconstruction network to obtain the second target reconstruction image corresponding to each of the original high-resolution face images. Calculate the second loss function value based on each of the original high-resolution face images and the second target reconstruction image corresponding to each of the original high-resolution face images. Step A6: Based on the second loss function value, optimize the model parameters of the face image super-resolution reconstruction network using the backpropagation algorithm to obtain the optimized face super-resolution reconstruction model; Step A7: If the second loss function converges, the optimized face super-resolution reconstruction model when the second loss function converges is determined as the best face super-resolution reconstruction model. The best face super-resolution reconstruction model is loaded through the initial face image super-resolution reconstruction network to obtain the target face image super-resolution reconstruction model. If the second loss function does not converge, repeat steps A5 to A6 until the value of the second loss function converges. The first deep feature extractor includes multiple transport connection blocks connected in sequence; specifically, it includes: For each of the transport connection blocks in the first deep feature extractor, the transport connection block performs deep feature extraction on the image input to the transport connection block to obtain the first-level fusion feature image corresponding to the transport connection block. In each pair of adjacent transport connection blocks, the first-level fusion feature image corresponding to the transport connection block that is earlier is used as the input to the transport connection block that is later. The first-level fusion feature image corresponding to the last transport connection block in the first deep feature extractor is subjected to convolution processing to obtain the first deep feature image; Each of the transmission connection blocks includes a global feature extractor, a local feature extractor, and an aggregation module. The global feature extractor and the local feature extractor are connected in parallel and then connected to the aggregation module. The global feature extractor is used to extract global feature information of the image, the local feature extractor is used to extract local feature information of the image, and the aggregation module is used to fuse the global feature information and local feature information of the image. The global feature extractor includes multiple Transformer modules with the same structure, which are connected in sequence. Each Transformer module includes a spatial convolutional self-attention layer and a fully connected feedforward layer. The local feature extractor includes multiple CNN modules with identical structures, and each of the CNN modules is connected in sequence. In step A3, the first loss function value is calculated based on each of the original high-resolution face images and the first target reconstructed image corresponding to each of the original high-resolution face images, specifically including: For each of the original high-resolution face images, identity features are extracted from the original high-resolution face images to obtain the first identity feature image corresponding to the original high-resolution face image; For each of the original high-resolution face images, identity features are extracted from the first initial reconstructed image corresponding to the original high-resolution face image to obtain the second identity feature image corresponding to the original high-resolution face image; The first loss function value is calculated based on each of the original high-resolution face images and the corresponding low-resolution face images, the first initial reconstructed image, the first target reconstructed image, the first identity feature image, and the second identity feature image. Based on each of the original high-resolution face images and the corresponding low-resolution face images, the first initial reconstructed image, the first target reconstructed image, the first identity feature image, and the second identity feature image, the first loss function value is calculated, specifically including: Based on each of the original high-resolution face images and the corresponding first initial reconstructed image, a reconstruction loss value is determined. This reconstruction loss value is determined using a first formula, which is: in, Represents the reconstruction loss value, variable This indicates the image number of the original high-resolution face image in the training set. This represents the total number of images in the training set. The training set index is indicated as The first initial reconstructed image corresponding to the original high-resolution face image, the The training set index is indicated as The original high-resolution face image, Represents the 1-norm; Based on the low-resolution face image corresponding to each of the original high-resolution face images and the reconstructed image of the first target, a dual-cycle loss value is determined. This dual-cycle loss value is determined using a second formula, which is: in, Represents the double-loop loss value, variable This indicates the image number of the original high-resolution face image in the training set. This represents the total number of images in the training set. The training set index is indicated as The second identity feature image corresponding to the original high-resolution face image. The training set index is indicated as The low-resolution face image corresponding to the original high-resolution face image. Represents the 2-norm; Based on the first and second identity feature images corresponding to each of the original high-resolution face images, an identity loss value is determined. This identity loss value is determined using a third formula, which is: in, Represents the identity loss value, variable This indicates the image number of the original high-resolution face image in the training set. This represents the total number of images in the training set. The training set index is indicated as The first identity feature image corresponding to the original high-resolution face image. The training set index is indicated as The second identity feature image corresponding to the original high-resolution face image. Represents the 2-norm; Based on the reconstruction loss value, the double-loop loss value, and the identity loss value, the first loss function value is determined. This first loss function value is determined using a fourth formula, which is: in, This represents the value of the first loss function. is a variable used to control the weight of the identity loss value.

6. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the face image super-resolution reconstruction method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the face image super-resolution reconstruction method as described in any one of claims 1 to 4.

Images