Face image super-resolution method and system based on facial structure maintaining distillation network

Abstract

The invention provides a facial image super-resolution method and system based on a facial structure maintaining distillation network, and belongs to the field of facial image super-resolution, and the method comprises the steps: collecting a low-resolution facial image, and carrying out the up-sampling of the low-resolution facial image, and obtaining a pseudo high-resolution image; constructing a shallow layer down-sampling residual module to obtain shallow layer features; constructing an efficient face maintenance distillation module, and inputting shallow layer features to carry out deep extraction and feature refining to obtain deep features; constructing a deep-layer up-sampling residual module, and carrying out step-by-step feature refining on the deep features to obtain refined face features; a face image reconstruction module is constructed to perform convolution operation to obtain a high-resolution face image, an overall loss function is constructed, network parameters are optimized to obtain a training model, and the high-resolution face image is updated; and constructing a face image super-resolution test module, and testing the test data according to the training model to obtain a qualified model. According to the method, the face image reconstruction quality is improved while the low calculation complexity is kept.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision face image super-resolution, and in particular to a face image super-resolution solution based on a facial structure preserving distillation network. Background Art

[0002] In recent years, deep learning-based image super-resolution (SR) reconstruction algorithms have made significant progress with the help of the powerful feature expression capabilities of convolutional neural networks. However, due to the limitations of storage and computing resources, the current mainstream face image super-resolution (FSR) techniques are mainly deployed on high-performance computing platforms equipped with graphics processing units or GPUs. This limitation of relying on high-performance computing platforms seriously restricts the widespread application of these super-resolution techniques on low-power platforms such as mobile devices (such as smartphones, car cameras, and embedded systems), thereby limiting their potential utility in practical applications. Given that the deployment of image super-resolution techniques on resource-constrained hardware has attracted increasing attention from the academic community, researchers have adopted a variety of strategies such as feature reuse, model search, and knowledge distillation to effectively reduce the storage space and computing resources required for the model. Although these techniques have made certain achievements in compressing models, they still face the problem of performance loss due to the reduction of information interaction within the network. In addition, when deployed on mobile devices, the relatively high demand for computing resources of these methods has raised new challenges in terms of memory usage efficiency and access latency. Besides, these methods have not yet demonstrated sufficient adaptability and optimization potential when processing face images with highly structured features.

[0003] As an important image restoration technique, FSR has shown great potential in multiple practical applications, including face detection, face recognition, facial expression recognition, and other computer vision tasks involving face images. Unlike Single-Image Super-Resolution (SISR), FSR focuses on restoring key facial structures, namely the shapes of facial components and facial contours. Although these key information only occupy a small part of the facial image, they are more difficult to restore under low-resolution conditions because they exhibit large pixel variations in local areas.

[0004] Currently, most FSR models use mean square error (MSE) or mean absolute error (MAE) to train deep convolutional neural networks. During the optimization process, MAE may cause some values ​​in the model parameters to tend to zero, resulting in a feature selection effect that makes the model more sparse. MSE, on the other hand, tends to evenly distribute differences, making smaller adjustments to parameters and weighting all pixels equally.

[0005] Therefore, finding a method that can both enhance the ability to restore key facial structure information and improve computational efficiency is a technical problem that needs to be urgently solved in this field. Summary of the invention

[0006] The present invention provides a facial image super-resolution solution based on a facial structure preserving distillation network, which is used to solve the defects of insufficient texture details and inaccurate structural information in practical applications on low-computing power devices in the prior art, and significantly improves the quality of facial images while maintaining low computational complexity.

[0007] The present invention provides a facial image super-resolution method based on a facial structure preserving distillation network, comprising the following steps:

[0008] S1. Collect low-resolution face image I LR , and for the low-resolution face image I LR Upsample to obtain a pseudo high-resolution image I LR↑ ;

[0009] S2, constructing a shallow downsampling residual module, transforming the pseudo high-resolution image I LR↑ Input to the shallow downsampling residual module for processing to obtain the shallow feature F S,3 ;

[0010] S3, construct an efficient face-preserving distillation module, and transform the shallow features F S,3 Input to the efficient face-preserving distillation module for depth extraction and feature refinement to obtain the deep feature F D ;

[0011] S4, build a deep upsampling residual module, and use the deep upsampling residual module to perform deep feature F D Gradually refine the features to obtain refined face features;

[0012] S5. Construct a facial image reconstruction module, perform convolution operations on the refined facial features, and obtain a high-resolution facial image I SR , and construct the overall loss function L based on the facial structure-preserving distillation network EFPDNet, optimize the network parameters of the facial structure-preserving distillation network, obtain the training model, and update the high-resolution face image I SR , until the upper limit of the number of training times based on the facial structure-preserving distillation network is reached;

[0013] S6. Construct a facial image super-resolution test module, test the test data according to the training model, and obtain a qualified model, wherein the qualified model is used to realize reconstruction of low-resolution facial images.

[0014] According to a facial image super-resolution method based on a facial structure preserving distillation network provided by the present invention, step S3 specifically includes:

[0015] S31. Construct multiple efficient face-preserving distillation modules; the efficient face-preserving distillation modules include several feature refining units, feature distillation units, and an efficient face-preserving attention mechanism, wherein each feature refining unit includes a point-by-point convolution and two 3×3 depth-separable convolutions, the efficient face-preserving attention mechanism includes a face attention branch and a convolution branch, and the face attention branch includes an hourglass module and a 3×3 depth-separable convolution;

[0016] S32, the shallow feature F S,3 Input into the feature refining unit for feature refinement to obtain multiple refined features

[0017] S33, use the feature distillation unit to refine multiple features Perform facial feature distillation to obtain distilled features And perform feature fusion, reuse features through point-by-point convolution to obtain fused features

[0018] S34, respectively use the convolution branch and the facial attention branch to fusion feature F fusion Process, multiply and add element by element, and get the output of the current efficient face-keeping attention mechanism;

[0019] S35. By stacking multiple facial structure-preserving distillation modules, we gradually learn the global facial attention weight parameters based on large kernel convolution, and obtain the deep feature F based on the global facial attention weight parameters of large kernel convolution and the output of the efficient facial preservation attention mechanism. D .

[0020] According to a facial image super-resolution method based on a facial structure preserving distillation network provided by the present invention, the convolution branch includes a 1×1 point-by-point convolution, a 5×5 depth-separable convolution, and a 5×5 depth-dilated convolution, and the calculation formula is as follows:

[0021]

[0022] in, represents the output features of the large kernel convolution decomposition module, PConv 1×1 (·) represents a point-wise convolution of size 1×1, DwConv 5×5 (·) represents a 5×5 depthwise separable convolution, DwDConv 5×5 (·) represents a 5×5 deep atrous convolution, F fusion Represents fusion features.

[0023] According to a facial image super-resolution method based on a facial structure-preserving distillation network provided by the present invention, the output calculation formula of the current efficient facial preservation attention mechanism is as follows:

[0024]

[0025] in, represents the output of the efficient face-preserving attention mechanism at the i-th index, represents the output of the efficient face-preserving attention mechanism under the i-1th index, β i represents the efficient face-preserving attention weight, σ represents the Sigmoid activation function, Represents the face attention branch operation.

[0026] According to a facial image super-resolution method based on a facial structure preserving distillation network provided by the present invention, the deep upsampling residual module includes 2 nearest neighbor difference operation units, 3 feature extractors, 1 hourglass module, a point-by-point convolution, a 3×3 depth-separable convolution and a Sigmoid activation function, wherein the feature extractor includes a reflection filling module, a point-by-point convolution and a 3×3 depth-separable convolution.

[0027] According to a facial image super-resolution method based on a facial structure preserving distillation network provided by the present invention, the overall loss function L based on the facial structure preserving distillation network is constructed. EFPDNet , specifically:

[0028] The target high-resolution image benchmark I HR and high-resolution face image I SR The image is preprocessed and the Sobel kernel function is used to calculate the horizontal gradient image and the vertical target high-resolution image reference image. The horizontal gradient image includes the target high-resolution image reference image I HR Horizontal gradient image and high-resolution face image I SR Horizontal gradient image The target high-resolution image reference image in the vertical direction includes the target high-resolution image reference image I HR The vertical gradient image and high-resolution face image I SR The vertical gradient image

[0029] The horizontal gradient image and the vertical target high-resolution image reference image are both gradient-blocked, and the variance of each gradient image block is calculated respectively;

[0030] According to the variance of the gradient image block, the variance of the horizontal gradient map in the channel dimension and the variance of the vertical gradient map in the channel dimension are determined, and the horizontal loss function and the vertical loss function are calculated to construct the loss function L based on the facial structure preservation distillation network. EFPDNet .

[0031] According to a facial image super-resolution method based on a facial structure preserving distillation network provided by the present invention, the loss function L based on the facial structure preserving distillation network EFPDNet The calculation formula is:

[0032] L EFPdNet =λ1L1+λ2L x +λ3L y ;

[0033]

[0034] Among them, L EFPDNet represents the loss function based on the facial structure preservation distillation network, λ1 represents the weight coefficient of pixel domain loss, λ2 represents the weight coefficient of horizontal direction loss, λ3 represents the weight coefficient of vertical direction loss, L1 represents the pixel domain loss function, L x Represents the horizontal loss function, L y represents the vertical loss function, represents the expectation of high-resolution face images, represents the horizontal gradient of the target high-resolution image reference map, Represents the horizontal gradient of the high-resolution face image, represents the vertical gradient of the target high-resolution image benchmark, represents the vertical gradient of the high-resolution face image, and ‖·‖1 represents the L1 norm.

[0035] The present invention also provides a facial image super-resolution system based on a facial structure preserving distillation network, which adopts the facial image super-resolution method as described above, and comprises:

[0036] An upsampling module configured to collect a low-resolution face image and upsample the low-resolution face image to obtain a pseudo high-resolution image;

[0037] A shallow downsampling residual module, configured to input the pseudo high-resolution image into the shallow downsampling residual module for processing to obtain shallow features;

[0038] An efficient face-preserving distillation module, configured to input the shallow features into the efficient face-preserving distillation module for deep extraction and feature refinement to obtain deep features;

[0039] A deep upsampling residual module is configured to use the deep upsampling residual module to gradually refine the deep features to obtain refined face features;

[0040] A facial image reconstruction module is configured to perform a convolution operation on the refined facial features to obtain a high-resolution facial image, construct an overall loss function based on a facial structure-preserving distillation network, optimize network parameters based on the facial structure-preserving distillation network, obtain a training model, and update the high-resolution facial image until the upper limit of the number of training times based on the facial structure-preserving distillation network is reached;

[0041] The face image super-resolution test module is configured to test the test data according to the training model to obtain a qualified model, and the qualified model is used to realize the reconstruction of the low-resolution face image.

[0042] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the facial image super-resolution methods described above is implemented.

[0043] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described facial image super-resolution methods.

[0044] The face image super-resolution solution based on the facial structure preserving distillation network provided by the present invention processes the image through a shallow downsampling residual module, so that the spatial dimension of the feature map is halved and the number of output channels is doubled. The hourglass module is used to help the network capture and fuse multi-scale features at different levels, thereby improving the perception of facial details and overall facial information. By learning the global facial attention weight parameters based on large kernel convolution, the interaction of the global structural information of the face image at the channel level is enhanced, and the key facial features can be effectively captured and maintained. While maintaining a low computational complexity, the quality of the face image is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 It is a flow chart of the facial image super-resolution method provided by the present invention;

[0047] Figure 2 It is a flowchart of the face image super-resolution method provided by the present invention;

[0048] Figure 3 It is a block diagram of an efficient face-preserving distillation module of the face image super-resolution method provided by the present invention;

[0049] Figure 4 It is a block diagram of an efficient face-preserving attention mechanism for a face image super-resolution method provided by the present invention;

[0050] Figure 5 This is a result diagram of an embodiment of the face image super-resolution method provided by the present invention;

[0051] Figure 6 is a block diagram of a facial image super-resolution system provided by the present invention;

[0052] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in combination with the drawings and embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] like Figure 1 As shown, an embodiment of the present invention provides a face image super-resolution method based on a facial structure preserving distillation network, comprising the following steps:

[0055] S1. Collect low-resolution face image I LR , and for the low-resolution face image I LR Upsample to obtain a pseudo high-resolution image I LR↑ .

[0056] In one embodiment of the present invention, an upsampling module 101 is provided to use bicubic interpolation to convert the low-resolution image I LR Upsample to the target high-resolution image benchmark The same spatial dimension, resulting in a pseudo high-resolution image Where H represents the height of the image, and W represents the width of the image. is the field of real numbers. Figure 2 In the above figure, up means upsampling.

[0057] In the embodiment of the present invention, the public LFW face dataset is preferably used. The LFW (Labeled Faces in the Wild, LFW) dataset is a face image database compiled by the Computer Vision Laboratory of the University of Massachusetts, USA, which aims to study the face recognition problem in unrestricted situations. The LFW dataset includes 13,323 celebrity face images from the Internet, showing the characteristics of various natural scenes, including different shooting directions, expressions and lighting conditions. The LFW dataset contains 5,749 different celebrities, of which 1,680 have 2 or more face pictures, 4,069 have only 1 picture, and only 85 have more than 15 pictures. Each face picture is assigned a corresponding name ID and serial number for distinction. The dataset contains 6,000 pairs of face images, of which 3,000 pairs are two different photos of the same person, and the other 3,000 pairs are each a face photo of different people. Therefore, the LFW dataset has been widely used to evaluate the performance of face verification algorithms, especially in face recognition tasks in natural scenes. The present invention randomly selected 2110 images from the LFW dataset, including different shooting directions, expressions and lighting conditions, and used 50 and 60 images as the validation set and test set, respectively. In the data processing stage, the image is reshaped to a size of 112×112×3 as the HR image, i.e., the target high-resolution image reference image. Then, the high-resolution image is downsampled to a low-resolution image using the bicubic interpolation method. Under the conditions of four-fold and eight-fold downsampling factors, the size of the low-resolution image is 28×28×3 and 14×14×3, respectively.

[0058] S2, constructing a shallow downsampling residual module, transforming the pseudo high-resolution image I LR↑ Input to the shallow downsampling residual module for processing to obtain the shallow feature F S,3 .

[0059] like Figure 2As shown, in one embodiment of the present invention, the shallow downsampling residual module 102 includes a first downsampling unit, a second downsampling unit, a third downsampling unit, an hourglass module, a 1×1 point-by-point convolution, a 3×3 depth-separable convolution, and an activation function, wherein the first sampling unit, the second sampling unit, and the third sampling unit have the same structure, and the processing method of the shallow downsampling residual module is:

[0060] The pseudo high-resolution image I LR↑ After processing by the first sampling unit, the identity mapping feature F is obtained I ;

[0061] The pseudo high-resolution image I LR↑ After being processed by the second sampling unit and the third sampling unit, the rough feature F is obtained. C ;

[0062] The rough feature F C Pass it to the hourglass module and go through a 1×1 point-by-point convolution, a 3×3 depth-separable convolution, and a sigmoid activation function to obtain the attention weight α;

[0063] The rough feature F C Multiply element-wise with the attention weight α and add the multiplication result to the identity mapping feature F I The output feature F of the shallow downsampling residual module 102 is obtained by element-by-element addition S .

[0064] In one embodiment of the present invention, the first downsampling unit includes a reflection filling operation unit, an inverse point-by-point convolution and a depth-separable convolution with a convolution kernel of 3×3 and a stride of 2, then the rough feature F C The calculation formula is:

[0065] F C =H FE↓ (H FE (I LR↑ ))=H FE↓ (DwConv 3×3,s＝2 PConv 1×1 (δ(I LR↑ )))

[0066] Among them, F C represents the rough feature, δ(·) represents the reflection filling operation, PConv 1×1 (·) indicates point-wise convolution with a convolution kernel of size 1×1, DwConv 3×3,s＝2 (·) represents a depthwise separable convolution with a kernel size of 3×3 and a stride s of 2. FE (·) indicates the feature extraction operation without reducing the size of the feature map, H FE↓(·) represents the feature extraction operation that reduces the size of the feature map;

[0067] The structures of the second down-sampling unit and the third up-sampling unit are the same as that of the first down-sampling unit, and are not described again here.

[0068] In one embodiment of the present invention, the facial structure-preserving distillation network has three shallow downsampling residual modules (wherein, the shallow downsampling residual modules are preferably recommended to be set to 3, and the number of shallow downsampling residual modules ranges from 1 to 5), multiple shallow downsampling residual modules are stacked, and the pseudo high-resolution image I LR↑ Input multiple shallow downsampling residual modules for processing, such as Figure 2 In the above example, three shallow downsampling residual modules are set, which are respectively recorded as shallow downsampling residual module 1, shallow downsampling residual module 2, and shallow downsampling residual module 3, and the shallow feature F is obtained. S,3 .

[0069] The image is processed through a shallow downsampling residual module, the spatial dimension of the output feature map is halved and the number of output channels is doubled.

[0070] The present invention reduces the spatial resolution of the image through a shallow downsampling residual module, thereby reducing the memory and computing resources required for the model during processing; and enables the network to capture features of different scales, which is very helpful in understanding the local and global information of facial images.

[0071] like Figure 3 and Figure 4 As shown in S3, construct an efficient face-preserving distillation module to transform the shallow feature F S，3 Input to the efficient face-preserving distillation module for depth extraction and feature refinement to obtain the deep feature F D .

[0072] In one embodiment of the present invention, step S3 specifically includes:

[0073] S31, build multiple efficient face-preserving distillation modules, e.g. Figure 2 In the method, N efficient face-preserving distillation modules are set, which are respectively recorded as efficient face-preserving distillation module 1, efficient face-preserving distillation module 2...efficient face-preserving distillation module N (it is preferably recommended that N is set to 3, and the value range of N is 1-10); the efficient face-preserving distillation module includes a plurality of feature refinement units, a feature distillation unit and an efficient face-preserving attention mechanism, wherein each feature refinement unit includes a point-by-point convolution and two 3×3 depth-separable convolutions, the efficient face-preserving attention mechanism includes a face attention branch and a convolution branch, and the face attention branch includes an hourglass module and a 3×3 depth-separable convolution;

[0074] S32, the shallow feature F S,3 Input into the feature refining unit for feature refinement to obtain multiple refined features For example Figure 3 In the embodiment, three feature refining units are provided, which are respectively denoted as feature refining unit 1, feature refining unit 2, and feature refining unit 3 (wherein, the feature refining unit is preferably recommended to be set to 3, and the number of feature refining units ranges from 1 to 5).

[0075] S33, use the feature distillation unit to refine multiple features Perform facial feature distillation to obtain distilled features And perform feature fusion, reuse features through point-by-point convolution, and obtain the fusion feature F fusion ;

[0076] Specifically, the formula for feature distillation is as follows:

[0077]

[0078] Among them, FD1(·) represents the first distillation layer for generating facial distilled features, FD2(·) represents the second distillation layer for generating facial distilled features, FD3(·) represents the third distillation layer for generating facial distilled features, FD4(·) represents the fourth distillation layer for generating facial distilled features, FR1(·) represents the first refining layer for gradually refining facial rough features, FR2(·) represents the second refining layer for gradually refining facial rough features, FR3(·) represents the third refining layer for gradually refining facial rough features, and FR4(·) represents the fourth refining layer for gradually refining facial rough features. represents the first distillation characteristic, represents the second distillation characteristic, represents the third distillation characteristic, represents the fourth distillation characteristic, represents the first refined feature, represents the second refined feature, Represents the third refined feature, F S Represents the output of a shallow downsampling module.

[0079] In one embodiment of the present invention, the efficient face-preserving distillation module includes a first feature refining unit, a second feature refining unit, a third feature refining unit, a feature distillation unit and an efficient face-preserving attention mechanism. The processing method of the efficient face-preserving distillation module is:

[0080] After performing point-by-point convolution on the shallow features, the features are input into the feature distillation unit to obtain the first features;

[0081] The shallow features are input into the first feature refining unit, and after point-by-point convolution and feature distillation units, the second features are obtained;

[0082] The shallow features are processed by the first feature refining unit and the second feature refining unit, and then processed by point-by-point convolution and feature distillation units to obtain the third feature;

[0083] The shallow feature is processed by the first feature refining unit, the second feature refining unit and the third feature refining unit, and then a point-by-point convolution and a 3×3 depth-separable convolution are used to perform feature fusion (i.e., feature concatenation) with the first feature, the second feature and the third feature to obtain the fourth feature;

[0084] The fourth feature is reused using point-wise convolution and processed using an efficient face-preserving attention mechanism to obtain the fifth feature;

[0085] The fifth feature is added element-wise to the shallow features to obtain the output of the efficient face-preserving distillation module, i.e., the deep features.

[0086] Specifically, the processing method of the feature refining unit is:

[0087] Input the shallow features into point-by-point convolution to obtain the first convolution feature;

[0088] Add the shallow feature to the first convolution feature element by element to obtain the second convolution feature;

[0089] Use two 3×3 depth-separable convolutions to process the second convolution feature respectively to obtain the third convolution feature and the fourth convolution feature;

[0090] The second convolution feature, the third convolution feature, and the fourth convolution feature are added element by element to obtain the output of the feature refinement unit, i.e., the refined feature F D .

[0091] In one embodiment of the present invention, feature fusion is performed by feature splicing, and the formula for feature fusion is:

[0092]

[0093] Among them, F fusion represents fusion features, Concat(·) represents feature concatenation operation, PConv 1×1 (·) represents a 1×1 point-wise convolution.

[0094] S34, respectively use the convolution branch and the facial attention branch to fusion feature F fusion Process, multiply and add element by element, and get the output of the current efficient face-keeping attention mechanism;

[0095] In one embodiment of the present invention, the convolution branch includes a 1×1 point-by-point convolution, a 5×5 depth-wise separable convolution, and a 5×5 depth-wise atrous convolution, and the calculation formula is as follows:

[0096]

[0097] in, represents the output features of the large kernel convolution decomposition module, PConv 1×1 (·) represents a point-wise convolution of size 1×1, DwConv 5×5 (·) represents a 5×5 depthwise separable convolution, DwDConv 5×5 (·) represents a 5×5 deep atrous convolution, F fusion Represents fusion features.

[0098] In one embodiment of the present invention, the output calculation formula of the current efficient face-keeping attention mechanism is as follows:

[0099]

[0100] in, represents the output of the efficient face-preserving attention mechanism at the i-th index, represents the output of the efficient face-preserving attention mechanism at the i-1th index, represents the efficient face-preserving attention weight, σ represents the Sigmoid activation function, represents the facial attention branch operation, H i Indicates the number of input channels, W i Indicates the number of output channels.

[0101] In one embodiment of the present invention, in an efficient face-preserving attention mechanism, the K×K convolution is first decomposed into a point-by-point convolution, a (2T-1)×(2T-1) depth-wise separable convolution, and a (K / T)×(K / T) depth-wise dilated convolution, where T=3 is the size of the dilation rate and the size of the convolution kernel K is preferably 5.

[0102] In one embodiment of the present invention, the facial attention branch includes an hourglass module and a depthwise separable convolution with a stride of 1 and a convolution kernel of 3×3. The hourglass module includes a first depthwise separable convolution unit, a second depthwise separable convolution unit, a third depthwise separable convolution unit, and a fourth depthwise separable convolution unit, wherein the first depthwise separable convolution unit, the second depthwise separable convolution unit, the third depthwise separable convolution unit, and the fourth depthwise separable convolution unit have the same structure, which are all depthwise separable convolution units with a stride of 2 and a convolution kernel of 3×3. The processing method of the hourglass unit includes:

[0103] Inputting the input feature into the first depth-separable convolution unit for processing to obtain the first depth-separable convolution feature;

[0104] Using a second depth-wise separable convolution unit and a third depth-wise separable convolution unit to process the first depth-wise separable convolution feature respectively to obtain a second depth-wise separable convolution feature and a third depth-wise separable convolution feature;

[0105] Adding the second depth-wise separable convolutional feature and the third depth-wise separable convolutional feature element-by-element to obtain a fourth depth-wise separable convolutional feature;

[0106] The fourth depth-wise separable convolution feature is input into the fourth depth-wise separable convolution unit for processing to obtain the output of the hourglass module.

[0107] It is understandable that the hourglass module helps capture information at different scales and retains high-level semantic information through skip connections. The hourglass module is used to help the network capture and fuse multi-scale features at different levels, thereby improving the perception of facial details and overall facial information.

[0108] In one embodiment of the present invention, the processing method of the efficient face-keeping attention mechanism is:

[0109] The input features are subjected to a 1×1 point-wise convolution, a 5×5 depth-wise separable convolution, and a 5×5 depth-wise dilated convolution to obtain the first processed features;

[0110] The first processing feature is processed using the hourglass module, and after a depth-wise separable convolution with a step size of 1 and a convolution kernel of 3×3 and a Sigmoid activation function, it is element-wise multiplied with the first processing feature to obtain the second processing feature;

[0111] The input features and the second processed features are added element-wise to obtain the output of the efficient face-preserving attention mechanism.

[0112] S35. By stacking multiple facial structure-preserving distillation modules, we gradually learn the global facial attention weight parameters based on large kernel convolution, and obtain the deep feature F based on the global facial attention weight parameters of large kernel convolution and the output of the efficient facial preservation attention mechanism. D .

[0113] Specifically, the deep feature F D The calculation formula is as follows:

[0114]

[0115] in represents the Nth, N-1th, …, 1st efficient face-preserving distillation modules, N is the total number of efficient face-preserving distillation modules, and F D Represents the deep features of the overall output of N efficient face-preserving distillation modules.

[0116] The present invention enhances the interaction of global structural information of face images at the channel level by learning global facial attention weight parameters based on large kernel convolution, and can effectively capture and maintain key facial features.

[0117] S4, build a deep upsampling residual module, and use the deep upsampling residual module to perform deep feature F D The features are gradually refined to obtain refined facial features.

[0118] In one embodiment of the present invention, the deep upsampling module 104 includes two nearest neighbor difference operation units, three feature extractors, an hourglass module, a point-by-point convolution, a 3×3 depth-separable convolution and a Sigmoid activation function, wherein each feature extractor includes a reflection filling module, a 1×1 point-by-point convolution and a 3×3 depth-separable convolution, and the processing method of the deep upsampling module is:

[0119] The deep feature F D Input them into two nearest neighbor difference operation units respectively to obtain the first interpolation feature and the second interpolation feature;

[0120] Passing the first interpolation feature through a feature extractor to obtain a first extracted feature;

[0121] Passing the second interpolation feature through two feature extractors to obtain a second extracted feature;

[0122] The second extracted feature is passed through an hourglass module, a 1×1 point-wise convolution, a 3×3 depth-separable convolution, and a Sigmoid activation function to obtain a third extracted feature;

[0123] Multiply the third extracted feature by the second extracted feature element by element to obtain a fourth extracted feature;

[0124] The fourth extracted feature is added to the first extracted feature element by element to obtain the output of the deep upsampling module, that is, the refined facial feature.

[0125] The present invention gradually increases the spatial resolution of the feature map by constructing a deep upsampling residual module to restore the image detail information, and fuses the multi-scale features extracted in the downsampling module with the upsampled feature map. This fusion helps to retain more details and texture information in high-resolution images.

[0126] S5. Construct a facial image reconstruction module, perform convolution operations on the refined facial features, and obtain a high-resolution facial image I SR , and construct the overall loss function L based on the facial structure-preserving distillation network EFPDNet , optimize the network parameters of the facial structure-preserving distillation network, obtain the training model, and update the high-resolution face image I SR , until the upper limit of the number of training times based on the facial structure-preserving distillation network is reached.

[0127] Specifically, high-resolution face image I SR The calculation formula is as follows:

[0128]

[0129] Among them, I SR represents a high-resolution face image, H FSPDNet (·) represents the overall operation of the distillation network based on facial structure preservation, H DURB,1 (·) represents the output of the first deep upsampling residual module, H DURB,2 (·) represents the output of the second deep upsampling residual module, H DURB,3 (·) represents the output of the third deep upsampling residual module, represents the deep features output by the last efficient face-preserving distillation module, Conv 3×3 Represented as a 3×3 convolution.

[0130] Furthermore, we construct an overall loss function L based on the facial structure-preserving distillation network. EFPDNet , specifically:

[0131] The target high-resolution image benchmark I HR and high-resolution face image I SR The image is preprocessed and the Sobel kernel function is used to calculate the horizontal gradient image and the vertical target high-resolution image reference image. The horizontal gradient image includes the target high-resolution image reference image I HR Horizontal gradient image and high-resolution face image I SR Horizontal gradient image The target high-resolution image reference image in the vertical direction includes the target high-resolution image reference image I HR The vertical gradient image and high-resolution face image I SR The vertical gradient image

[0132] The horizontal gradient image and the vertical target high-resolution image reference image are both gradient-blocked, and the variance of each gradient image block is calculated respectively;

[0133] According to the variance of the gradient image block, the variance of the horizontal gradient map in the channel dimension and the variance of the vertical gradient map in the channel dimension are determined, and the horizontal loss function and the vertical loss function are calculated to construct the loss function L based on the facial structure preservation distillation network. EFPDNet .

[0134] Among them, when calculating the loss, the edge information extracted by the Sobel operator is used as the weight to weight the original pixel-level loss (L1 loss), so that the loss function pays more attention to the reconstruction quality of the edge information, prompting the network to pay more attention to the details of the edge during the training process.

[0135] In one embodiment of the present invention, the horizontal gradient image and the vertical target high-resolution image reference image are expanded into image blocks of size m×m, and the corresponding dimension size is obtained. Gradient image patch Among them, w represents the width of the gradient image block, h represents the height of the gradient image block, Represents the horizontal gradient image block of the target high-resolution image reference image, Represents the vertical gradient image patch of the target high-resolution image reference image, Represents the horizontal gradient image block of the high-resolution face image, Represents the vertical gradient image block of the high-resolution face image;

[0136] Calculate the variance of the expanded gradient image block in the channel dimension. The calculation formula is as follows:

[0137]

[0138] where v i represents the channel dimension variance of the i-th gradient image block, is the average value of the i-th gradient image block, G i ' ,j Represents the expanded i-th gradient image block.

[0139] Furthermore, the loss function L of the facial structure-preserving distillation network is EFPDNet The calculation formula is:

[0140] L EFPdNet =λ1L1+λ2L x +λ3L y ;

[0141]

[0142] Among them, L EFPDNet represents the loss function based on the facial structure preservation distillation network, λ1 represents the weight coefficient of pixel domain loss, λ2 represents the weight coefficient of horizontal direction loss, λ3 represents the weight coefficient of vertical direction loss, L1 represents the pixel domain loss function, L x Represents the horizontal loss function, L y Represents the vertical loss function, E ISR represents the expectation of high-resolution face images, represents the horizontal gradient of the target high-resolution image reference map, Represents the horizontal gradient of the high-resolution face image, represents the vertical gradient of the target high-resolution image benchmark, represents the vertical gradient of the high-resolution face image, and ‖·‖1 represents the L1 norm.

[0143] S6. Construct a facial image super-resolution test module, test the test data according to the trained model, and use it to reconstruct low-resolution facial images after obtaining a qualified model.

[0144] This paper effectively solves the problems of insufficient texture details and inaccurate structural information in the practical application of face image super-resolution on low-computing devices by using a facial structure-preserving distillation network, especially under low-parameter model conditions. This method significantly improves the quality of face images while maintaining low computational complexity, making it more suitable for application on resource-constrained platforms such as mobile devices.

[0145] In one embodiment of the present invention, the facial structure-preserving distillation network includes three shallow downsampling modules, N efficient face-preserving distillation modules and three deep upsampling modules (wherein the number of shallow downsampling modules must be consistent with the number of deep upsampling modules to ensure that the input and output image sizes are consistent.):

[0146] Remember low-resolution face image I LR The image size is H'×W'×3. After upsampling, the pseudo high-resolution image I LR↑ The image size is H×W×3, where H represents the height of the image and W represents the width of the image;

[0147] The pseudo high-resolution image I LR↑ After the first shallow downsampling module, the first shallow downsampling feature is obtained, the size is C represents the number of channels;

[0148] The first shallow downsampling feature passes through the second shallow downsampling module to obtain the second shallow downsampling feature, the size of which is

[0149] The second shallow downsampling feature passes through the third shallow downsampling module to obtain the third shallow downsampling feature, the size of which is

[0150] The third shallow downsampled feature is passed through N efficient face-preserving distillation modules to obtain a distilled feature with a size of

[0151] The distilled features are passed through the first deep upsampling module to obtain the first deep upsampling features, which are of size

[0152] The first deep upsampling feature is passed through the second deep upsampling module to obtain the second deep upsampling feature, the size of which is

[0153] The second deep upsampling features are passed through the third deep upsampling module and 3×3 ordinary convolution to obtain a high-resolution face image with a size of H×W×3.

[0154] like Figure 5 As shown, a specific embodiment is used for illustration. In this embodiment, the LFW dataset is used, which contains 202,599 face images of 10,177 people:

[0155] In the data processing stage, the original face images were randomly cropped to 128×128×3 and used as HR label images. The training set, validation set, and test set in the dataset randomly selected 2000, 50, and 50 images, respectively. This data setting can cover different characters, postures, and facial expressions, comprehensively consider the training needs in different scenarios, and simulate the performance of the model in various situations.

[0156] We randomly selected 2110 images from the LFW dataset, including different shooting orientations, expressions, and lighting conditions, and used 50 and 60 images as the validation set and test set, respectively.

[0157] In the data processing stage, the images are reshaped into 112 × 112 × 3 size as HR images.

[0158] The HR image is downsampled to the LR image using the bicubic interpolation method. Under the conditions of four times and eight times downsampling factors, the size of the LR image is 28×28×3 and 14×14×3 respectively. It should be noted that all training and testing are based on the brightness channel in the YCbCr color space, and 4 times and 8 times sampling multiples are used for training and testing.

[0159] The SR reconstruction results are evaluated using four evaluation indicators: Peak signal-to-noise ratio (PSNR), structural similarity (SSIM), feature similarity (Feture SIMilarity) and visual information fidelity (VIF) to verify the performance of SR reconstruction under the brightness channel.

[0160] In each iteration of training, the number of channels of the distillation structure and the number of channels of the face attention module in FSPDNet are set to 64, patch_size = 32 when the downsampling factor is ×4, and patch_size = 16 when the downsampling factor is ×8.

[0161] The network uses the loss function of the present invention and the Adam optimizer for model training, and its hyperparameters are λ1=0.9, λ2=0.05, λ3=0.05, β1=0.9, β2=0.999, and batch size batch_size=32. Data enhancement is performed on all training images, and they are randomly rotated 90 degrees, 180 degrees, 270 degrees, and flipped horizontally. The overall iteration cycle of FSPDNet is 500 epochs, and the initial learning rate is set to 1e-4. The experiment of the present invention is based on the deep learning library PyTorch, using Python3.8.5 as the operating environment, the system platform is Ubuntu 18.04, and the experiment is conducted on NVIDIA3090GPU.

[0162] The face image SR methods selected for comparison include: Bicubic, EDSR, SAN, MTC, DIDnet, SPARNet, SMSR and BSRN. Bicubic is a classic image interpolation algorithm; EDSR is a classic deep residual network SR method; SAN is a SR method based on a second-order attention mechanism; MTC is a FSR method based on a multi-view texture compensation network; DIDnet is a FSR method based on a dual identity constraint network; SPARNet is a FSR method based on a spatial attention network; SMSR is a SR method based on a sparse mask network; BSRN is a SR method based on a blueprint separable residual network.

[0163] Table 1 shows the comparison results under the conditions of reconstruction magnification of 4 and 8 through the above four evaluation indicators, where HR represents the original high-resolution face image benchmark image.

[0164] Table 1 Comparison results between the present invention and seven face image SR methods

[0165]

[0166]

[0167] As can be seen from Table 1, compared with the other seven face image SR methods, FSPDNet (Facial structure preserving distillation network) has achieved significant improvement in the quality of facial image reconstruction compared with other lightweight methods.

[0168] like Figure 6 As shown, the present invention provides a face image super-resolution system based on a facial structure preserving distillation network, which adopts the above-mentioned face image super-resolution method and includes:

[0169] An upsampling module configured to collect a low-resolution face image and upsample the low-resolution face image to obtain a pseudo high-resolution image;

[0170] A shallow downsampling residual module, configured to input the pseudo high-resolution image into the shallow downsampling residual module for processing to obtain shallow features;

[0171] An efficient face-preserving distillation module, configured to input the shallow features into the efficient face-preserving distillation module for deep extraction and feature refinement to obtain deep features;

[0172] A deep upsampling residual module is configured to use the deep upsampling residual module to gradually refine the deep features to obtain refined face features;

[0173] A facial image reconstruction module is configured to perform a convolution operation on the refined facial features to obtain a high-resolution facial image, construct an overall loss function based on a facial structure-preserving distillation network, optimize network parameters based on the facial structure-preserving distillation network, obtain a training model, and update the high-resolution facial image until the upper limit of the number of training times based on the facial structure-preserving distillation network is reached;

[0174] The face image super-resolution test module is configured to test the test data according to the training model to obtain a qualified model, and the qualified model is used to realize the reconstruction of the low-resolution face image.

[0175] The facial image super-resolution device provided by the present invention is described below. The facial image super-resolution device described below and the facial image super-resolution method described above can be referenced to each other.

[0176] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the face image super-resolution method, the method comprising: collecting a low-resolution face image, and upsampling the low-resolution face image to obtain a pseudo high-resolution image; constructing a shallow downsampling residual module, inputting the pseudo high-resolution image into the shallow downsampling residual module for processing to obtain shallow features; constructing an efficient face-keeping distillation module, inputting the shallow features into the efficient face-keeping distillation module for deep extraction and feature refinement to obtain deep features; constructing a deep upsampling residual module, and using the deep upsampling residual module to process the deep The method comprises the following steps: stepwise refining features of the high-resolution features to obtain refined facial features; constructing a facial image reconstruction module, performing convolution operations on the refined facial features to obtain high-resolution facial images, constructing an overall loss function based on a facial structure-preserving distillation network, optimizing network parameters based on the facial structure-preserving distillation network, obtaining a training model, and updating the high-resolution facial images until the upper limit of the number of training times based on the facial structure-preserving distillation network is reached; constructing a facial image super-resolution testing module, testing the test data according to the training model, and obtaining a qualified model, wherein the qualified model is used to realize the reconstruction of low-resolution facial images.

[0177] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0178] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the facial image super-resolution method provided by the above methods, which includes: collecting a low-resolution facial image and upsampling the low-resolution facial image to obtain a pseudo high-resolution image; constructing a shallow downsampling residual module, inputting the pseudo high-resolution image into the shallow downsampling residual module for processing to obtain shallow features; constructing an efficient facial preservation distillation module, and inputting the shallow features into the efficient facial preservation distillation module for deep extraction and feature refinement. , obtain deep features; construct a deep upsampling residual module, use the deep upsampling residual module to gradually refine the deep features to obtain refined face features; construct a face image reconstruction module, perform convolution operations on the refined face features to obtain high-resolution face images, and construct an overall loss function based on the facial structure-preserving distillation network, optimize the network parameters based on the facial structure-preserving distillation network, obtain a training model, and update the high-resolution face image until the upper limit of the number of training times based on the facial structure-preserving distillation network is reached; construct a face image super-resolution test module, test the test data according to the training model, and obtain a qualified model, which is used to realize the reconstruction of low-resolution face images.

[0179] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the facial image super-resolution method provided by the above-mentioned methods, the method comprising: collecting a low-resolution facial image, and upsampling the low-resolution facial image to obtain a pseudo high-resolution image; constructing a shallow downsampling residual module, and inputting the pseudo high-resolution image into the shallow downsampling residual module for processing to obtain shallow features; constructing an efficient face-preserving distillation module, and inputting the shallow features into the efficient face-preserving distillation module for deep extraction and feature refinement to obtain deep features; constructing a deep upsampling module A facial image reconstruction module is constructed to perform convolution operations on the refined facial features to obtain high-resolution facial images, and an overall loss function based on a facial structure-preserving distillation network is constructed to optimize the network parameters based on the facial structure-preserving distillation network to obtain a training model, and the high-resolution facial images are updated until the upper limit of the number of training times based on the facial structure-preserving distillation network is reached; a facial image super-resolution test module is constructed to test the test data according to the training model to obtain a qualified model, and the qualified model is used to realize the reconstruction of low-resolution facial images.

[0180] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0181] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A facial image super-resolution method based on facial structure preserving distillation network, characterized in that: The following steps are involved: S1. Collect low-resolution face image I LR , and for the low-resolution face image I LR Upsample to obtain a pseudo high-resolution image I LR↑ ; S2, constructing a shallow downsampling residual module, transforming the pseudo high-resolution image I LR↑ Input to the shallow downsampling residual module for processing to obtain the shallow feature F S，3 ; S3, construct an efficient face-preserving distillation module, and transform the shallow features F S，3 Input to the efficient face-preserving distillation module for depth extraction and feature refinement to obtain the deep feature F D ; S4, build a deep upsampling residual module, and use the deep upsampling residual module to perform deep feature F D Gradually refine the features to obtain refined face features; S5. Construct a facial image reconstruction module, perform convolution operations on the refined facial features, and obtain a high-resolution facial image I SR , and construct the overall loss function L based on the facial structure-preserving distillation network EFPDNet , optimize the network parameters of the facial structure-preserving distillation network, obtain the training model, and update the high-resolution face image I SR , until the upper limit of the number of training times based on the facial structure-preserving distillation network is reached; S6. Construct a facial image super-resolution test module, test the test data according to the training model, and obtain a qualified model, wherein the qualified model is used to realize reconstruction of low-resolution facial images.

2. The facial image super-resolution method based on facial structure preserving distillation network according to claim 1, characterized in that: Step S3 specifically includes: S31. Construct multiple efficient face-preserving distillation modules; the efficient face-preserving distillation modules include several feature refining units, feature distillation units, and an efficient face-preserving attention mechanism, wherein each feature refining unit includes a point-by-point convolution and two 3×3 depth-separable convolutions, the efficient face-preserving attention mechanism includes a face attention branch and a convolution branch, and the face attention branch includes an hourglass module and a 3×3 depth-separable convolution; S32, the shallow feature F S，3 Input into the feature refining unit for feature refining to obtain multiple refined features S33, use the feature distillation unit to refine multiple features Perform facial feature distillation to obtain distilled features And perform feature fusion, reuse features through point-by-point convolution, and obtain the fusion feature F fusion ; S34, respectively use the convolution branch and the facial attention branch to fusion feature F fusion Process, multiply and add element by element, and get the output of the current efficient face-keeping attention mechanism; S35. By stacking multiple facial structure-preserving distillation modules, we gradually learn the global facial attention weight parameters based on large kernel convolution, and obtain the deep feature F based on the global facial attention weight parameters of large kernel convolution and the output of the efficient facial preservation attention mechanism. D .

3. The facial image super-resolution method based on facial structure preserving distillation network according to claim 2 is characterized in that: The convolution branch includes a 1×1 point-by-point convolution, a 5×5 depth-wise separable convolution, and a 5×5 depth-wise dilated convolution. The calculation formula is as follows: in, represents the output features of the large kernel convolution decomposition module, PConv 1×1 (·) represents a point-wise convolution of size 1×1, DwConv 5×5 (·) represents a 5×5 depthwise separable convolution, DwDConv 5×5 (·) represents a 5×5 deep atrous convolution, F fusion Represents fusion features.

4. The facial image super-resolution method based on facial structure preserving distillation network according to claim 3 is characterized in that: The output calculation formula of the current efficient face-preserving attention mechanism is as follows: in, represents the output of the efficient face-preserving attention mechanism at the i-th index, represents the output of the efficient face-preserving attention mechanism under the i-1th index, β i represents the efficient face-preserving attention weight, σ represents the Sigmoid activation function, Represents the face attention branch operation.

5. The facial image super-resolution method based on facial structure preserving distillation network according to claim 1, characterized in that: The deep upsampling residual module includes 2 nearest neighbor difference operation units, 3 feature extractors, 1 hourglass module, a point-by-point convolution, a 3×3 depth-separable convolution and a Sigmoid activation function, wherein the feature extractor includes a reflection filling module, a point-by-point convolution and a 3×3 depth-separable convolution.

6. The facial image super-resolution method based on facial structure preserving distillation network according to claim 1, characterized in that: The overall loss function L of the facial structure-preserving distillation network is constructed. EFPDNet , specifically: The target high-resolution image benchmark I HR and high-resolution face image I SR The image is preprocessed and the Sobel kernel function is used to calculate the horizontal gradient image and the vertical target high-resolution image reference image. The horizontal gradient image includes the target high-resolution image reference image I HR Horizontal gradient image and high-resolution face image I SR Horizontal gradient image The target high-resolution image reference image in the vertical direction includes the target high-resolution image reference image I HR Vertical gradient image and high-resolution face image I SR Vertical gradient image The horizontal gradient image and the vertical target high-resolution image reference image are both gradient-blocked, and the variance of each gradient image block is calculated respectively; According to the variance of the gradient image block, the variance of the horizontal gradient map in the channel dimension and the variance of the vertical gradient map in the channel dimension are determined, and the horizontal loss function and the vertical loss function are calculated to construct the loss function L based on the facial structure preservation distillation network. EFPDNet .

7. The facial image super-resolution method based on facial structure preserving distillation network according to claim 6, characterized in that: The overall loss function L based on the facial structure-preserving distillation network EFPDNet The calculation formula is: L EFPDNet =λ1L1+λ2L x +λ3L y ; Among them, L EFPDNet represents the overall loss function based on the facial structure-preserving distillation network, λ1 represents the weight coefficient of pixel domain loss, λ2 represents the weight coefficient of horizontal direction loss, λ3 represents the weight coefficient of vertical direction loss, L1 represents the pixel domain loss function, L x Represents the horizontal loss function, L y represents the vertical loss function, represents the expectation of high-resolution face images, represents the horizontal gradient of the target high-resolution image reference map, Represents the horizontal gradient of the high-resolution face image, represents the vertical gradient of the target high-resolution image benchmark, represents the vertical gradient of the high-resolution face image, and ||·||1 represents the L1 norm.

8. A face image super-resolution system based on a facial structure preserving distillation network, characterized in that: The facial image super-resolution method according to any one of claims 1 to 7 comprises: An upsampling module configured to collect a low-resolution face image and upsample the low-resolution face image to obtain a pseudo high-resolution image; A shallow downsampling residual module, configured to input the pseudo high-resolution image into the shallow downsampling residual module for processing to obtain shallow features; An efficient face-preserving distillation module, configured to input the shallow features into the efficient face-preserving distillation module for deep extraction and feature refinement to obtain deep features; A deep upsampling residual module is configured to use the deep upsampling residual module to gradually refine the deep features to obtain refined face features; A facial image reconstruction module is configured to perform a convolution operation on the refined facial features to obtain a high-resolution facial image, construct an overall loss function based on a facial structure-preserving distillation network, optimize network parameters based on the facial structure-preserving distillation network, obtain a training model, and update the high-resolution facial image until the upper limit of the number of training times based on the facial structure-preserving distillation network is reached; The face image super-resolution test module is configured to test the test data according to the training model to obtain a qualified model, and the qualified model is used to realize the reconstruction of the low-resolution face image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the facial image super-resolution method as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the facial image super-resolution method as claimed in any one of claims 1 to 7 is implemented.

Images