Face image super-resolution method and system based on face structure preserving distillation network

Abstract

The present application provides a kind of based on face structure keeps distillation network face image super-resolution method and system, belong to face image super-resolution field, including: acquisition low-resolution face image and carry out upsampling, obtain pseudo high-resolution image;Constitute shallow layer down-sampling residual module, obtain shallow layer feature;Constitute efficient face keeping distillation module, input shallow layer feature carries out depth extraction and feature refining, obtain depth feature;Constitute deep layer up-sampling residual module, to depth feature carries out step-by-step refining feature, obtain refined face feature;Constitute face image reconstruction module and carry out convolution operation, obtain high-resolution face image, and construct overall loss function, optimize network parameter and obtain training model, update high-resolution face image;Constitute face image super-resolution test module, according to training model test data is tested, obtain qualified model.The present application improves the quality of face image reconstruction while maintaining lower computational complexity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision face image super-resolution, and particularly relates to a face image super-resolution scheme based on a face structure preserving distillation network. BACKGROUND

[0002] In recent years, deep learning-based image super-resolution (SR) reconstruction algorithms have made significant progress with the help of the powerful feature expression capability of convolutional neural networks. However, due to the limitations of storage and computing resources, the current mainstream face super-resolution (FSR) technology is mainly deployed on high-performance computing platforms equipped with a graphics processing unit or a central processing unit. This limitation of relying on high-performance computing platforms seriously hinders the wide application of these super-resolution technologies on low-power platforms such as mobile devices (such as smartphones, vehicle-mounted cameras, and embedded systems), thereby limiting their potential utility in practical applications. Given the increasing attention to the deployment of image super-resolution technology on resource-constrained hardware in the academic community, researchers have adopted various strategies such as feature reuse, model search, and knowledge distillation to effectively reduce the storage space and computing resources required by the model. Although these technologies have made certain achievements in compressing the model, they still face the problem of performance loss due to reduced information interaction within the network. In addition, when deployed on mobile devices, the relatively high demand for computing resources by these methods raises new challenges in terms of memory usage efficiency and access latency. In addition to this, these methods have not yet shown sufficient adaptability and optimization potential when dealing with face images with highly structured features.

[0003] FSR, as an important image restoration technology, has shown great potential in a variety of practical applications, including face detection, face recognition, facial expression recognition, and other computer vision tasks involving face images. Unlike single-image super-resolution (SISR), the focus of FSR is on restoring key facial structures, i.e., the shape of facial components and facial contours. Although these key information only occupies a small part of the face image, they are more difficult to recover under low-resolution conditions due to their large pixel changes within local regions.

[0004] Currently, most FSR models use Mean Square Error (MSE) or Mean Absolute Error (MAE) to train deep convolutional neural networks. During the optimization process, MAE can cause certain values in the model parameters to tend to zero, resulting in a feature selection effect that makes the model more sparse. While MSE tends to average the differences and make small adjustments to the parameters, it equally weights all pixels.

[0005] Therefore, finding a method that can both enhance the recovery ability of key facial structure information and improve computational efficiency is a technical problem that needs to be solved in the field. SUMMARY

[0006] The present application provides a face image super-resolution scheme based on a facial structure preserving distillation network, which solves the defects of insufficient texture details and inaccurate structure information in the actual application of low-power devices in the prior art, and significantly improves the quality of face images while maintaining low computational complexity.

[0007] The present application provides a face image super-resolution method based on a facial structure preserving distillation network, comprising the following steps:

[0008] S1, collecting a low-resolution face image I LR , and upsampling the low-resolution face image I LR to obtain a pseudo-high-resolution image I LR↑ ;

[0009] S2, constructing a shallow down-sampling residual module, inputting the pseudo-high-resolution image I LR↑ to the shallow down-sampling residual module for processing to obtain a shallow feature F S,3 ;

[0010] S3, constructing an efficient facial structure preserving distillation module, inputting the shallow feature F S,3 to the efficient facial structure preserving distillation module for deep extraction and feature refinement to obtain a deep feature F D ;

[0011] S4, constructing a deep up-sampling residual module, using the deep up-sampling residual module to gradually refine the deep feature F D to obtain a refined face feature;

[0012] S5, constructing a face image reconstruction module, performing convolution operation on the refined face feature to obtain a high-resolution face image I SR , and constructing an overall loss function L EFPDNet, and optimizing network parameters of the face structure preserving distillation network to obtain a training model, and updating the high-resolution face image I SR , until reaching an upper limit of a training number of the face structure preserving distillation network.

[0013] S6, constructing a face image super-resolution test module, testing test data according to the training model to obtain a qualified model, and the qualified model is used to realize reconstruction of the low-resolution face image.

[0014] According to the face image super-resolution method based on the face structure preserving distillation network, step S3 specifically comprises:

[0015] S31, constructing a plurality of efficient face preserving distillation modules; the efficient face preserving distillation module comprises a plurality of feature refining units, a feature distillation unit and an efficient face preserving attention mechanism, wherein each feature refining unit comprises a point-by-point convolution and two 3*3 depth separable convolutions, the efficient face preserving attention mechanism comprises a face attention branch and a convolution branch, the face attention branch comprises an hourglass module and a 3*3 depth separable convolution;

[0016] S32, inputting the shallow feature F S,3 to the feature refining unit for feature refining to obtain a plurality of refined features F

[0017] S33, using the feature distillation unit to respectively perform face feature distillation on the plurality of refined features F to obtain distillation features F and performing feature fusion, performing feature reuse through the point-by-point convolution to obtain fusion features F

[0018] S34, respectively using the convolution branch and the face attention branch to process the fusion features F fusion , and performing element-by-element multiplication and addition to obtain an output of the current efficient face preserving attention mechanism;

[0019] S35, gradually learning global face attention weight parameters based on a large kernel convolution through stacking of the plurality of face structure preserving distillation modules, and obtaining deep features F D based on the global face attention weight parameters based on the large kernel convolution and the output of the efficient face preserving attention mechanism.

[0020] According to the face image super-resolution method based on the face structure preserving distillation network, the convolution branch comprises a 1*1 point-by-point convolution, a 5*5 depth separable convolution and a 5*5 depth hole convolution, and the calculation formula is as follows:

[0021]

[0022] in, PConv represents the output characteristics of the large kernel convolution decomposition module. 1×1 (·) represents a pointwise convolution of size 1×1, DwConv 5×5 (·) represents a depthwise separable convolution of size 5×5, DwDConv 5×5 (·) represents a 5×5 depthwise dilated convolution, F fusion This indicates the fusion feature.

[0023] According to the present invention, a face image super-resolution method based on a face structure-preserving distillation network is provided. The output calculation formula of the current efficient face-preserving attention mechanism is as follows:

[0024]

[0025] in, This represents the output of the efficient face-preserving attention mechanism at the i-th index. β represents the output of the efficient face-preserving attention mechanism at the (i-1)th index. i This represents the weights for maintaining attention efficiently on the face, where σ represents the Sigmoid activation function. This indicates the facial attention branch operation.

[0026] According to the present invention, a face image super-resolution method based on a facial structure-preserving distillation network is provided. The deep upsampling residual module includes two nearest neighbor difference operation units, three feature extractors, one hourglass module, one pointwise convolution, one 3×3 depth-separable convolution, and one sigmoid activation function. The feature extractor includes a reflection filling module, one pointwise convolution, and one 3×3 depth-separable convolution.

[0027] According to the present invention, a face image super-resolution method based on a face structure-preserving distillation network is provided, wherein the overall loss function L based on the face structure-preserving distillation network is constructed. EFPDNet Specifically:

[0028] Reference image I of the target high-resolution image HR and high-resolution face images I SR Image preprocessing is performed, and the Sobel kernel function is used to calculate the gradient image in the horizontal direction and the target high-resolution image reference image in the vertical direction, respectively; the gradient image in the horizontal direction includes the target high-resolution image reference image I. HR Horizontal gradient image and high-resolution face images I SR Horizontal gradient image The target high-resolution image reference map in the vertical direction includes a target high-resolution image reference map I HR The vertical direction gradient image of the high-resolution face image And the high-resolution face image I SR The vertical direction gradient image of the high-resolution face image

[0029] The gradient image in the horizontal direction and the target high-resolution image reference map in the vertical direction are both gradient blocked, and the variance of each gradient image block is calculated.

[0030] The variance of the gradient image in the horizontal direction in the channel dimension and the variance of the gradient image in the vertical direction in the channel dimension are determined according to the variance of the gradient image block, and the horizontal direction loss function and the vertical direction loss function are calculated to construct the loss function L of the face structure maintaining distillation network EFPDNet .

[0031] According to the face structure maintaining distillation network-based face image super-resolution method provided by the application, the loss function L of the face structure maintaining distillation network EFPDNet The calculation formula is:

[0032] L EFPdNet =λ1L1+λ2L x +λ3L y ;

[0033]

[0034] Wherein, L EFPDNet Indicates the loss function of the face structure maintaining distillation network, λ1 indicates the weight coefficient of the pixel domain loss, λ2 indicates the weight coefficient of the horizontal direction loss, λ3 indicates the weight coefficient of the vertical direction loss, L1 indicates the pixel domain loss function, L x Indicates the horizontal direction loss function, L y Indicates the vertical direction loss function, Indicates the expectation of the high-resolution face image, Indicates the gradient of the target high-resolution image reference map in the horizontal direction, Indicates the gradient of the high-resolution face image in the horizontal direction, Indicates the gradient of the target high-resolution image reference map in the vertical direction, Indicates the gradient of the high-resolution face image in the vertical direction, and ‖·‖1 indicates the L1 norm.

[0035] The application also provides a face image super-resolution system based on a face structure maintaining distillation network, which adopts the face image super-resolution method as described above, comprising:

[0036] An up-sampling module configured to collect a low-resolution face image and up-sample the low-resolution face image to obtain a pseudo high-resolution image;

[0037] A shallow down-sampling residual module configured to input the pseudo high-resolution image into the shallow down-sampling residual module for processing to obtain a shallow feature;

[0038] An efficient face preserving distillation module configured to input the shallow feature into the efficient face preserving distillation module for deep extraction and feature refining to obtain a deep feature;

[0039] A deep up-sampling residual module configured to use the deep up-sampling residual module to gradually refine the deep feature to obtain a refined face feature;

[0040] A face image reconstruction module configured to perform convolution operation on the refined face feature to obtain a high-resolution face image, construct an overall loss function based on the face structure preserving distillation network, optimize network parameters based on the face structure preserving distillation network, obtain a training model, and update the high-resolution face image until reaching an upper limit of a training number based on the face structure preserving distillation network;

[0041] A face image super-resolution test module configured to test test data according to the training model to obtain a qualified model, the qualified model being used to realize reconstruction of a low-resolution face image.

[0042] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the face image super-resolution method according to any one of the above when executing the program.

[0043] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the face image super-resolution method according to any one of the above.

[0044] The face image super-resolution method based on the face structure preserving distillation network provided by the application can process an image through a shallow down-sampling residual module, halve the spatial dimension of a feature map, double the number of output channels, use an hourglass module to help the network capture and fuse multi-scale features at different levels, thereby improving the perception ability of the network for face details and overall face information, enhance the interaction of global face structure information at the channel level by learning global face attention weight parameters based on large kernel convolution, effectively capture and preserve key face features, and improve the quality of face images while maintaining low computational complexity. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0046] Figure 1 is a flowchart of the face image super-resolution method provided by the present application;

[0047] Figure 2 is a flowchart of the face image super-resolution method provided by the present application;

[0048] Figure 3 is a block diagram of the efficient face preserving distillation module of the face image super-resolution method provided by the present application;

[0049] Figure 4 is a block diagram of the efficient face preserving attention mechanism of the face image super-resolution method provided by the present application;

[0050] Figure 5 is an embodiment result graph of the face image super-resolution method provided by the present application;

[0051] Figure 6 is a block diagram of the face image super-resolution system provided by the present application;

[0052] Figure 7 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will combine the drawings and embodiments in the present application to clearly and completely describe the technical solutions in the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0054] As shown in Figure 1 , the embodiment of the present application provides a face image super-resolution method based on face structure preserving distillation network, comprising the following steps:

[0055] S1, collecting a low-resolution face image I LR , and upsampling the low-resolution face image I LR to obtain a pseudo high-resolution image I LR↑ .

[0056] In an embodiment of the present application, an up-sampling module 101 is configured to use bicubic interpolation to up-sample the low-resolution image I LR to the same spatial dimension as the target high-resolution image reference I , obtaining a pseudo high-resolution image I , where H represents the height of the image, W represents the width of the image, , and the real field. Figure 2 , where up represents up-sampling.

[0057] In an embodiment of the present application, the disclosed LFW face dataset is preferably used. The LFW (Labeled Faces in the Wild) dataset is a face image database compiled by the Computer Vision Laboratory of the University of Massachusetts, which aims to study face recognition problems in non-restricted situations. The LFW dataset includes 13323 celebrity face images from the Internet, which exhibit various natural scene characteristics, including different shooting orientations, expressions, and lighting conditions. The LFW dataset contains 5749 different celebrities, of which 1680 have 2 or more face pictures, 4069 have only 1 picture, and only 85 have more than 15 pictures. Each face picture is assigned a corresponding name ID and serial number for differentiation. The dataset contains 6000 pairs of face images, of which 3000 pairs are two different photos of the same person, and the other 3000 pairs are each a face photo of a different person. Therefore, the LFW dataset has been widely used to evaluate the performance of face verification (Face Verification) algorithms, especially in the task of face recognition in natural scenes. In the LFW dataset, 2110 images are randomly selected, including different shooting orientations, expressions, and lighting conditions, and 50 and 60 of them are respectively used as the verification set and the test set. In the data processing stage, the images are resized to 112x112x3 as HR images, i.e., target high-resolution image reference images. Then, bicubic interpolation is used to down-sample the high-resolution images to low-resolution images, and under the condition of four and eight down-sampling factors, the sizes of the low-resolution images are 28x28x3 and 14x14x3, respectively.

[0058] S2, a shallow down-sampling residual module is constructed, and the pseudo high-resolution image I LR↑ is input into the shallow down-sampling residual module for processing, obtaining a shallow feature F S,3 .

[0059] As Figure 2As shown, in an embodiment of the present application, the shallow down-sampling residual module 102 includes a first down-sampling unit, a second down-sampling unit, a third down-sampling unit, an hourglass module, a 1x1 point-wise convolution, a 3x3 depthwise separable convolution, and an activation function, wherein the first down-sampling unit, the second down-sampling unit, and the third down-sampling unit have the same structure, and the processing method of the shallow down-sampling residual module is as follows:

[0060] The pseudo high-resolution image I LR↑ is processed by the first down-sampling unit to obtain an identity mapping feature F I .

[0061] The pseudo high-resolution image I LR↑ is processed by the second down-sampling unit and the third down-sampling unit to obtain a coarse feature F C .

[0062] The coarse feature F C is transmitted to the hourglass module, and after a 1x1 point-wise convolution, a 3x3 depthwise separable convolution, and a sigmoid activation function, an attention weight a is obtained.

[0063] The coarse feature F C is element-wise multiplied by the attention weight a, and the multiplication result is element-wise added to the identity mapping feature F I to obtain the output feature F S of the shallow down-sampling residual module 102.

[0064] In an embodiment of the present application, the first down-sampling unit includes a reflection padding operation unit, an inverse point-wise convolution, and a depthwise separable convolution with a convolution kernel of 3x3 and a step of 2, and the calculation formula of the coarse feature F C is as follows:

[0065] F C =H FE↓ (H FE (I LR↑ ))=H FE↓ (DwConv 3×3,s＝2 PConv 1×1 (δ(I LR↑ )))

[0066] Wherein, F C represents the coarse feature, δ(·) represents the reflection padding operation, PConv 1×1 (·) represents the point-wise convolution with a convolution kernel of 1x1, DwConv 3×3,s＝2 (·) represents the depthwise separable convolution with a convolution kernel of 3x3 and a step s of 2, and H FE (·) represents the feature extraction operation without reducing the size of the feature map, H FE↓(·) represents a feature extraction operation that reduces the size of the feature map;

[0067] The structures of the second downsampling unit and the third upsampling unit are the same as that of the first downsampling unit, and will not be described here.

[0068] In an embodiment of the present application, the face structure preserving distillation network has three shallow layer downsampling residual modules (preferably, three shallow layer downsampling residual modules are recommended to be set, and the number of shallow layer downsampling residual modules ranges from 1 to 5), the plurality of shallow layer downsampling residual modules are stacked, and the pseudo high-resolution image I LR↑ The plurality of shallow layer downsampling residual modules are input for processing, for example Figure 2 In an embodiment, three shallow layer downsampling residual modules are set, which are respectively denoted as shallow layer downsampling residual module 1, shallow layer downsampling residual module 2, and shallow layer downsampling residual module 3, to obtain shallow layer features F S,3 .

[0069] The spatial dimension of the feature map is halved and the number of output channels is doubled by processing the image through the shallow layer downsampling residual module.

[0070] The present application reduces the spatial resolution of the image through the shallow layer downsampling residual module, reduces the memory and computing resources required by the model during processing, and enables the network to capture features of different scales, which is very helpful for understanding the local and global information of the face image.

[0071] As shown in Figure 3 and Figure 4 S3, an efficient face preserving distillation module is constructed, and the shallow layer features F S，3 are input into the efficient face preserving distillation module for deep extraction and feature refinement to obtain deep features F D .

[0072] In an embodiment of the present application, step S3 specifically includes:

[0073] S31, a plurality of efficient face preserving distillation modules are constructed, for example Figure 2 In an embodiment, N efficient face preserving distillation modules are set, which are respectively denoted as efficient face preserving distillation module 1, efficient face preserving distillation module 2, and efficient face preserving distillation module N (preferably, three efficient face preserving distillation modules are recommended to be set, and N ranges from 1 to 10); the efficient face preserving distillation module includes a plurality of feature refinement units, feature distillation units, and an efficient face preserving attention mechanism, wherein each feature refinement unit includes a point-wise convolution and two 3x3 depth separable convolutions, the efficient face preserving attention mechanism includes a face attention branch and a convolution branch, the face attention branch includes an hourglass module and a 3x3 depth separable convolution;

[0074] S32, input the shallow feature F S,3 Input to the feature refining unit for feature refining to obtain a plurality of refined features For example Figure 3 In the embodiment, three feature refining units are provided, which are respectively denoted as a feature refining unit 1, a feature refining unit 2 and a feature refining unit 3 (wherein the number of feature refining units is preferably 3, and the number of feature refining units is in the range of 1-5).

[0075] S33, using a feature distillation unit to respectively distill the plurality of refined features to obtain distilled features and performing feature fusion, and performing feature reuse through point-by-point convolution to obtain fused features F fusion ;

[0076] Specifically, the formula of feature distillation is as follows:

[0077]

[0078] Wherein, FD1(·) represents a first distillation layer for generating facial distillation features, FD2(·) represents a second distillation layer for generating facial distillation features, FD3(·) represents a third distillation layer for generating facial distillation features, FD4(·) represents a fourth distillation layer for generating facial distillation features, FR1(·) represents a first refining layer for refining facial rough features, FR2(·) represents a second refining layer for refining facial rough features, FR3(·) represents a third refining layer for refining facial rough features, and FR4(·) represents a fourth refining layer for refining facial rough features, represents a first distillation feature, represents a second distillation feature, represents a third distillation feature, represents a fourth distillation feature, represents a first refined feature, represents a second refined feature, represents a third refined feature, F S represents the output of a shallow layer down-sampling module.

[0079] In an embodiment of the present application, the high-efficiency face maintaining distillation module includes a first feature refining unit, a second feature refining unit, a third feature refining unit, a feature distillation unit and a high-efficiency face maintaining attention mechanism, and the processing method of the high-efficiency face maintaining distillation module is:

[0080] After point-by-point convolution of the shallow feature, the first feature is input to the feature distillation unit;

[0081] The shallow feature is input into the first feature refining unit, and after point-by-point convolution and feature distillation unit, the second feature is obtained;

[0082] The shallow feature is input into the first feature refining unit, and after point-by-point convolution and feature distillation unit, the second feature is obtained;

[0083] The shallow feature is input into the first feature refining unit, and after point-by-point convolution and feature distillation unit, the second feature is obtained;

[0084] The shallow feature is input into the first feature refining unit, and after point-by-point convolution and feature distillation unit, the second feature is obtained;

[0085] The shallow feature is input into the first feature refining unit, and after point-by-point convolution and feature distillation unit, the second feature is obtained;

[0086] Specifically, the processing method of the feature refining unit is:

[0087] The shallow feature is input into the first feature refining unit, and after point-by-point convolution and feature distillation unit, the second feature is obtained;

[0088] The shallow feature is input into the first feature refining unit, and after point-by-point convolution and feature distillation unit, the second feature is obtained;

[0089] The shallow feature is input into the first feature refining unit, and after point-by-point convolution and feature distillation unit, the second feature is obtained;

[0090] The shallow feature is input into the first feature refining unit, and after point-by-point convolution and feature distillation unit, the second feature is obtained; D .

[0091] In an embodiment of the present application, the feature fusion is performed in the form of feature splicing, and the formula of feature fusion is:

[0092]

[0093] Wherein, F fusion represents the fusion feature, Concat(·) represents the feature splicing operation, PConv 1×1 (·) represents 1×1 point-by-point convolution.

[0094] S34, respectively using convolution branch and face attention branch to process the fusion feature F fusion , and element-wise multiplication and addition are performed to obtain the output of the current efficient face keeping attention mechanism;

[0095] In an embodiment of the present application, the convolution branch includes a 1x1 pointwise convolution, a 5x5 depth separable convolution, and a 5x5 depth dilated convolution, and the calculation formula is as follows:

[0096]

[0097] wherein, represents the output feature of the large kernel convolution decomposition module, PConv 1×1 (·) represents a 1x1 pointwise convolution, DwConv 5×5 (·) represents a 5x5 depth separable convolution, DwDConv 5×5 (·) represents a 5x5 depth dilated convolution, F fusion represents the fusion feature.

[0098] In an embodiment of the present application, the output calculation formula of the current efficient face keeping attention mechanism is as follows:

[0099]

[0100] wherein, represents the output of the efficient face keeping attention mechanism under the i-th index, represents the output of the efficient face keeping attention mechanism under the i-1-th index, represents the efficient face keeping attention weight, and sigma represents a Sigmoid activation function, represents the face attention branch operation, H i represents the input channel number, W i represents the output channel number.

[0101] In an embodiment of the present application, in the efficient face keeping attention mechanism, firstly, a KxK convolution is decomposed into a pointwise convolution, a (2T-1)x(2T-1) depth separable convolution, and a (K / T)x(K / T) depth dilated convolution, wherein T=3 is the size of the expansion rate, and the size of the convolution kernel K is preferably 5.

[0102] In an embodiment of the present application, the face attention branch includes an hourglass module and a depth separable convolution with a step size of 1 and a convolution kernel of 3x3, and the hourglass module includes a first depth separable convolution unit, a second depth separable convolution unit, a third depth separable convolution unit, and a fourth depth separable convolution unit, wherein the first depth separable convolution unit, the second depth separable convolution unit, the third depth separable convolution unit, and the fourth depth separable convolution unit have the same structure, and are all depth separable convolution units with a step size of 2 and a convolution kernel of 3x3, and the processing method of the hourglass unit includes:

[0103] The input feature is input to the first deep separable convolution unit for processing to obtain first deep separable convolution feature;

[0104] The first deep separable convolution feature is processed by using the second deep separable convolution unit and the third deep separable convolution unit respectively to obtain second deep separable convolution feature and third deep separable convolution feature;

[0105] The second deep separable convolution feature and the third deep separable convolution feature are element-wise added to obtain fourth deep separable convolution feature;

[0106] The fourth deep separable convolution feature is input to the fourth deep separable convolution unit for processing to obtain the output of the hourglass module.

[0107] It can be understood that the hourglass module helps to capture information at different scales, and the high-level semantic information is preserved through the skip connection. The hourglass module is used to help the network capture and fuse multi-scale features at different levels, thereby improving the perception ability of the face details and the overall information of the face.

[0108] In an embodiment of the present application, the processing method of the efficient face-keeping attention mechanism is:

[0109] The input feature is processed through a 1x1 point convolution, a 5x5 deep separable convolution and a 5x5 deep hole convolution to obtain first processing feature;

[0110] The first processing feature is processed by using the hourglass module, and is multiplied by the first processing feature through a deep separable convolution with a step of 1 and a convolution kernel of 3x3 and a Sigmoid activation function to obtain second processing feature;

[0111] The input feature and the second processing feature are element-wise added to obtain the output of the efficient face-keeping attention mechanism.

[0112] S35, the global face attention weight parameter based on large kernel convolution is learned step by step through the plurality of face structure reservation distillation module stacks, and the global face attention weight parameter based on large kernel convolution and the output of the efficient face-keeping attention mechanism obtain the deep feature F D .

[0113] Specifically, the calculation formula of the deep feature F D is as follows:

[0114]

[0115] wherein denotes the Nth, (N-1)th, …, 1st efficient face maintaining distillation module, N is the total number of efficient face maintaining distillation modules, F D denotes the deep feature output by the N efficient face maintaining distillation modules.

[0116] The application enhances the interaction of global structure information of a face image at a channel level by learning global face attention weight parameters based on large kernel convolution, and can effectively capture and maintain key face features.

[0117] S4, a deep up-sampling residual module is constructed, and the deep up-sampling residual module is used to process the deep feature F D to obtain a refined face feature.

[0118] In an embodiment of the application, the deep up-sampling module 104 includes two nearest neighbor difference operation units, three feature extractors, one hourglass module, one pointwise convolution, one 3x3 depth separable convolution, and one Sigmoid activation function, wherein each feature extractor includes one reflection padding module, one 1x1 pointwise convolution, and one 3x3 depth separable convolution, and the processing method of the deep up-sampling module is as follows:

[0119] The deep feature F D is input into the two nearest neighbor difference operation units respectively to obtain first interpolation features and second interpolation features.

[0120] The first interpolation features pass through one feature extractor to obtain first extracted features.

[0121] The second interpolation features pass through two feature extractors to obtain second extracted features.

[0122] The second extracted features pass through one hourglass module, one 1x1 pointwise convolution, one 3x3 depth separable convolution, and one Sigmoid activation function to obtain third extracted features.

[0123] The third extracted features are multiplied element by element with the second extracted features to obtain fourth extracted features.

[0124] The fourth extracted features are added element by element with the first extracted features to obtain the output of the deep up-sampling module, i.e., a refined face feature.

[0125] The application gradually increases the spatial resolution of the feature map by constructing a deep up-sampling residual module to restore the detail information of the image, and fuses the multi-scale features extracted in the down-sampling module with the up-sampled feature map. This fusion helps to retain more details and texture information in the high-resolution image.

[0126] S5, constructing a face image reconstruction module, performing convolution operation on the refined face features to obtain a high-resolution face image I SR , and constructing an overall loss function L based on the face structure preserving distillation network EFPDNet , optimizing the network parameters based on the face structure preserving distillation network to obtain a trained model, and updating the high-resolution face image I SR , until reaching an upper limit of the training times based on the face structure preserving distillation network.

[0127] Specifically, the high-resolution face image I SR is calculated according to the following formula:

[0128]

[0129] wherein, I SR represents the high-resolution face image, H FSPDNet (·) represents the overall operation based on the face structure preserving distillation network, H DURB,1 (·) represents the output of the first deep up-sampling residual module, H DURB,2 (·) represents the output of the second deep up-sampling residual module, H DURB,3 (·) represents the output of the third deep up-sampling residual module, represents the deep feature output by the last efficient face preserving distillation module, Conv 3×3 represents a 3x3 size convolution.

[0130] Further, an overall loss function L based on the face structure preserving distillation network is constructed EFPDNet , specifically as follows:

[0131] The target high-resolution image reference I HR and the high-resolution face image I SR are preprocessed, and are calculated using Sobel kernel functions respectively to obtain a horizontal gradient image and a vertical target high-resolution image reference, respectively; wherein the horizontal gradient image includes a horizontal gradient image HR of the target high-resolution image reference I and a horizontal gradient image SR of the high-resolution face image I The vertical target high-resolution image reference includes a vertical gradient image HR of the target high-resolution image reference I and a vertical gradient image SR of the high-resolution face image I

[0132] The gradient image in the horizontal direction and the target high-resolution image reference map in the vertical direction are both gradient blocked, and the variance of each gradient image block is calculated;

[0133] The variance of the gradient image block in the horizontal direction and the variance of the gradient image block in the vertical direction are determined according to the variance of the gradient image block, and the horizontal direction loss function and the vertical direction loss function are calculated to construct the loss function L of the face structure preserving distillation network EFPDNet .

[0134] In the loss calculation, the edge information extracted by the Sobel operator is used as a weight to weight the original pixel-level loss (L1 loss), so that the loss function pays more attention to the reconstruction quality of the edge information, and promotes the network to pay more attention to the edge details in the training process.

[0135] In an embodiment of the present application, the gradient image in the horizontal direction and the target high-resolution image reference map in the vertical direction are expanded into m x m image blocks to obtain corresponding dimension sizes Gradient image block Wherein, w represents the width of the gradient image block, h represents the height of the gradient image block, represents the gradient image block in the horizontal direction of the target high-resolution image reference map, represents the gradient image block in the vertical direction of the target high-resolution image reference map, represents the gradient image block in the horizontal direction of the high-resolution face image, represents the gradient image block in the vertical direction of the high-resolution face image;

[0136] The variance of the expanded gradient image block in the channel dimension is calculated, and the calculation formula is as follows:

[0137]

[0138] Wherein v i represents the channel dimension variance of the i-th gradient image block, is the average value of the i-th gradient image block, G i ′ ,j represents the i-th expanded gradient image block.

[0139] Further, the loss function L of the face structure preserving distillation network is calculated EFPDNet The calculation formula is:

[0140] L EFPdNet =λ1L1+λ2L x +λ3L y ;

[0141]

[0142] wherein, L EFPDNet represents a loss function based on the face structure preserving distillation network, represents a weight coefficient of a pixel domain loss, represents a weight coefficient of a horizontal direction loss, represents a weight coefficient of a vertical direction loss, L1 represents a pixel domain loss function, L x represents a horizontal direction loss function, L y represents a vertical direction loss function, E ISR represents an expectation of a high-resolution face image, represents a gradient of a target high-resolution image reference in a horizontal direction, represents a gradient of a high-resolution face image in a horizontal direction, represents a gradient of a target high-resolution image reference in a vertical direction, represents a gradient of a high-resolution face image in a vertical direction, and ‖·‖1 represents an L1 norm.

[0143] S6, a face image super-resolution test module is constructed, test data are tested according to the model obtained by training, and after a qualified model is obtained, the low-resolution face image is reconstructed.

[0144] The application effectively solves the problems of insufficient texture details and inaccurate structure information in the practical application of face image super-resolution on low-power devices based on the face structure preserving distillation network, especially under the condition of a low-parameter model. The method significantly improves the quality of the face image while maintaining a low computational complexity, making it more suitable for application on mobile devices and other resource-constrained platforms.

[0145] In an embodiment of the application, the face structure preserving distillation network includes 3 shallow down-sampling modules, N efficient face preserving distillation modules and 3 deep up-sampling modules (wherein the number of the shallow down-sampling modules needs to be consistent with the number of the deep up-sampling modules to ensure that the input and output image sizes are consistent.).

[0146] Let the image size of a low-resolution face image I LR be H'xW'x3, after up-sampling, the image size of a pseudo high-resolution image I LR↑ is HxWx3, wherein H represents the height of the image, and W represents the width of the image.

[0147] The pseudo high-resolution image I LR↑ is subjected to a first shallow down-sampling module to obtain a first shallow down-sampling feature with a size of C represents the number of channels.

[0148] The first shallow down-sampling feature is subjected to a second shallow down-sampling module to obtain a second shallow down-sampling feature with a size of

[0149] The second shallow layer down-sampling feature is input into a third shallow layer down-sampling module to obtain a third shallow layer down-sampling feature with a size of

[0150] The third shallow layer down-sampling feature is input into N efficient face preserving distillation modules to obtain a distillation feature with a size of

[0151] The distillation feature is input into a first deep layer up-sampling module to obtain a first deep layer up-sampling feature with a size of

[0152] The first deep layer up-sampling feature is input into a second deep layer up-sampling module to obtain a second deep layer up-sampling feature with a size of

[0153] The second deep layer up-sampling feature is input into a third deep layer up-sampling module and a 3x3 normal convolution to obtain a high-resolution face image with a size of HxWx3.

[0154] As shown in FIG. 1, a specific embodiment is described in the present embodiment, and in the present embodiment, an LFW dataset is used, which contains 202599 face images of 10177 people: Figure 5

[0155] In the data processing stage, the original face image is randomly cropped to a size of 128x128x3, and is taken as an HR label image. The training set, the validation set and the test set in the dataset are randomly selected to be 2000, 50 and 50 images respectively. Through such data setting, different people, postures and facial expressions can be covered, and the training requirements in different scenarios are comprehensively considered to simulate the performance of the model in various situations.

[0156] In the LFW dataset, 2110 images are randomly selected, including different shooting directions, expressions and lighting conditions, and 50 and 60 images are taken as the validation set and the test set respectively.

[0157] In the data processing stage, the image is reshaped to a size of 112x112x3 as an HR image.

[0158] The HR image is down-sampled to an LR image using a bicubic interpolation method. Under the condition of a four-fold and an eight-fold down-sampling factor, the size of the LR image is 28x28x3 and 14x14x3 respectively. It should be noted that all the training and testing are based on the luminance channel in the YCbCr color space, and the training and testing are performed using a four-fold and an eight-fold sampling factor.

[0159] ​The performance of the SR reconstruction in the luminance channel is tested by using four evaluation indexes, including Peak signal-to-noise ratio (PSNR), Structural SIMilarity (SSIM), Feture SIMilarity (FSIM) and Visual Information Fidelity (VIF) to evaluate the SR reconstruction results.

[0160] In each iteration of training, the channel number of the distillation structure in the FSPDNet and the channel number of the facial attention module are both set to 64, patch_size = 32 when the down-sampling factor is x4, and patch_size = 16 when the down-sampling factor is x8.

[0161] The network is trained by using the loss function and the Adam optimizer of the application, and the hyperparameters thereof are λ1 = 0.9, λ2 = 0.05, λ3 = 0.05, β1 = 0.9, β2 = 0.999, and the batch size batch_size = 32. Data augmentation is performed on all training images, and the images are randomly rotated by 90 degrees, 180 degrees, 270 degrees and horizontally flipped. The overall iteration period of the FSPDNet is 500 epochs, the initial learning rate is set to 1e-4, the experiment of the application is based on the deep learning library PyTorch, Python 3.8.5 is used as the running environment, the system platform is Ubuntu 18.04, and the experiment is performed on an NVIDIA 3090 GPU.

[0162] The selected face image SR methods for comparison include Bicubic, EDSR, SAN, MTC, DIDnet, SPARNet, SMSR and BSRN. Bicubic is a classic image interpolation algorithm; EDSR is a classic deep residual network SR method; SAN is a SR method based on a second-order attention mechanism; MTC is a multi-view texture compensation network FSR method; DIDnet is a dual-identity constraint network-based FSR method; SPARNet is a spatial attention network-based FSR method; SMSR is a sparse mask network-based SR method; and BSRN is a blueprint separable residual network-based SR method.

[0163] Table 1 shows the comparison results under the conditions of reconstruction multiples of 4 and 8 by using the above four evaluation indexes, wherein HR represents the original high-resolution face image reference.

[0164] Table 1 Comparison results of the application and seven face image SR methods

[0165]

[0166]

[0167] As can be seen from Table 1, compared with other seven face image SR methods, FSPDNet (Facial structure preserving distillation network) based on face structure preserving distillation network has achieved significant improvement in face image reconstruction quality compared with other lightweight methods.

[0168] As Figure 6 shown, the present application provides a face image super-resolution system based on a facial structure preserving distillation network, which adopts the face image super-resolution method described above, comprising:

[0169] an upsampling module configured to collect a low-resolution face image and perform upsampling on the low-resolution face image to obtain a pseudo-high-resolution image;

[0170] a shallow down-sampling residual module configured to input the pseudo-high-resolution image into the shallow down-sampling residual module for processing to obtain shallow features;

[0171] an efficient face preserving distillation module configured to input the shallow features into the efficient face preserving distillation module for deep extraction and feature refinement to obtain deep features;

[0172] a deep up-sampling residual module configured to use the deep up-sampling residual module to gradually refine the deep features to obtain refined face features;

[0173] a face image reconstruction module configured to perform convolution operation on the refined face features to obtain a high-resolution face image, construct an overall loss function based on the facial structure preserving distillation network, optimize the network parameters based on the facial structure preserving distillation network, obtain a training model, update the high-resolution face image, and stop until the upper limit of the training times based on the facial structure preserving distillation network is reached;

[0174] a face image super-resolution test module configured to test test data according to the training model to obtain a qualified model, which is used to realize the reconstruction of the low-resolution face image.

[0175] The face image super-resolution device provided by the present application is described below, and the face image super-resolution device described below can be correspondingly referred to the face image super-resolution method described above.

[0176] Figure 7 An example of an electronic device entity structure schematic diagram is shown as Figure 7As shown, the electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute a face image super-resolution method, which includes: collecting a low-resolution face image, and performing up-sampling on the low-resolution face image to obtain a pseudo-high-resolution image; constructing a shallow down-sampling residual module, inputting the pseudo-high-resolution image into the shallow down-sampling residual module for processing to obtain a shallow feature; constructing an efficient face preservation distillation module, inputting the shallow feature into the efficient face preservation distillation module for depth extraction and feature refining to obtain a depth feature; constructing a deep up-sampling residual module, using the deep up-sampling residual module to gradually refine the depth feature to obtain a refined face feature; constructing a face image reconstruction module, performing convolution operation on the refined face feature to obtain a high-resolution face image, and constructing an overall loss function based on the face structure preservation distillation network, optimizing the network parameters based on the face structure preservation distillation network to obtain a training model, updating the high-resolution face image until reaching an upper limit of the training times based on the face structure preservation distillation network; constructing a face image super-resolution test module, testing test data according to the training model to obtain a qualified model, and the qualified model is used to realize reconstruction of the low-resolution face image.

[0177] In addition, the logical instructions in the memory 730 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0178] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer-readable storage medium, and the computer program being executable by a processor to enable a computer to perform the face image super-resolution method provided by any of the above methods, the method comprising: collecting a low-resolution face image, and performing up-sampling on the low-resolution face image to obtain a pseudo high-resolution image; constructing a shallow down-sampling residual module, inputting the pseudo high-resolution image into the shallow down-sampling residual module for processing to obtain a shallow feature; constructing an efficient face preserving distillation module, inputting the shallow feature into the efficient face preserving distillation module for depth extraction and feature refinement to obtain a depth feature; constructing a deep up-sampling residual module, using the deep up-sampling residual module to perform step-by-step refined feature on the depth feature to obtain a refined face feature; constructing a face image reconstruction module, performing convolution operation on the refined face feature to obtain a high-resolution face image, and constructing an overall loss function based on the face structure preserving distillation network, optimizing the network parameters of the face structure preserving distillation network to obtain a training model, updating the high-resolution face image until reaching an upper limit of the training times of the face structure preserving distillation network; constructing a face image super-resolution test module, testing the test data according to the training model to obtain a qualified model, and the qualified model is used to realize the reconstruction of the low-resolution face image.

[0179] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the face image super-resolution method provided by any of the above methods, the method comprising: collecting a low-resolution face image, and performing up-sampling on the low-resolution face image to obtain a pseudo high-resolution image; constructing a shallow down-sampling residual module, inputting the pseudo high-resolution image into the shallow down-sampling residual module for processing to obtain a shallow feature; constructing an efficient face preserving distillation module, inputting the shallow feature into the efficient face preserving distillation module for depth extraction and feature refinement to obtain a depth feature; constructing a deep up-sampling residual module, using the deep up-sampling residual module to perform step-by-step refined feature on the depth feature to obtain a refined face feature; constructing a face image reconstruction module, performing convolution operation on the refined face feature to obtain a high-resolution face image, and constructing an overall loss function based on the face structure preserving distillation network, optimizing the network parameters of the face structure preserving distillation network to obtain a training model, updating the high-resolution face image until reaching an upper limit of the training times of the face structure preserving distillation network; constructing a face image super-resolution test module, testing the test data according to the training model to obtain a qualified model, and the qualified model is used to realize the reconstruction of the low-resolution face image.

[0180] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0181] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0182] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A face image super-resolution method based on a face structure preserving distillation network, characterized in that, The method comprises the following steps: S1, collect a low-resolution face image , and up-sample the low-resolution face image to obtain a pseudo-high-resolution image ; S2, constructing a shallow layer down-sampling residual module, inputting the pseudo high-resolution image to the shallow layer down-sampling residual module for processing to obtain shallow layer features ; S3, constructing an efficient face-preserving distillation module, inputting the shallow features to the efficient face-preserving distillation module for deep extraction and feature refinement to obtain deep features ; S4, construct a deep up-sampling residual module, use the deep up-sampling residual module to process the deep feature gradually refine the features to obtain refined face features; S5, constructing a face image reconstruction module, performing convolution operation on the refined face features to obtain a high-resolution face image , and constructing an overall loss function based on the face structure preserving distillation network , optimizing network parameters based on the face structure preserving distillation network to obtain a training model, and updating the high-resolution face image , until reaching an upper limit of a training number based on the face structure preserving distillation network S6, constructing a face image super-resolution test module, testing the trained model using test data to obtain a qualified model, the qualified model being used to realize reconstruction of a low-resolution face image; Step S3 specifically comprises: S31, constructing a plurality of efficient face preserving distillation modules; the efficient face preserving distillation module comprises a plurality of feature refining units, a feature distillation unit and an efficient face preserving attention mechanism, wherein each feature refining unit comprises a point-by-point convolution and two 3x3 depth separable convolutions, the efficient face preserving attention mechanism comprises a face attention branch and a convolution branch, the face attention branch comprises an hourglass module and a 3x3 depth separable convolution; S32, the shallow features The input is fed into the feature refinement unit for feature refinement, resulting in multiple refined features. ; S33, using the feature distillation unit to respectively distill the plurality of refining features performing face feature distillation to obtain distilled features and performing feature fusion, performing feature reuse through point-by-point convolution to obtain fused features ; S34, respectively using the convolution branch and the face attention branch on the fusion feature processing, and element-wise multiplication and addition are performed to obtain the output of the current efficient face keeping attention mechanism; S35, learn global face attention weight parameters based on large kernel convolution step by step through the multi-face structure preservation distillation module stack, and the output of the global face attention weight parameters based on large kernel convolution and the efficient face preservation attention mechanism obtains a deep feature ; The convolution branch includes one point-wise convolution, a 5x5 depthwise separable convolution, and a 5x5 depthwise atrous convolution, and the calculation formula is as follows: ； wherein, represents the output feature of the large kernel convolution decomposition module, represents a point-wise convolution of size represents a depthwise separable convolution of size represents depthwise dilated convolution of size represents the fused feature; The output calculation formula of the current efficient face preserving attention mechanism is as follows: ； ； wherein, represents the output of the efficient face keeping attention mechanism under the th index, represents the output of the efficient face keeping attention mechanism under the th index, represents the efficient face keeping attention weight, represents an activation function, represents the face attention branch operation.

2. The face image super-resolution method based on the face structure preserving distillation network according to claim 1, characterized in that, The deep up-sampling residual module includes 2 nearest neighbor difference operation units, 3 feature extractors, 1 hourglass module, one point-wise convolution, one 3x3 depthwise separable convolution, and one activation function, wherein the feature extractor includes one reflection padding module, one point-wise convolution, and one 3x3 depthwise separable convolution.

3. The face image super-resolution method based on the face structure preserving distillation network according to claim 1, characterized in that, The construction is based on the overall loss function of the face structure maintaining distillation network Specifically, Reference image of the target high-resolution image and high-resolution face images Image preprocessing is performed, and the Sobel kernel function is used to calculate the gradient image in the horizontal direction and the target high-resolution image reference image in the vertical direction, respectively; the gradient image in the horizontal direction includes the target high-resolution image reference image. Horizontal gradient image and high-resolution face images Horizontal gradient image The vertical target high-resolution image reference map includes the target high-resolution image reference map. Vertical gradient image and high-resolution face images Vertical gradient image ; The horizontal gradient image and the vertical target high-resolution image reference map are both gradient blocked, and the variance of each gradient image block is calculated; The variance of the gradient image block is determined to determine the variance of the horizontal direction gradient image in the channel dimension and the variance of the vertical direction in the channel dimension, and a horizontal direction loss function and a vertical direction loss function are calculated to construct a loss function based on the face structure preserving distillation network .

4. The face image super-resolution method based on the face structure preserving distillation network according to claim 3, characterized in that, The overall loss function of the face structure preserving distillation network The calculation formula is: ； ； ； wherein, represents the overall loss function of the face structure preserving distillation network, represents a weight coefficient of the pixel domain loss, represents a weight coefficient of the horizontal direction loss, represents a weight coefficient of the vertical direction loss, represents the pixel domain loss function, represents the horizontal direction loss function, represents the vertical direction loss function, represents the expectation of the high-resolution face image, represents the gradient of the target high-resolution image reference map in the horizontal direction, represents the gradient of the high-resolution face image in the horizontal direction, represents the gradient of the target high-resolution image reference map in the vertical direction, represents the gradient of the high-resolution face image in the vertical direction, represents the L1 norm.

5. A face image super-resolution system based on a face structure preserving distillation network, characterized in that, The face image super-resolution method comprises the following steps: an up-sampling module configured to collect a low-resolution face image and perform up-sampling on the low-resolution face image to obtain a pseudo-high-resolution image; a shallow down-sampling residual module configured to input the pseudo-high-resolution image into the shallow down-sampling residual module for processing to obtain shallow features; an efficient face preserving distillation module configured to input the shallow features into the efficient face preserving distillation module for deep extraction and feature refinement to obtain deep features; a deep up-sampling residual module configured to use the deep up-sampling residual module to gradually refine the deep features to obtain refined face features; a face image reconstruction module configured to perform convolution operation on the refined face features to obtain a high-resolution face image, construct an overall loss function based on the face structure preserving distillation network, optimize the network parameters based on the face structure preserving distillation network, obtain a trained model, update the high-resolution face image until the upper limit of the training times based on the face structure preserving distillation network is reached; a face image super-resolution test module configured to test test data according to the trained model to obtain a qualified model, the qualified model being used to realize reconstruction of a low-resolution face image.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the face image super-resolution method according to any one of claims 1 to 4 when executing the program. 7.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the face image super-resolution method according to any one of claims 1 to 4 when executed by the processor.

Images