Method for improving resolution of human face image, storage medium and electronic device

Abstract

The application provides a method for improving the resolution of a human face image, a storage medium and an electronic device, and the method comprises the following steps: receiving a human face image to be processed; restoring part of the structure information of the human face image through a spatial attention network to obtain an intermediate feature map; predicting the attribute information of the human face through a human face attribute prediction module, and predicting the prior information of the three-dimensional structure of the human face through a three-dimensional human face reconstruction branch module; and fusing the intermediate feature map, the attribute information of the human face and the prior information of the three-dimensional structure of the human face to obtain a high-resolution human face image. By introducing and fusing the attribute information of the human face and the prior information of the three-dimensional structure of the human face in the process of restoring the resolution of the human face image, the texture details of the human face image can be effectively highlighted, and the restoration effect of the human face image can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to a method, storage medium, and electronic device for improving the resolution of human face images. Background Technology

[0002] Facial images are a crucial component of image data. Face super-resolution is a specific field of image super-resolution restoration. While general image super-resolution methods perform well in everyday scenarios, they are not adept at restoring facial details. Therefore, it is essential to distinguish face super-resolution from general super-resolution methods and conduct targeted research. Furthermore, as a vital research area in computer vision, advancements in technologies such as face recognition, face alignment, and 3D face animation video generation have all benefited from high-quality facial images.

[0003] Existing face super-resolution restoration methods cannot effectively utilize the three-dimensional features and attribute information of the face, resulting in loss of restored facial details and poor restoration results. Summary of the Invention

[0004] Therefore, a technical solution is needed to improve the resolution of facial images, and to solve the problems of loss of detail and poor restoration effect that existing technologies easily cause when restoring super-resolution facial images.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for improving the resolution of a face image, the method comprising the following steps:

[0006] S1: Receive the face image to be processed;

[0007] S2: Recover some structural information of the face image through a spatial attention network to obtain an intermediate feature map;

[0008] S3: Predict facial attribute information through the facial attribute prediction module, and predict prior information about the three-dimensional structure of the face through the three-dimensional face reconstruction branch module;

[0009] S4: The intermediate feature map, the attribute information of the face, and the prior information of the three-dimensional structure of the face are fused to obtain a high-resolution face image.

[0010] In some embodiments, the face image to be processed is a low-resolution image, and the process further includes the following steps before step S2:

[0011] Interpolate low-resolution images to obtain blurred images of the desired resolution.

[0012] Step S2 includes:

[0013] The intermediate feature map is obtained by recovering some structural information of the blurred image through a spatial attention network.

[0014] In some embodiments, the spatial attention network includes an encoder, a feature processing layer, and a decoder in the order of image processing;

[0015] The encoder is used to downsample the face image to reduce the width and height of the feature map and to initially extract the feature map;

[0016] The feature processing layer is used to increase the depth of the network and process the downsampled feature map;

[0017] The decoder is used to upsample the feature map processed by the feature processing layer to restore the original width and height of the face image.

[0018] In some embodiments, the spatial attention network employs a funnel network structure.

[0019] In some embodiments, the method further includes:

[0020] Training the face attribute prediction module specifically includes: cropping face attribute labels from face images to form a training set, which is then input into a first deep residual network for training; the first deep residual network is configured to modify the stride of the first convolutional kernel (7x7) in the network to 1, and to change the output of the final fully connected layer to 16 dimensions to correspond to the cropped face attribute information.

[0021] The prediction of facial attribute information by the facial attribute prediction module includes:

[0022] The intermediate feature map is input into the trained face attribute prediction module to obtain the corresponding face attribute information.

[0023] In some embodiments, the prior information for predicting the three-dimensional structure of the face through the three-dimensional face reconstruction branch module includes:

[0024] Predict three-dimensional face parameters of an input face image, wherein the three-dimensional face parameters include one or more of identity, expression, texture, pose, and illumination;

[0025] The final coordinates and color values ​​of each vertex of the model are calculated based on the theory of three-dimensional deformable human face model;

[0026] Use a differentiable rendering tool to render a 3D face mesh onto the original input face image;

[0027] Calculate the loss function between the original input face image and the rendered face image, and then backpropagate to update the network weights.

[0028] In some embodiments, step S4 includes:

[0029] The previous layer feature m i-1 The input is downsampled into a convolutional module F1 with a stride of 2 to obtain the feature map f1;

[0030] The feature map f1 is compared with the feature map r of the same size output by the convolutional layer corresponding to the 3D face reconstruction branch module. i-1 By splicing the data along the channels, feature map f2 is obtained;

[0031] The feature map corresponding to the face attribute prediction module is a i- 1. Input convolution calculation module A, calculate the attribute space attention sa through the sigmoid function, and multiply the attention sa with the feature map f2 pixel by pixel to obtain the feature map f3;

[0032] m i-1 The input is a 3×3 convolution D with a stride of 2. This is downsampled and added pixel-by-pixel to the feature map f3 to calculate the residual, resulting in the output feature map m of this layer. i ;

[0033] Based on feature map m i Obtain high-resolution facial images.

[0034] In some embodiments, step S4 is followed by step S5:

[0035] The high-resolution face image is compared with the real image, and confidence information is output.

[0036] In a second aspect, the present invention also provides a storage medium storing a computer program that, when executed, implements the method described in the first aspect of the present invention.

[0037] In a third aspect, the present invention also provides a server, comprising:

[0038] The storage medium is the storage medium described in the second aspect of the present invention;

[0039] A processor, electrically connected to the storage medium, is configured to execute a computer program stored in the storage medium to implement the method as described in the first aspect of the invention.

[0040] Unlike existing technologies, this invention has the following characteristics:

[0041] This invention provides a method, storage medium, and electronic device for improving the resolution of facial images. The method includes: receiving a facial image to be processed; recovering partial structural information of the facial image through a spatial attention network to obtain an intermediate feature map; predicting facial attribute information through a facial attribute prediction module and predicting prior information of the facial 3D structure through a 3D facial reconstruction branch module; and fusing the intermediate feature map, facial attribute information, and prior information of the facial 3D structure to obtain a high-resolution facial image. This invention, by introducing the fusion of facial attribute information and prior information of the facial 3D structure during the facial image resolution recovery process, can effectively highlight the texture details of the facial image and improve the facial image recovery effect. Attached Figure Description

[0042] Figure 1 This is a flowchart of a method for improving the resolution of a human face image according to a first embodiment of the present invention;

[0043] Figure 2 This is a flowchart of a method for improving the resolution of a human face image according to a second embodiment of the present invention;

[0044] Figure 3 This is a flowchart of a method for improving the resolution of a human face image according to a third embodiment of the present invention;

[0045] Figure 4 This is a flowchart of a method for improving the resolution of a human face image according to the fourth embodiment of the present invention;

[0046] Figure 5 This is a flowchart of a method for improving the resolution of a human face image according to the fifth embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of an electronic device according to one embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of the overall network structure of a computer program according to one embodiment of the present invention;

[0049] Figure 8 This is a schematic diagram of the spatial attention network according to one embodiment of the present invention;

[0050] Figure 9 This is a schematic diagram of the structure of a face attribute prediction module according to one embodiment of the present invention;

[0051] Figure 10 This is a schematic diagram of the structure of a facial feature fusion branch according to one embodiment of the present invention;

[0052] Figure 11This chart compares a method for improving facial image resolution according to one embodiment of the present invention with other existing methods based on sensory evaluation metrics. Detailed Implementation

[0053] To illustrate the possible application scenarios, technical principles, implementable specific solutions, and achievable objectives and effects of this application in detail, the following description, in conjunction with the listed specific embodiments and accompanying drawings, provides a detailed explanation. The embodiments described herein are merely illustrative of the technical solutions of this application and are therefore intended to limit the scope of protection of this application.

[0054] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0055] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0056] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0057] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0058] Unless otherwise specified, the use of terms such as “comprising,” “including,” “having,” or other similar expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes elements, such that a process, method, or product that includes a series of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such process, method, or product.

[0059] As understood in the Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceeding" are understood to exclude the stated number; expressions such as "above," "below," and "within" are understood to include the stated number. Furthermore, in the description of the embodiments in this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups" and "multiple times," unless otherwise explicitly specified.

[0060] Please see Figure 1 In a first aspect, the present invention provides a method for improving the resolution of a face image, the method comprising the following steps:

[0061] S1: Receive the face image to be processed;

[0062] S2: Recover some structural information of the face image through a spatial attention network to obtain an intermediate feature map;

[0063] S3: Predict facial attribute information through the facial attribute prediction module, and predict prior information about the three-dimensional structure of the face through the three-dimensional face reconstruction branch module;

[0064] S4: The intermediate feature map, the attribute information of the face, and the prior information of the three-dimensional structure of the face are fused to obtain a high-resolution face image.

[0065] This invention effectively highlights the texture details of face images and improves the face image restoration effect by fusing the attribute information of the face and the prior information of the three-dimensional structure of the face during the face image resolution restoration process.

[0066] like Figure 2 As shown, the face image to be processed is a low-resolution image. Before step S2, the method further includes: interpolating the low-resolution image to obtain a blurred image of the required resolution size; step S2 includes: recovering some structural information of the blurred image through a spatial attention network to obtain an intermediate feature map.

[0067] Specifically, the low-resolution image can be converted into a blurred image of the desired resolution size by performing bicubic interpolation. Low-resolution images have too few pixels, making it difficult for the model to learn effective semantic and structural information. Bicubic interpolation converts the low-resolution face image into a blurred image of the desired resolution size, which then serves as input to the network model. Bicubic interpolation performs cubic interpolation based on the gray values ​​of 16 points surrounding the sampling point. This considers not only the gray value influence of the four directly adjacent points but also the influence of the rate of change of gray values ​​between neighboring points, resulting in a magnified image that more closely approximates the desired resolution size.

[0068] In some embodiments, the spatial attention network includes an encoder, a feature processing layer, and a decoder in the order of image processing; the encoder is used to downsample the face image to reduce the width and height of the feature map and to initially extract the feature map; the feature processing layer is used to deepen the network and process the downsampled feature map; the decoder is used to upsample the feature map processed by the feature processing layer to restore the original width and height of the face image.

[0069] like Figure 3 As shown, the method includes: first, step S301, the encoder performs downsampling on the face image to reduce the width and height of the feature map and initially extracts the feature map; then, step S302, the feature processing layer is used to deepen the network and process the downsampled feature map; then, step S303, the decoder upsamples the feature map processed by the feature processing layer to restore the original width and height of the face image.

[0070] Furthermore, the encoder includes a first preset number of dual attention units, the feature processing layer includes a second preset number of dual attention units, and the decoder includes a third preset number of dual attention units. Preferably, the first and third preset numbers are 3, and the second preset number is 8. The dual attention module enables the network to pay more attention to locations such as the corners of the mouth, the corners of the eyes, and the facial contours, which play a crucial role in evaluating the quality of super-resolution faces.

[0071] The dual attention unit employs a cascaded structure of channel attention and spatial attention. Channel attention allows the network to learn the importance of each facial feature channel, reducing computational cost and enhancing channel feature representation. Spatial attention helps the network learn which part of the facial feature map is most important, enhancing the network's sensitivity to structural information.

[0072] Preferably, the spatial attention network adopts a funnel network structure (Hourglass), such as... Figure 8 As shown, convolutional layers and max-pooling layers are used in the downsampling stage, while nearest-neighbor upsampling and cross-layer connections are used in the upsampling stage. The residual module extracts higher-level features while preserving information from the original layers, without changing the data size, only the data depth. Finally, two 1x1 convolutions are used to generate the final output. Employing a multi-scale structure to fuse information from multiple scales to extract structural information about the face is crucial for understanding facial features.

[0073] In some embodiments, the method further includes:

[0074] Training the face attribute prediction module specifically includes: cropping face attribute labels from face images to form a training set, which is then input into a first deep residual network for training; the first deep residual network is configured to modify the stride of the first convolutional kernel (7x7) in the network to 1, and to change the output of the final fully connected layer to 16 dimensions to correspond to the cropped face attribute information.

[0075] The prediction of facial attribute information by the facial attribute prediction module includes:

[0076] The intermediate feature map is input into the trained face attribute prediction module to obtain the corresponding face attribute information.

[0077] Since facial attributes are often limited by the scene and require additional input to the network, we construct a facial attribute branch to directly predict face-related attributes. This branch is pre-trained (i.e., the facial attribute prediction module mentioned above) and used as post-supervision in the super-resolution network. Simultaneously, the intermediate feature maps from the facial attribute prediction network are utilized to assist the training process of the super-resolution network.

[0078] Before training, the attribute information on the face image can be filtered to select more representative attribute information, and then the network structure can be modified according to the attribute prediction task.

[0079] The selection criteria for attribute information are as follows: First, remove color information that can be easily identified through low-resolution images, such as hair color; second, remove obvious structural information, such as small eyes, curved eyebrows, and whether someone is fat or thin; third, retain attribute information that is easily misjudged, such as straight hair and curly hair; fourth, retain occlusion attributes, such as wearing glasses or a tie. The final selected attribute information is shown in Table 1 below:

[0080] Table 1. Cropped Face Attribute Labels

[0081]

[0082] The face attribute prediction module in this application uses a modified deep residual network to predict attributes. Specific modifications include: 1. Adjusting the stride of the first convolutional kernel (7x7) from 2 to 1 based on the resolution of the input face image, without downsampling. 2. Changing the output of the final fully connected layer to 16 dimensions to correspond to the cropped face attributes, and removing the softmax layer.

[0083] As shown in Table 1, 16 facial attribute information can be selected from the face image as a reference, with satisfied attributes set to 1 and unsatisfied attributes set to 0. A pre-trained facial attribute prediction network is used as the network weights for the facial attribute prediction branch to participate in the subsequent super-resolution process.

[0084] like Figure 4 As shown, the prior information for predicting the 3D structure of the face through the 3D face reconstruction branch module includes:

[0085] First, proceed to step S401 to predict the three-dimensional face parameters of the input face image. The three-dimensional face parameters include any one or more of identity, expression, texture, pose, and lighting.

[0086] Then proceed to step S402, which calculates the final coordinates and color values ​​of each vertex of the model based on the theory of three-dimensional deformable human face model;

[0087] Then, in step S403, a differentiable rendering tool is used to render the 3D face mesh onto the original input face image.

[0088] Then, step S404 is performed to calculate the loss function between the original input face image and the rendered face image, and the network weights are updated via backpropagation.

[0089] The prior information includes facial shape, texture, expression, posture, and facial shadows. This invention reconstructs a 3D face from intermediate feature maps by constructing a 3D face reconstruction branch (i.e., the aforementioned 3D face reconstruction branch module), thereby obtaining more three-dimensional structural information and more refined texture information. Figure 9 As shown, the output 3D face parameters are used as post-supervision, and the intermediate features of the prediction process are fused into the face feature fusion branch to assist the face super-resolution process.

[0090] like Figure 5 As shown, step S4 includes:

[0091] First, proceed to step S501 to process the previous layer feature m. i-1 The input is downsampled into a convolutional module F1 with a stride of 2 to obtain the feature map f1;

[0092] Then, in step S502, the feature map f1 is compared with the feature map r of the same size output by the convolutional layer corresponding to the 3D face reconstruction branch module. i-1 By splicing the data along the channels, feature map f2 is obtained;

[0093] Then, in step S503, the feature map corresponding to the face attribute prediction module is processed. i- 1. Input convolution calculation module A, calculate the attribute space attention sa through the sigmoid function, and multiply the attention sa with the feature map f2 pixel by pixel to obtain the feature map f3;

[0094] Then proceed to step S504 to move m i-1 The input is a 3×3 convolution D with a stride of 2. This is downsampled and added pixel-by-pixel to the feature map f3 to calculate the residual, resulting in the output feature map m of this layer. i ;

[0095] Then proceed to step S505 based on feature map m i Obtain high-resolution facial images.

[0096] To better guide the face super-resolution process by integrating facial attribute information and 3D prior information, this invention constructs a facial feature fusion branch, using the 3D reconstructed facial features as an auxiliary feature map for super-resolution, directly connecting them on the channel dimension. Meanwhile, the facial attribute branch can focus on the key regions required for facial attributes, making it suitable for spatial attention and guiding the super-resolution network to notice key positions in the feature map.

[0097] like Figure 10 As shown, firstly, the features m from the previous layer... i-1 The input convolutional module F1 with a stride of 2 is downsampled to obtain feature map f1, which is then compared with the feature map r of the same size output from the corresponding convolutional layer of the 3D face reconstruction branch. i-1 The feature map is concatenated by channel, where C represents the concatenation function. The convolutional module F2 uses a 1×1 convolution to compress the number of channels in the feature map and extracts features to obtain feature map f2. The formula is shown below:

[0098]

[0099]

[0100] In the above formula, the plus sign in circles indicates pixel-by-pixel addition, and the multiplication sign in circles indicates pixel-by-pixel multiplication.

[0101] The feature map corresponding to the face attribute prediction branch is a i-1 The input convolutional computation module A (consisting of 1×1 convolution, feature extraction, and average pooling) is then processed by the sigmoid function to obtain the attribute space attention *sa*. This attention is then multiplied pixel-by-pixel by the feature map *f2*, and processed by the post-processing module F3 to obtain the feature map *f3*. Finally, *m*... i-1 The input is a 3×3 convolution with a stride of 2, downsampled by f3, pixel-by-pixel summed to obtain the residual, thus yielding the output feature map m of this layer. i After passing through three feature mixing modules, the feature map is input into the channel attention module to continue feature processing and upsampling operations, thereby obtaining a high-resolution face image.

[0102] In some embodiments, step S5 is included after step S4: comparing the high-resolution face image with the real image and outputting confidence information. A general discriminator for faces is used to judge and learn between the real image and the generated image, so as to enable the generator part to generate more realistic results.

[0103] In a second aspect, the present invention also provides a storage medium storing a computer program that, when executed, implements the method described in the first aspect of the present invention.

[0104] In the third aspect, such as Figure 6 As shown, the present invention also provides an electronic device 10, including a storage medium 102 and a processor 101, wherein the storage medium 102 is the storage medium as described in the second aspect; the processor 101 is electrically connected to the storage medium 102 and is used to execute a computer program stored in the storage medium to implement the method as described in the first aspect.

[0105] Preferably, the electronic device can be a computer device, including but not limited to: personal computers, servers, general-purpose computers, special-purpose computers, network devices, embedded devices, programmable devices, smart mobile terminals, smart home devices, wearable smart devices, and in-vehicle smart devices. Storage media include but are not limited to: RAM, ROM, magnetic disks, magnetic tapes, optical discs, flash memory, USB flash drives, portable hard drives, memory cards, memory sticks, network server storage, and network cloud storage. Processors include but are not limited to CPUs (Central Processing Units), GPUs (Graphics Processing Units), and MCUs (Microprocessors).

[0106] This invention employs a three-branch parallel structure: a face attribute prediction branch, a face feature fusion branch, and a face 3D reconstruction branch. For example... Figure 7 As shown, from left to right, the components are the spatial attention network module, the face attribute prediction branch, the face 3D reconstruction branch, the face feature fusion branch, and the discriminator. The three branches adopt a parallel structure, with two auxiliary branches (face attribute prediction and face 3D reconstruction) pre-trained. The feature fusion module integrates the attribute and reconstruction branch information into the face super-resolution restoration process. Overall, the method employs a generative adversarial network approach, with the spatial attention network module and the three-branch structure forming the generator. A face-specific discriminator is used to improve the quality of the generated images. This model uses face attributes and 3D face priors as auxiliary components in the collaborative face super-resolution restoration process, accurately predicting face-related attributes, restoring more three-dimensional structural information and detailed textures, and generating high-quality super-resolution face images.

[0107] The present invention has the following advantages:

[0108] First, the usual attributes need to be obtained additionally according to the scene. This invention solves the limitation of using face attributes as a priori scene, and designs an attribute prediction branch to directly predict face attribute information from low-resolution faces.

[0109] Second, the prior knowledge obtained by the face 3D reconstruction branch designed in this invention from predicting face 3D parameters from the middle features of the face is richer and more accurate than that obtained from low-resolution reconstruction. It can guide the super-resolution process from high-dimensional information such as shape, expression and posture.

[0110] Third, the present invention designs a feature fusion branch that efficiently utilizes and fuses semantic information of attributes and three-dimensional facial structural information into the super-resolution process.

[0111] Fourth, the experimental results of the model designed using this invention are as follows: Figure 11 As shown, the model designed in this invention outperforms existing face super-resolution methods on the face dataset CelebA based on sensory perception, improving sensory perception evaluation metrics (Fréchet inception distance (FID) by 15% and Learned Perceptual ImagePatch Similarity (LPIPS) by 14%), and has good generalization ability.

[0112] Although the above embodiments have been described, those skilled in the art, once they understand the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the above are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural transformations made using the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for improving the resolution of a face image, characterized in that, The method includes the following steps: S1: Receive the face image to be processed; S2: Recover some structural information of the face image through a spatial attention network to obtain an intermediate feature map; S3: Predict facial attribute information through the facial attribute prediction module, and predict prior information about the three-dimensional structure of the face through the three-dimensional face reconstruction branch module; S4: The intermediate feature map, the attribute information of the face, and the prior information of the three-dimensional structure of the face are fused to obtain a high-resolution face image; The method further includes: Training the face attribute prediction module specifically includes: cropping face attribute labels from face images to form a training set, which is then input into a first deep residual network for training; the first deep residual network is configured to modify the stride of the first convolutional kernel (7x7) in the network to 1, and to change the output of the final fully connected layer to 16 dimensions to correspond to the cropped face attribute information. The prediction of facial attribute information by the facial attribute prediction module includes: The intermediate feature map is input into the trained face attribute prediction module to obtain the corresponding face attribute information; The prior information for predicting the 3D structure of the face through the 3D face reconstruction branch module includes: Predict three-dimensional face parameters of an input face image, wherein the three-dimensional face parameters include one or more of identity, expression, texture, pose, and illumination; The final coordinates and color values ​​of each vertex of the model are calculated based on the theory of three-dimensional deformable human face model; Use a differentiable rendering tool to render a 3D face mesh onto the original input face image; Calculate the loss function between the original input face image and the rendered face image, and then backpropagate to update the network weights; Step S4 includes: The previous layer feature m i-1 The input is downsampled into a convolutional module F1 with a stride of 2 to obtain the feature map f1; The feature map f1 is compared with the feature map r of the same size output by the convolutional layer corresponding to the 3D face reconstruction branch module. i-1 By splicing the data along the channels, feature map f2 is obtained; The feature map corresponding to the face attribute prediction module is a i- 1. Input convolution calculation module A, calculate the attribute space attention sa through the sigmoid function, and multiply the attention sa with the feature map f2 pixel by pixel to obtain the feature map f3; m i-1 The input is a 3×3 convolution D with a stride of 2. This is downsampled and added pixel-by-pixel to the feature map f3 to calculate the residual, resulting in the output feature map m of this layer. i ; Based on feature map m i Obtain high-resolution facial images.

2. The method for improving the resolution of a face image as described in claim 1, characterized in that, The face image to be processed is a low-resolution image, and the process before step S2 includes: Interpolate low-resolution images to obtain blurred images of the desired resolution. Step S2 includes: The intermediate feature map is obtained by recovering some structural information of the blurred image through a spatial attention network.

3. The method for improving the resolution of a face image as described in claim 1, characterized in that, The spatial attention network includes an encoder, a feature processing layer, and a decoder in the order of image processing. The encoder is used to downsample the face image to reduce the width and height of the feature map and to initially extract the feature map; The feature processing layer is used to increase the depth of the network and process the downsampled feature map; The decoder is used to upsample the feature map processed by the feature processing layer to restore the original width and height of the face image.

4. The method for improving the resolution of a face image as described in claim 1 or 3, characterized in that, The spatial attention network adopts a funnel network structure.

5. The method for improving the resolution of a face image as described in claim 1, characterized in that, Step S4 is followed by step S5: The high-resolution face image is compared with the real image, and confidence information is output.

6. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed, implements the method as described in any one of claims 1 to 5.

7. An electronic device, characterized in that, include: The storage medium is the storage medium as described in claim 6; A processor, electrically connected to the storage medium, is configured to execute a computer program stored in the storage medium to implement the method as described in any one of claims 1 to 5.

Images