Face super-resolution method for occluded face image based on multi-level supervision attention mechanism super network and random erasing data enhancement
By constructing a multi-level supervised attention mechanism super-resolution network and a random erasure data augmentation method, the problem of super-resolution reconstruction of face images under occlusion conditions was solved, achieving high-resolution face image reconstruction with complete structure and clear details, thus improving the effectiveness of criminal investigation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF SCI & TECH
- Filing Date
- 2026-02-04
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies struggle to generate high-resolution facial images with complete structure, clear details, and high quality under occluded conditions, especially in criminal investigations where the restoration of low-resolution facial images is ineffective.
We employ a super-resolution network based on a multi-level supervised attention mechanism and a random erasure data augmentation method. Through a structure prior prediction module and a channel-space attention module, we construct a basic super-resolution completion sub-network, a fine-grained complex atom network, and a global optimization sub-network. Combined with data augmentation and multi-level supervised training, we reconstruct high-resolution face images.
It effectively overcomes the effects of occlusion, generating high-resolution facial images with complete structure, clear details, and high quality, thus improving the efficiency and accuracy of criminal investigations.
Smart Images
Figure CN122115209A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing and face super-resolution technology, specifically relating to a super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation. Background Technology
[0002] Face super-resolution (FSR) is an important branch of image super-resolution. Due to the fixed structure and complex texture of face images, FSR tasks are particularly challenging. FSR has significant research implications: in academia, FSR can effectively improve the accuracy of downstream face analysis tasks, such as face alignment, face parsing, face recognition, and 3D face reconstruction; in industry, FSR can provide high-resolution face images for systems such as intelligent transportation, identity verification, security monitoring, and sentiment analysis.
[0003] Currently, the field of criminal investigation has high requirements for facial image super-resolution. During criminal investigations, police typically compare images captured by cameras with information in databases to confirm the identity of suspects. However, due to poor camera angles, excessive shooting distances, or low camera resolution, it is difficult to obtain high-resolution facial images of suspects. Deploying a large number of cameras or upgrading the physical performance of camera equipment would significantly increase costs. Facial super-resolution reconstruction technology, however, can restore clear facial images from low-resolution images using only algorithmic models under existing hardware conditions, assisting police in solving cases more quickly.
[0004] Existing face image super-resolution methods can be broadly categorized into two main types: traditional super-resolution algorithms and deep learning-based super-resolution algorithms, which have emerged in recent years with the rapid development of deep learning technology. Traditional super-resolution algorithms primarily rely on image processing, statistical models, and prior knowledge. These methods achieved some success in early research, and their core idea is to utilize statistical information in the image and space to recover high-quality, high-resolution images through specific operations or optimization strategies. Deep learning-based super-resolution algorithms are characterized by using deep neural networks to learn the mapping relationship from low resolution to high resolution.
[0005] Despite some progress in face super-resolution tasks, super-resolution of face images under occlusion conditions remains a highly challenging problem. Due to the significant loss of useful information in occluded low-resolution images, it is difficult to directly extract sufficient information to determine an accurate high-resolution face image. Therefore, given an occluded low-resolution face image, it is easy to generate multiple corresponding high-resolution face images. Summary of the Invention
[0006] To overcome the shortcomings of the existing technologies, this invention provides a super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation. Based on the structure prior prediction module and the channel-space attention module, this invention studies a more accurate, efficient, and universal face super-resolution method, enabling the gradual reconstruction of structurally complete, detailed, and high-quality face images from occluded low-resolution face images.
[0007] To achieve the above objectives, the technical solution of the present invention is: a super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation, comprising the following steps:
[0008] S1: Construct the structure prior prediction module, channel-space attention module, upsampling module, and residual module respectively;
[0009] S2: Concatenate input convolutional layers, channel-space attention modules, upsampling modules, and residual modules to construct a basic super-resolution completion sub-network; cascade input convolutional layers, channel-space attention modules, upsampling modules, structure prior prediction modules, and residual modules to construct a fine-grained complex atom network and a global optimization sub-network; then cascade the basic super-resolution completion sub-network, the fine-grained complex atom network, and the global optimization sub-network to construct a multi-level supervised attention mechanism super-resolution network;
[0010] S3: For high-resolution face images in the public dataset, perform face region cropping, random facial content erasure, and Bicubic downsampling to construct face image pairs with occlusion (low resolution) and no occlusion (high resolution); the image pairs are divided into training and test sets according to a predetermined ratio, and random content erasure data augmentation is applied to the training set.
[0011] S4: An occluded face super-resolution model based on a multi-level supervised attention mechanism super-resolution network trained on the training set;
[0012] S5: Input the low-resolution occluded face images from the test set into the trained occluded face super-resolution model, and output the reconstructed high-resolution unoccluded face images.
[0013] Furthermore, the construction process of the structural prior estimation module in step S1 is as follows:
[0014] A1: Construct a multi-scale feature extraction module, which includes a first batch of normalization layers, a first convolutional layer (3×3 kernel size, stride 1, padding 1), a second batch of normalization layers, a second convolutional layer, a third batch of normalization layers, and a third convolutional layer arranged in sequence; wherein, the number of output feature channels of the first, second, and third convolutional layers are half, a quarter, and a quarter of the number of input channels, respectively; the module concatenates the outputs of the three convolutional layers in the channel dimension to form a fused feature, and adds the fused feature to the original input feature element by element to achieve channel compression from coarse to fine and multi-scale information extraction;
[0015] A2: Construct an hourglass network module, employing a recursive feature flow mechanism that proceeds from top to bottom and from bottom to top. In the top-down path, high-resolution features are progressively downsampled, and deep semantic information is extracted layer by layer through a multi-scale feature extraction module. In the bottom-up path, low-resolution input features are progressively upsampled to restore spatial resolution, and detailed information is extracted layer by layer through a multi-scale feature extraction module. This extracted detail information is then added element-wise to the high-resolution features from the top-down path, achieving multi-scale fusion of local details and global semantics, thereby enhancing the network's ability to capture structural information and detailed features.
[0016] A3: Constructing a Stacked Hourglass Network: Input features are initially encoded using an input convolutional layer (7×7 kernel size, stride 2, padding 3). Multiple hourglass network modules are then stacked to achieve top-down and bottom-up multi-scale feature flows. Within each hourglass module, global semantic information is extracted through downsampling, spatial resolution is restored through upsampling, and features are fused element-wise with those from high-resolution paths to achieve joint modeling of local details and global contextual information. The output features of each hourglass network module undergo channel adjustment via convolutional layers and normalization operations. Finally, a 1×1 convolution generates heatmap features of facial key points, and ground truth supervision is applied to these heatmap features. To enhance cross-module information flow, the output of each hourglass module is accumulated with the current feature and the channel-mapped heatmap prediction before being fed into the next module, thus forming a deep supervision mechanism and multi-level residual connections.
[0017] Furthermore, the construction process of the channel-space attention module in step S1 is as follows:
[0018] B1: Construct a channel attention module, which receives input features and performs average pooling and max pooling respectively to obtain two channel attention features; input the two channel attention features into a multilayer perceptron module (containing one receiving convolutional layer, one PReLU layer and one output convolutional layer) to output channel attention enhancement features; sum the two channel attention enhancement features and pass them through a sigmoid function to obtain channel attention weights, which are then multiplied pixel by pixel with the input features to construct effective channel enhancement features;
[0019] B2: Construct a spatial attention module, perform average pooling and max pooling on the effective channel enhancement features along the channel dimension to obtain spatial enhancement features; after concatenation along the channels, pass through the output convolutional layer (convolutional kernel size is 7×7, stride is 2, padding is 3) and pass through Sigmoid to obtain a spatial attention map, multiply it element-wise with the effective channel enhancement features to further realize spatial enhancement, and serve as the final output of the module.
[0020] Furthermore, the construction process of the basic super-resolution completion sub-network in step S2 is as follows:
[0021] C1: Constructing the residual module: The residual module consists of a convolutional layer (with a kernel size of 3×3, a stride of 1, and padding of 1), a batch layer, a ReLU activation function, and skip connections;
[0022] C2: Constructing the Basic Super-Resolution Completion Sub-Network: The basic super-resolution completion sub-network consists of convolutional layers, channel-spatial attention modules, upsampling layers, and residual modules. First, one input convolutional layer is constructed (kernel size 3×3, stride 1, padding 1, output channels 64); then, one channel-spatial attention module and one upsampling layer are cascaded; finally, three consecutive residual modules and one output convolutional layer (kernel size 3×3, stride 1, padding 1, output channels 3) are cascaded to construct the image reconstruction module, outputting a preliminary super-resolution and completed face image.
[0023] The construction process of the fine-grained complex atom network is as follows: First, an input convolutional layer is constructed with a kernel size of 3×3, a stride of 1, padding of 1, and an output channel of 64. Then, a channel-based spatial attention module, an upsampling layer, and a face prior prediction module are cascaded. Finally, the face features and facial structure prior features output by the upsampling layer are processed by channel-dimensional cascading and input into the residual module, outputting a restored fine-grained face image.
[0024] The construction process of the global optimization sub-network is as follows: First, an input convolutional layer is constructed with a kernel size of 3×3, a stride of 1, padding of 1, and an output channel of 64. Then, a channel-based spatial attention module, an upsampling layer, and a face prior prediction module are cascaded. Finally, the face features and facial structure prior features output by the upsampling layer are processed by channel-dimensional cascading and input into the residual module to output a globally complete face image.
[0025] Furthermore, the construction process of the multi-level supervised attention mechanism super-resolution network in step S2 is as follows:
[0026] D1: Construct a basic super-resolution completion sub-network to perform preliminary reconstruction of the input low-resolution face image to generate a coarse super-resolution and completion result; cascade a fine-grained complex atom network to perform fine-grained feature enhancement on the coarsely reconstructed face image to obtain a high-resolution face image with enhanced details.
[0027] D2: A cascaded global optimization subnetwork receives high-resolution face images with enhanced details and generates the final high-quality face image; at the same time, a discriminator module is constructed to act on the final high-quality face image to improve the structural consistency and perceptual quality of the reconstructed image.
[0028] Furthermore, the construction process in step S3 is specifically as follows:
[0029] E1: Preprocess the published Celeba face dataset, perform face detection and cropping on all face images, and construct high-resolution face images with the facial region occupying more than 80% of the entire image as ground truth high-resolution unoccluded face images;
[0030] E2: Define the occlusion area in the high-resolution unoccluded face image, erase irregularly shaped content in the occlusion area along random directions, update a white mask image to mark the occluded content, and set random colors for the pixels of the occluded content;
[0031] E3: Use the Bicubic interpolation method to downsample and obtain a low-resolution image corresponding to the occluded face image;
[0032] E4: Pair the generated low-resolution occluded face images with high-resolution unoccluded face images to form training or test sample pairs;
[0033] E5: Store the training and test sets separately according to the rule of dividing the training set and the test set (90% training and 10% test), ensuring that each low-resolution occluded face image corresponds to a unique high-resolution unoccluded face image, and repeatedly perform random content erasure on the training set to achieve training data augmentation.
[0034] Furthermore, the construction process in step S4 is specifically as follows:
[0035] F1: Construct the mean squared error as the loss function;
[0036] F2: Construct a perceptual loss function based on the VGG19 model;
[0037] F3: Construct a face landmark heatmap loss function based on a 2D-FAN model;
[0038] F4: Constructing the adversarial loss function;
[0039] F5: Initialize the parameters of the basic super-completion subnetwork, fine-grained complex atom network, and global optimization subnetwork in the multi-level supervised attention mechanism super-resolution network;
[0040] F6: Combine all the above loss functions to perform multi-level supervised training on the output of all sub-networks;
[0041] F7: Set the learning rate, number of model training iterations, and number of batch training samples;
[0042] F8: Train the multi-level supervised attention super-resolution network using low-resolution occluded images and corresponding high-resolution unoccluded images from the training dataset, and update the network parameters using an optimization algorithm;
[0043] F9: Train the super-resolution model for occluded face images until the number of iterations reaches a preset value.
[0044] Furthermore, the perceptual loss function based on the VGG19 model in step F2 is:
[0045]
[0046] in, This represents the joint distribution of the super-resolution results and the corresponding ground truth faces of each sub-network. This represents the feature map extracted by the pre-trained VGG19 model from the convolutional layer (after activation) before all its max-pooling layers. Represents a true high-resolution image. This represents the generated super-resolution image.
[0047] Furthermore, the face landmark heatmap loss function based on the 2D-FAN model in step F3 is:
[0048]
[0049] Where K represents the number of key points (e.g., 68 facial key points). This represents the super-resolution result generated by the structure prior prediction module acting on the fine-grained complex atom network and the global optimization sub-network, showing the predicted heatmap of the k-th key point. The ground truth heatmap of the k-th key point is generated by applying a 2D-FAN model to a ground truth high-resolution face image.
[0050] Furthermore, the adversarial loss function in step F4 is:
[0051]
[0052]
[0053] in, To constrain the adversarial loss function of the globally optimized subnetwork, Let E be the adversarial loss function constraining the discriminator, used to train the discriminator to distinguish between real and generated images, and let D be the average value over the samples. This represents the output of the discriminator model, indicating the probability that the input is "true" (typically in the range [0, 1]).
[0054] Compared with the prior art, the significant advantages of the present invention are: the face super-resolution model constructed by the present invention overcomes the problems of image structure distortion and numerous artificial artifacts under the influence of occlusion in the existing face super-resolution methods, enabling the present invention to reconstruct a face image with complete structure, clear details, and high quality. Attached Figure Description
[0055] Figure 1 This is a schematic diagram of the method flow of the present invention.
[0056] Figure 2 This is a schematic diagram of a super-resolution network structure for a multi-level supervisory attention mechanism.
[0057] Figure 3 This is a schematic diagram of the structure of a facial prior prediction network.
[0058] Figure 4 This is a schematic diagram of the channel-space attention module structure.
[0059] Figure 5 This is a diagram illustrating the comparison of test results with those of other similar methods. Detailed Implementation
[0060] This invention proposes a super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation, comprising the following steps: cropping face regions, randomly erasing facial content, and resizing a publicly available face dataset to construct a low-resolution occluded face / high-resolution unoccluded face image pair dataset; dividing the dataset into training and test sets; and performing multiple data augmentations on the training set; constructing a multi-level supervised attention mechanism face image super-resolution network guided by facial structure priors; training the multi-level supervised attention mechanism face image super-resolution network on the training set; and performing super-resolution on low-resolution occluded face images in the test set. This invention constructs an occluded face image super-resolution model based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation, whose super-resolution network consists of a basic super-resolution completion sub-network, a fine-grained complex atom network, and a global optimization sub-network. The super-resolution model for occluded face images constructed in this invention overcomes the problems of structural distortion and artifacts in the results of existing face super-resolution methods when processing low-resolution face images with occlusion. This invention enables the reconstruction of high-resolution face images with complete structure, accurate details, and high fidelity. The invention will be further explained below with reference to the accompanying drawings and specific embodiments.
[0061] This invention provides a super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation, referring to... Figure 1 It includes the following steps:
[0062] Step 1: Construct the structural prior prediction module, which includes steps 1.1-1.2:
[0063] Step 1.1: Construct a multi-scale feature extraction module;
[0064] Step 1.2: Construct the hourglass network module;
[0065] Step 1.2: Construct a stacked hourglass network to generate heatmap features of facial key points, with the specific structure as follows: Figure 3 As shown;
[0066] Step 2: Construct the channel-space attention module, which includes steps 2.1-2.3:
[0067] Step 2.1: Construct a channel attention module to receive features. Perform average pooling and max pooling respectively to obtain the dimension as The channel attention features are obtained; they are input into a shared multilayer perceptron to output channel attention enhancement features; the two channel attention enhancement features are summed and passed through a sigmoid function to obtain the channel attention weights, which are then multiplied pixel-by-pixel with the input features to construct effective channel enhancement features;
[0068] Step 2.2: Construct a spatial attention module, performing average pooling and max pooling on the effective channel enhancement features along the channel dimension to obtain a dimension of Features; after concatenation along the channels, the spatial attention map is obtained through the output convolutional layer and then through the sigmoid function. This attention map is then multiplied element-wise with the effective channel enhancement features to further enhance spatial attention, and serves as the final output of the module. The specific structure is as follows: Figure 4 As shown.
[0069] Step 3: Construct the basic super-resolution completion sub-network, which includes steps 3.1-3.2:
[0070] Step 3.1: Constructing the residual module: The residual module consists of a convolutional layer with a kernel size of 3*3 and a stride of 1, a batch processing layer, a ReLU activation function, and skip connections;
[0071] Step 3.2: Constructing the basic super-resolution and completion sub-network: First, construct one input convolutional layer (kernel size 3×3, stride 1, padding 1, output channels 64); then cascade one channel—a spatial attention module—and one upsampling layer; finally, cascade three consecutive residual modules and one output convolutional layer (kernel size 3×3, stride 1, padding 1, output channels 3) to construct the image reconstruction module, outputting a preliminary super-resolution and completed face image. The specific structure is as follows: Figure 2 As shown.
[0072] Step 4: Construct a fine-grained complex atom network, which includes step 4.1:
[0073] Step 4.1: First, construct one input convolutional layer (kernel size 3×3, stride 1, padding 1, output channels 64); then cascade one channel-based spatial attention module, one upsampling layer, and one face prior prediction module; finally, perform channel-dimensional cascade processing on the face features and facial structure prior features output from the upsampling layer, and input them together into the residual module to output a restored fine-grained face image. The specific structure is as follows: Figure 2 As shown.
[0074] Step 5: Construct a global optimization subnetwork, which includes step 5.1:
[0075] Step 5.1: First, construct one input convolutional layer (kernel size 3×3, stride 1, padding 1, output channels 64); then cascade one channel-based spatial attention module, one upsampling layer, and one face prior prediction module; finally, perform channel-dimensional cascade processing on the face features and facial structure prior features output from the upsampling layer, and input them together into the residual module, outputting a fully restored global face image. The specific structure is as follows: Figure 2 As shown.
[0076] Step 6: Construct a super-resolution network based on a multi-level supervised attention mechanism, which includes steps 6.1-6.2:
[0077] Step 6.1: Construct a basic super-resolution completion sub-network to reconstruct a coarse super-resolution result; cascade a fine-grained complex atom network to obtain a high-resolution face image with enhanced details;
[0078] Step 6.2: Construct a global optimization sub-network to generate the final high-quality face image; construct a discriminator module to be applied to the final high-quality face image to improve the structural consistency and perceptual quality of the reconstructed image.
[0079] Step 7: Preprocess the dataset, splitting it into the dataset and the test set, which includes steps 7.1-7.5:
[0080] Step 7.1: Preprocess the published Celeba face dataset, perform face detection and cropping on all face images, and construct high-resolution face images with the facial region occupying more than 80% of the entire image as ground truth high-resolution unoccluded face images;
[0081] Step 7.2: Define the occlusion area in the high-resolution unoccluded face image, erase irregularly shaped content in the occlusion area along random directions, and at the same time, update a white mask image to mark the occluded content and set random colors for the pixels of the occluded content;
[0082] Step 7.3: Use the Bicubic interpolation method to downsample and obtain the low-resolution image corresponding to the occluded face image;
[0083] Step 7.4: Pair the generated low-resolution occluded face images with high-resolution unoccluded face images to form training or test sample pairs;
[0084] Step 7.5: Store the training and test sets separately according to the division rules (90% training and 10% test) to ensure that each low-resolution occluded face image corresponds to a unique high-resolution unoccluded face image, and repeatedly perform random content erasure on the training set to enhance the training data.
[0085] Step 8: Train a super-resolution model for occluded face images based on a multi-level supervised attention mechanism super-resolution network, which includes steps 8.1-8.9:
[0086] Step 8.1: Construct a loss function based on mean squared error to impose pixel-level constraints on the super-resolution face images output by each sub-network;
[0087] Step 8.2: Construct the perceptual loss function based on the VGG19 model, the specific structure of which is as follows:
[0088]
[0089] in, This represents the joint distribution of the super-resolution results and the ground truth face. This represents the feature map extracted by the pre-trained VGG19 model from the convolutional layer (after activation) before all its max-pooling layers. Represents a true high-resolution image. This represents the generated super-resolution image.
[0090] Step 8.3: Construct a face landmark heatmap loss function based on a 2D-FAN model, the specific structure of which is as follows:
[0091]
[0092] Where K represents the number of key points (e.g., 68 facial key points). This represents the super-resolution result generated by the structure prior prediction module acting on the fine-grained complex atom network and the global optimization sub-network, showing the predicted heatmap of the k-th key point. The ground truth heatmap of the k-th key point is generated by applying a 2D-FAN model to a ground truth high-resolution face image.
[0093] Step 8.4: Construct the adversarial loss function, the specific structure of which is as follows:
[0094]
[0095]
[0096] in, To constrain the adversarial loss function of the globally optimized subnetwork, Let E be the adversarial loss function constraining the discriminator, used to train the discriminator to distinguish between real and generated images, and let D be the average value over the samples. This represents the output of the discriminator model, indicating the probability that the input is "true" (typically in the range [0, 1]).
[0097] Step 8.5: Initialize the parameters of the basic super-completion sub-network, fine-grained complex atom network, and global optimization sub-network in the multi-level supervised attention mechanism super-resolution network;
[0098] Step 8.6: Combine all the above loss functions and perform multi-level supervised training on the output results of all sub-networks;
[0099] Step 8.7: Set the learning rate, the number of model training iterations, and the number of training samples per batch;
[0100] Step 8.8: Train the multi-level supervised attention mechanism super-resolution network using low-resolution occluded images and corresponding high-resolution unoccluded images from the training dataset, and update the network parameters using an optimization algorithm;
[0101] Step 8.9: Train the super-resolution model for occluded face images until the number of iterations reaches a preset value.
[0102] Step 9: Input the images from the test dataset into the trained super-resolution model for occluded face images based on a multi-level supervised attention mechanism super-resolution network, and output the reconstructed high-resolution occluded face images.
[0103] In this embodiment, the face image finally output by the above method is compared with the face images obtained by other methods. The comparison result is as follows: Figure 5 As shown, from left to right, the images are: low-resolution occluded face image, SRGAN, FHC, SR3, PGDIFF, SFMNET, DIFFACE, WFEN, the result of this invention, and the original high-resolution image. The comparison demonstrates that the high-resolution face image reconstructed by this invention has a more complete structure, richer texture details, and higher realism in visual perception.
[0104] The specific embodiments of the present invention have been described in detail above. It should be noted that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.
Claims
1. A super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation, characterized in that, Includes the following steps: S1: Construct the structure prior prediction module, channel-space attention module, upsampling module, and residual module respectively; S2: Concatenate input convolutional layers, channel-space attention modules, upsampling modules, and residual modules to construct a basic super-resolution completion sub-network; cascade input convolutional layers, channel-space attention modules, upsampling modules, structure prior prediction modules, and residual modules to construct a fine-grained complex atom network and a global optimization sub-network; then cascade the basic super-resolution completion sub-network, the fine-grained complex atom network, and the global optimization sub-network to construct a multi-level supervised attention mechanism super-resolution network; S3: For high-resolution face images in the public dataset, perform face region cropping, random facial content erasure, and Bicubic downsampling to construct face image pairs with occlusion (low resolution) and no occlusion (high resolution); the image pairs are divided into training and test sets according to a predetermined ratio, and random content erasure data augmentation is applied to the training set. S4: An occluded face super-resolution model based on a multi-level supervised attention mechanism super-resolution network trained on the training set; S5: Input the low-resolution occluded face images from the test set into the trained occluded face super-resolution model, and output the reconstructed high-resolution unoccluded face images.
2. The super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation as described in claim 1, characterized in that, The construction process of the structural prior prediction module in step S1 is as follows: A1: Construct a multi-scale feature extraction module, which includes a first batch of normalization layers, a first convolutional layer, a second batch of normalization layers, a second convolutional layer, a third batch of normalization layers, and a third convolutional layer arranged in sequence; wherein, the number of output feature channels of the first, second, and third convolutional layers are half, a quarter, and a quarter of the number of input channels, respectively; the module concatenates the outputs of the three convolutional layers in the channel dimension to form a fused feature, and adds the fused feature to the original input feature element by element to achieve channel compression from coarse to fine and multi-scale information extraction; A2: Construct an hourglass network module, employing a top-down and bottom-up recursive feature flow mechanism. In the top-down path, high-resolution features are progressively downsampled, and deep semantic information is extracted layer by layer through a multi-scale feature extraction module. In the bottom-up path, low-resolution input features are progressively upsampled to restore spatial resolution, and detailed information is extracted layer by layer through a multi-scale feature extraction module. This is then added element by element to the high-resolution features from the top-down path, achieving multi-scale fusion of local details and global semantics. A3: Constructing a stacked hourglass network: Input features are initially encoded using input convolutional layers; then, multiple hourglass network modules are stacked to achieve multi-scale feature flow from top to bottom and bottom to top. Within each hourglass module, global semantic information is extracted through downsampling, spatial resolution is restored through upsampling, and features are fused element-wise with those from high-resolution paths to achieve joint modeling of local details and global contextual information. The output features of each hourglass network module undergo channel adjustment through convolutional layers and normalization operations, and finally, heatmap features of facial key points are generated through 1×1 convolution, with ground truth supervision set for the heatmap features. To enhance cross-module information flow, the output of each hourglass module is accumulated with the current features and the channel-mapped heatmap prediction before being fed into the next module, thus forming a deep supervision mechanism and multi-level residual connections.
3. The super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation as described in claim 1, characterized in that, The construction process of the channel-space attention module in step S1 is as follows: B1: Construct a channel attention module, receive input features and perform average pooling and max pooling respectively to obtain two channel attention features; input the two channel attention features into the multilayer perceptron module respectively, and output channel attention enhancement features; The attention enhancement features of the two channels are summed and passed through a Sigmoid function to obtain the channel attention weights. These weights are then multiplied pixel-by-pixel with the input features to construct effective channel enhancement features. B2: Construct a spatial attention module to perform average pooling and max pooling on the channel dimension of the effective channel augmentation features to obtain spatial augmentation features; After concatenation along the channels, the spatial attention map is obtained by passing it through the output convolutional layer and the Sigmoid function. It is then multiplied element-wise with the effective channel enhancement features to achieve spatial enhancement, which is used as the final output of the module.
4. The method for super-resolution of occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation as described in claim 1, characterized in that, The construction process of the basic super-resolution completion subnetwork in step S2 is as follows: C1: Constructing the residual module: The residual module consists of convolutional layers, batch layers, ReLU activation functions, and skip connections; C2: Constructing the basic super-resolution completion sub-network: The basic super-resolution completion sub-network consists of convolutional layers, channel-space attention modules, upsampling layers, and residual modules. First, one input convolutional layer is constructed; then, one channel-space attention module and one upsampling layer are cascaded; finally, three consecutive residual modules and one output convolutional layer are cascaded to construct the image reconstruction module, which outputs a preliminary super-resolution and completed face image. The construction process of the fine-grained complex atom network is as follows: First, an input convolutional layer is constructed with a kernel size of 3×3, a stride of 1, padding of 1, and an output channel of 64. Then, a channel-based spatial attention module, an upsampling layer, and a face prior prediction module are cascaded. Finally, the face features and facial structure prior features output by the upsampling layer are processed by channel-dimensional cascading and input into the residual module, which outputs a restored fine-grained face image. The construction process of the global optimization sub-network is as follows: First, construct an input convolutional layer with a kernel size of 3×3, a stride of 1, padding of 1, and an output channel of 64; then cascade a channel-based spatial attention module, an upsampling layer, and a face prior prediction module; finally, perform channel-dimensional cascade processing on the face features and facial structure prior features output by the upsampling layer, input them together into the residual module, and output a globally complete face image.
5. The super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation as described in claim 4, characterized in that, The construction process of the multi-level supervised attention mechanism super-resolution network in S2 is as follows: D1: Construct a basic super-resolution completion sub-network to perform preliminary reconstruction of the input low-resolution face image to generate a coarse super-resolution and completion result; cascade a fine-grained complex atom network to perform fine-grained feature enhancement on the coarsely reconstructed face image to obtain a high-resolution face image with enhanced details. D2: A cascaded global optimization subnetwork receives high-resolution face images with enhanced details and generates the final high-quality face image; at the same time, a discriminator module is constructed to act on the final high-quality face image.
6. The method for super-resolution of occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation as described in claim 1, characterized in that, The process of step S3 is as follows: E1: Preprocess the published Celeba face dataset, perform face detection and cropping on all face images, and construct high-resolution face images with the facial region occupying more than 80% of the entire image as ground truth high-resolution unoccluded face images; E2: Define the occlusion area in the high-resolution unoccluded face image, erase irregularly shaped content in the occlusion area along random directions, update a white mask image to mark the occluded content, and set random colors for the pixels of the occluded content; E3: Use the Bicubic interpolation method to downsample and obtain a low-resolution image corresponding to the occluded face image; E4: Pair the generated low-resolution occluded face images with high-resolution unoccluded face images to form training or test sample pairs; E5: Store the training and test sets separately according to the rules for dividing the set, ensuring that each low-resolution occluded face image corresponds to a unique high-resolution unoccluded face image, and repeatedly perform random content erasure on the training set to enhance the training data.
7. The super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation as described in claim 1, characterized in that, The process of step S4 is as follows: F1: Construct the mean squared error as the loss function; F2: Construct a perceptual loss function based on the VGG19 model; F3: Construct a face landmark heatmap loss function based on a 2D-FAN model; F4: Constructing the adversarial loss function; F5: Initialize the parameters of the basic super-completion subnetwork, fine-grained complex atom network, and global optimization subnetwork in the multi-level supervised attention mechanism super-resolution network; F6: Combine all the above loss functions to perform multi-level supervised training on the output of all sub-networks; F7: Set the learning rate, number of model training iterations, and number of batch training samples; F8: Train the multi-level supervised attention super-resolution network using low-resolution occluded images and corresponding high-resolution unoccluded images from the training dataset, and update the network parameters using an optimization algorithm; F9: Train the super-resolution model for occluded face images until the number of iterations reaches a preset value.
8. The super-resolution method for occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation as described in claim 7, characterized in that, The perceptual loss function based on the VGG19 model in step F2 for: ； in, This represents the joint distribution of the super-resolution results and the corresponding ground truth faces of each sub-network. This represents the feature map extracted by the pre-trained VGG19 model from the convolutional layers before all its max-pooling layers. Represents a true high-resolution image. This represents the generated super-resolution image.
9. The method for super-resolution of occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation according to claim 7, characterized in that, The face landmark heatmap loss function based on the 2D-FAN network in step F3 for: ； Where K represents the number of key points, This represents the super-resolution result generated by the structure prior prediction module acting on the fine-grained complex atom network and the global optimization sub-network, showing the predicted heatmap of the k-th key point. The ground truth heatmap of the k-th key point is generated by applying a 2D-FAN model to a ground truth high-resolution face image.
10. The method for super-resolution of occluded face images based on a multi-level supervised attention mechanism super-resolution network and random erasure data augmentation according to claim 7, characterized in that, The adversarial loss function in step F4 is: ； ； in, To constrain the adversarial loss function of the globally optimized subnetwork, Let E be the adversarial loss function constraining the discriminator, used to train the discriminator to distinguish between real and generated images, and let D be the average value over the samples. This represents the output of the discriminator model, which indicates the probability that the input is "true".