An ultra-lightweight low-resolution dark-light face enhancement method and system

Abstract

This invention discloses an ultra-lightweight, low-resolution, low-light face enhancement method and system. The method comprises: extracting features from an input low-resolution, low-light face image I to obtain multiple feature maps, which are then fused to obtain an enhanced feature map; extracting deep features from the enhanced feature map using a series of convolutional neural networks (CNNs) to obtain a deep feature map; fusing the deep feature map with the corresponding upsampled low-resolution, low-light face image I to obtain a high-quality feature map, which is then further convolved with sub-pixel data to obtain a high-quality, clear face image; training a face enhancement network by minimizing the Euclidean distance between the high-quality, clear face image and its corresponding true clear face gold standard; and enhancing test data using the trained network, which is then evaluated. The effectiveness of the method has been demonstrated through extensive experiments and applied to low-quality, low-light face image enhancement.

Description

Technical Field

[0001] The present invention relates to image enhancement technology, and in particular to an ultra-lightweight low-resolution dark-light face enhancement method and system. Background Art

[0002] Generally, high-quality, clear facial images provide accurate and effective information for subsequent tasks such as visual perception and emotion analysis. However, existing photosensitive devices often record facial images in dimly lit scenes, often exhibiting low contrast, color distortion, and blur. This severely degrades the performance of downstream algorithms for tasks such as face recognition, sparse face representation, and facial emotion analysis. To address this issue, low-light image enhancement methods are widely used on various mobile and edge devices. They attempt to restore images from dim backgrounds to clear images with high dynamic range (HDR). From a technical perspective, these methods can be roughly divided into two categories: traditional algorithms and deep learning-based algorithms. Traditional algorithms empirically reconstruct dim images into HDR images using artificial priors and statistical methods. These learning methods often perform well in qualitative analysis but poorly in quantitative analysis. In contrast, deep learning methods demonstrate impressive performance in both quantitative and qualitative analysis. In particular, they can reconstruct lost contours and details through nonlinear inference for image texture restoration.

[0003] While existing deep learning models excel at reconstructing high-quality facial images in dimly lit environments, their size (most exceeding 1MB) often falls short of current application requirements. This is especially true for embedded devices, where storage space needs to account for operating systems, sensor read / write capabilities, and other factors. Commonly used deep learning models simply cannot run on these resource-constrained devices. Summary of the Invention

[0004] Purpose of the invention: One purpose of the present invention is to provide an ultra-lightweight low-resolution dark-light face enhancement method, which pays attention to the low-light face enhancement effect while ensuring the fastest execution speed of the model to the greatest extent.

[0005] Another object of the present invention is to provide an ultra-lightweight low-resolution low-light face enhancement system.

[0006] Technical solution: The present invention provides an ultra-lightweight, low-resolution, low-light face enhancement method, comprising the following steps:

[0007] Construct a face enhancement network model for low-resolution low-light face enhancement and train it using a face image dataset; use the trained face enhancement network model to predict the collected face image test samples to obtain high-quality clear face images, which are then evaluated using evaluation indicators;

[0008] The face enhancement network model includes: a data enhancement module, several series-connected deep feature extraction modules and a high-quality feature fusion module. First, the data enhancement module performs parallel convolution feature extraction on the input low-resolution dark light face image I, and the obtained multiple feature maps are spliced ​​and convolutionally fused to obtain an enhanced feature map F'; the enhanced feature map F' is subjected to deep feature extraction by several series-connected deep feature extraction modules to obtain a deep feature map F d ; Through the high-quality feature fusion module, the deep feature map F d Further processing, the specific operation is: through an information fusion module to transform the deep feature map F d It is fused with the corresponding upsampled low-resolution dark-light face image I, and then further passed through a convolution layer to obtain a high-quality feature map F h , perform sub-pixel convolution on the high-quality feature map Fh to obtain a high-quality clear face image Y pred .

[0009] Furthermore, before extracting features from the low-resolution dark-light face image I, it is necessary to perform a normalization operation on the data samples in the face image dataset, normalizing the data range of the data samples to between 0 and 1.

[0010] Furthermore, the data enhancement module includes n parallel convolutional layers, which use n parallel convolutional layers to extract features from the low-resolution dark-light face image I to obtain n feature maps, and then obtain a feature map matrix F composed of n feature maps through a splicing operation. s ={F1, F2, ..., F n}, F1, F2, …, F n These are the feature maps obtained by extracting features from the 1st to the nth convolutional layers respectively;

[0011] Then, a convolutional layer is used to transform the feature map matrix F s After fusion, the enhanced feature map F' is obtained as follows:

[0012] F'=CNN 3*3 (F s )

[0013] Among them, CNN 3*3 Is to deal with F s The convolutional layer of , F' represents the enhanced feature map.

[0014] Furthermore, several series-connected deep feature extraction modules each include a channel separation module, m parallel convolution layers and a convolution fusion module connected in sequence, and several series-connected deep feature extraction modules are used to extract deep features from the enhanced feature map F' in sequence to obtain a deep feature map F d , specifically:

[0015] The first deep feature extraction module extracts deep features from the enhanced feature map F': first, the channel separation module performs channel separation on the enhanced feature map F' to enhance the features of facial images in different channel domains; then, the m separated feature maps are respectively subjected to 3*3 convolution and activation through m parallel convolutional layers to achieve deep feature extraction; finally, the feature maps after 3*3 convolution are spliced ​​to achieve channel fusion, and fused using a 1*1 convolution kernel to obtain a fused feature map;

[0016] The remaining series-connected deep feature extraction modules perform channel separation, deep feature extraction, and convolution fusion on the output results of the previous deep feature extraction module in turn, and finally obtain a deep feature map F d :

[0017] F d =CNN 1*1 (CON(CNN 3*3 (SP(F′)));

[0018] Among them, F' is the enhanced feature map, SP is the channel separation operation, CNN 3*3 and CNN 1*1 All are convolutional layers, and CON represents the channel fusion layer.

[0019] Furthermore, after the face enhancement network model training is completed, the trained face enhancement network model is further quantized through a dynamic quantization algorithm to obtain a quantized face enhancement network model; and the quantized face enhancement network model is used to predict the collected face image test samples.

[0020] Furthermore, the Euclidean distance is used to train the face enhancement network model, specifically:

[0021] High-quality clear face image Y pred and the corresponding real clear face gold standard Y gt The Euclidean distance between them is:

[0022]

[0023] Among them, Y pred represents the face image predicted by the network, Y gt Represents the face image Y predicted by the network predThe corresponding real clear face gold standard, Dis(Y pred , Y gt ) represents the face image Y pred and Y gt The Euclidean distance, Y i pred and Y i gt They respectively represent the faces belonging to the face image Y pred and Y gt Pixels of

[0024] Use the loss function to express the objective function of face enhancement and complete the training of the face enhancement network model;

[0025] The loss function calculated by Euclidean distance is:

[0026] L(Y pred , Y gt )=Dis(Y pred , Y gt );

[0027] Among them, Dis(Y pred , Y gt ) represents the high-quality clear face image Y output by the model pred and the corresponding real clear face gold standard Y gt The Euclidean distance between .

[0028] Furthermore, the evaluation indicators include peak signal-to-noise ratio PSNR and structural similarity SSIM;

[0029] The peak signal-to-noise ratio (PSNR) is calculated as follows:

[0030]

[0031] Among them, MAX I Indicates the maximum value of the image point color;

[0032] The calculation method of structural similarity SSIM is:

[0033]

[0034] in, is the high-quality clear face image Y output by the model pred The average value of Yes and Y pred The corresponding real clear face gold standard Y gt The average value of It's Y pred The variance of It's Y gt The variance of Represents Y pred Standard deviation and Y gt Standard deviation The product of , c1 and c2 are smoothing parameters.

[0035] Based on the same inventive concept, the present invention provides an ultra-lightweight, low-resolution, low-light face enhancement system, comprising:

[0036] Model building and training unit: builds a face enhancement network model for low-resolution low-light face enhancement and trains it using a face image dataset;

[0037] And a test evaluation module, which is used to use the trained face enhancement network model to predict the collected face image test samples, obtain high-quality clear face images, and evaluate them using evaluation indicators;

[0038] The face enhancement network model includes: a data enhancement module, several series-connected deep feature extraction modules and a high-quality feature fusion module. First, the data enhancement module performs parallel convolution feature extraction on the input low-resolution dark light face image I, and the obtained multiple feature maps are spliced ​​and convolutionally fused to obtain an enhanced feature map F'; the enhanced feature map F' is subjected to deep feature extraction by several series-connected deep feature extraction modules to obtain a deep feature map F d ; Through the high-quality feature fusion module, the deep feature map F d Further processing, the specific operation is: through an information fusion module to transform the deep feature map F d It is fused with the corresponding upsampled low-resolution dark-light face image I, and then further passed through a convolution layer to obtain a high-quality feature map F h , perform sub-pixel convolution on the high-quality feature map Fh to obtain a high-quality clear face image Y pred .

[0039] Based on the same inventive concept, the present invention provides an ultra-lightweight, low-resolution, low-light face enhancement device, including a memory and a processor, wherein:

[0040] a memory for storing computer programs capable of running on the processor;

[0041] The processor is configured to execute the steps of the above-mentioned ultra-lightweight low-resolution low-light face enhancement method when running the computer program.

[0042] Based on the same inventive concept, the present invention provides a storage medium having a computer program stored thereon, which, when executed by at least one processor, implements the steps of the above-mentioned ultra-lightweight, low-resolution, low-light face enhancement method.

[0043] Beneficial effects: Compared with the existing technology, the method of the present invention restores a high-quality face image by integrating traditional formulas with limited convolution blocks; specifically, firstly, multiple parallel convolution layers are used to extract features of the input image, and multiple parallel convolution layers can capture the characteristics of the input image from different perspectives. These feature maps are complementary to each other and will guide the downstream convolution layer to extract the required image features; then, the present invention uses multiple serial deep feature extraction modules to deeply extract the features of the face. These deep feature extraction modules reconstruct the color of the image by using channel separation technology to restore the lost color information of the input image; finally, the present invention uses an upper The sampling operation and information fusion operation are used to fuse the extracted features with the input image. This is because the CNN network will lose some useful input information as the number of convolution layers increases. Here, upsampling and information fusion are used to supplement these lost details for the extracted image features; the present invention greatly enhances the reconstruction ability of the model by using methods such as channel separation and upsampling; in order to further streamline the model, the present invention uses a dynamic quantization method to implement INT8 quantization on the trained face enhancement network model, and the size of the model is reduced by more than 40%. A large number of experimental results show that the method of the present invention can accurately reconstruct a clear face image with limited computing resources.

[0044] The present invention uses the CNN network for face enhancement. The effectiveness of the method is demonstrated through extensive experiments. Experiments on a real low-light face dataset show that the method proposed in the present invention has good performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of the method of the present invention;

[0046] Figure 2 This is a network framework diagram of low-quality low-light face enhancement according to the present invention. DETAILED DESCRIPTION

[0047] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] The present invention provides an ultra-lightweight low-resolution low-light face enhancement method. First, a face image dataset is constructed. Then, a face enhancement network model for low-resolution low-light face enhancement is constructed and trained using the face image dataset. Finally, the trained face enhancement network model is used to predict the collected face image test samples to obtain high-quality clear face images, which are then evaluated using evaluation indicators. The face enhancement network model includes: a data enhancement module, a deep feature extraction module, and a high-quality feature fusion module. First, the data enhancement module extracts features from the input low-resolution low-light face image I to obtain multiple feature maps, which are then fused to obtain an enhanced feature map F'. Then, several deep feature extraction modules are constructed in series to perform deep feature extraction on the enhanced feature map F' to obtain a deep feature map F d ; High-quality feature fusion module for deep feature map F d Further processing, the specific operation is: through the information fusion module to transform the deep feature map F d It is fused with the corresponding upsampled low-resolution dark-light face image I, and then further passed through a convolution layer to obtain a high-quality feature map Fh. The high-quality feature map Fh is sub-pixel convolved (i.e., sub-pixel upsampling) to obtain a high-quality clear face image Y pred ; For high-quality clear face image Y pred and the corresponding real clear face gold standard Y gt Calculates the Euclidean distance.

[0049] After the face enhancement network model training is completed, in order to further streamline the model parameters and size, a dynamic quantization algorithm can be used to perform INT8 quantization operations on the trained face enhancement model, thereby effectively reducing the size of the model.

[0050] like Figure 1 As shown, the specific steps include:

[0051] S1. Extract features from the input low-resolution dark-light face image I with the help of the data enhancement module to obtain multiple feature maps, and fuse the multiple feature maps to obtain an enhanced feature map F';

[0052] The details are as follows: construct a face image dataset S = {(x1, y1), (x2, y2), ..., (x L ,y L )}, where x i Represents a low-resolution dark-light face image I,y i Indicates that x iThe corresponding real clear face gold standard, i = 1, ..., L, L represents the number of samples in the data set; the maximum and minimum normalization method is used to control the data range in the face image data set between 0 and 1; several parallel convolution layers are used to extract features from the low-resolution dark light face image I obtained after normalization and then spliced ​​to obtain a feature map matrix F composed of several feature maps s ={F1, F2, ..., F n}, the feature map matrix contains the features of different granularities of the face; then, a convolutional layer is used to transform the feature map matrix F s The enhanced feature map F' is obtained by processing and fusing the processing results together. The enhanced feature map F' is:

[0053] F′=CNN(F s ) (1);

[0054] Among them, CNN is used to process F s The convolution operation is performed, and F' represents the enhanced feature map.

[0055] like Figure 2 As shown, in this embodiment, the data enhancement module includes 6 parallel convolution layers, each of which includes a 3*3 convolution kernel and an activation function RELU. The 6 parallel convolution layers are used to extract features from the low-resolution dark light face image I after uniform scale, and then spliced ​​to obtain a feature map matrix F consisting of 6 feature maps. s ={F1, F2, ..., F6}, F1, F2, ..., F6 are the feature maps extracted from the 1st to 6th convolutional layers respectively; then the feature map matrix F s After being processed by a 1*1 convolution kernel, the enhanced feature map F' is obtained.

[0056] S2, with the help of several deep feature extraction modules constructed in series, perform deep feature extraction on the enhanced feature map F' to obtain a deep feature map F d ;

[0057] The details are as follows: the first deep feature extraction module performs deep feature extraction on the enhanced feature map F' obtained in step S1. First, the enhanced feature map F' is separated by the channel separation module to enhance the features of the face images in different channel domains; then the separated feature maps are respectively subjected to 3*3 convolution and activation to realize deep feature extraction; finally, the convolved feature maps are spliced ​​to realize channel fusion, and a fused feature map is obtained by using a 1*1 convolution kernel fusion; the remaining series of deep feature extraction modules perform channel separation, deep feature extraction and convolution fusion on the output result of the previous deep feature extraction module (i.e., the fused feature map) in turn, and finally obtain a deep feature map F d:

[0058] F d =CNN 1*1 (CON(CNN 3*3 (SP(F′)))) (2);

[0059] Among them, F' is the enhanced feature map, SP is the channel separation operation, CNN 3*3 It is a convolution layer with a convolution kernel of 3*3, CNN 1*1 It is a convolution layer with a convolution sum of 1*1, and CON represents the channel fusion operation.

[0060] like Figure 2 As shown, in this embodiment, three depth feature extraction modules are connected in series to extract depth features from the enhanced feature map F' in sequence to obtain a depth feature map F d Each deep feature extraction module consists of a channel separation module, four parallel convolutional layers, a splicing module and a 1*1 convolutional layer. The four parallel convolutional layers all use a 3*3 convolution kernel and a ReLU activation function.

[0061] S3, high-quality feature fusion module for deep feature map F d Further processing is performed to obtain a high-quality clear face image Y pred ;

[0062] Specifically, the high-quality feature fusion module includes an upsampling module, an information fusion module, a convolution layer and a sub-pixel convolution (i.e., sub-pixel upsampling). The upsampling module is bilinear upsampling, and the convolution layer includes a 1*1 convolution kernel, an activation function ReLU and a 3*3 convolution kernel. First, the depth feature map F obtained in step S2 is d The corresponding upsampled low-resolution dark-light face image I is fused in the channel domain dimension through the information fusion module; then, a 1*1 convolution kernel and a 3*3 convolution kernel are used to amplify the channel domain to obtain a high-quality feature map Fh; the high-quality feature map F h Perform sub-pixel convolution to predict a high-quality clear face image Y pred , Y pred The formula is:

[0063] Y pred =sub(CNN 1*1 (Up(I)+F d )) (3);

[0064] Among them, sub represents sub-pixel convolution, CNN 1*1 Represents the convolution operation, and Up represents bilinear upsampling.

[0065] S4, by minimizing the high-quality clear face image Y pred and the corresponding real clear face gold standard Y gt The Euclidean distance between them is used to complete the training of the face enhancement network;

[0066] Specifically: Use Euclidean distance to describe the distance between two face images. The calculation formula of Euclidean distance is as follows:

[0067]

[0068] Among them, Y pred represents the face image predicted by the network, Y gt Represents the face image Y predicted by the network pred The corresponding real clear face gold standard, Dis(Y pred , Y gt ) represents the face image Y pred and Y gt The Euclidean distance, x i and y i They respectively represent the faces belonging to the face image Y pred and Y gt The computational complexity of the Euclidean distance is O(N 2 ).

[0069] Use the loss function to express the objective function of face enhancement and complete the training of the face enhancement model;

[0070] The loss function calculated by Euclidean distance is:

[0071] L(Y pred , Y gt )=Dis(Y pred , Y gt ) (5);

[0072] Among them, Dis(Y pred , Y gt ) represents the Euclidean distance between the face image output by the model and the gold standard.

[0073] like Figure 2As shown, the face enhancement network model constructed in the embodiment of the present invention includes: a data enhancement module, three serially connected deep feature extraction modules and a high-quality feature fusion module. The data enhancement module includes 6 parallel convolution layers, a splicing module and a 1*1 convolution layer connected in sequence. The 6 parallel convolution layers each include a 3*3 convolution kernel and an activation function RELU: Each deep feature extraction module includes a channel separation module, 4 parallel convolution layers, a splicing operation and a 1*1 convolution layer connected in sequence; the 4 parallel convolution layers each include a 3*3 convolution kernel and an activation function RELU: The high-quality feature fusion module includes an upsampling module, an information fusion module, a convolution layer and a sub-pixel convolution layer. The upsampling module performs a deep feature map F output by the deep feature extraction module. d The corresponding low-resolution dark-light face image I is upsampled, and the information fusion module combines the upsampled image and the depth feature map F d For fusion, the convolution layer includes a 1*1 convolution kernel, an activation function ReLU and a 3*3 convolution kernel.

[0074] The face enhancement network model first uses 6 parallel convolutional layers to extract features from the input image, then uses 3 serial deep feature proposal modules to deeply reconstruct the features, then uses an information fusion module to fuse the upsampled input image and the deeply extracted feature map, and finally uses a convolutional layer and a sub-pixel convolution to generate a high-quality face image.

[0075] After the face enhancement network model is trained, in order to further streamline the model parameters and size, a dynamic quantization algorithm can be used to perform INT8 quantization on the trained face enhancement model, thereby effectively reducing the size of the model.

[0076] S5. Use the trained face enhancement network model to predict the collected face image test samples to obtain high-quality clear face images, and evaluate them using evaluation indicators;

[0077] Specifically: predict the test sample and enhance a low-resolution low-light face image. In order to evaluate the performance of the face enhancement results, PSNR and SSIM are selected as evaluation indicators to evaluate the face enhancement results.

[0078] PSNR stands for Peak Signal-to-Noise Ratio, which is determined by the minimum mean square error (MSE).

[0079]

[0080]

[0081] Among them, Y i predand Y i gt They represent the high-quality clear face image Y output by the model pred The pixel and Y pred The corresponding real clear face gold standard Y gt Pixels; MAX I Indicates the maximum value of the image point color, which is generally 255. n represents the number of training set samples, and i represents the current sample.

[0082] SSIM stands for structural similarity, which compares the brightness, contrast and structural characteristics between images.

[0083] c1=(k1L) 2

[0084] c2=(k2L) 2

[0085]

[0086] in, is the high-quality clear face image Y output by the model pred The average value of Yes and Y pred The corresponding real clear face gold standard Y gt The average value of It's Y pred The variance of It's Y gt The variance of L is the dynamic range of pixel values, k1 and k2 are both preset hyperparameters, and the values ​​here are: k1 = 0.01, k2 = 0.03, Represents Y pred Standard deviation and Y gt Standard deviation The product of , c1 and c2 are smoothing parameters.

[0087] The test data is enhanced to obtain high-quality clear face images, which are evaluated using the customized metrics described above. The effectiveness of this method is demonstrated through extensive experiments and applied to low-quality dark-light faces.

[0088] This method uses the above two indicators to evaluate the completion results. Two representative algorithms in the field of face enhancement are selected: SwinIR and ELAN. The indicator results are shown in Table 1. The method of the present invention (Ours) achieves the best results in both PSNR and SSIM, indicating that the enhancement results of the present invention are closer to the real results.

[0089] Table 1 Results of low-light face enhancement indicators

[0090]

[0091] The present invention provides an ultra-lightweight low-resolution low-light face enhancement system, comprising: a model construction and training unit and a test and evaluation unit, wherein the model construction and training unit constructs a face enhancement network model for low-resolution low-light face enhancement and uses a face image dataset for training; the test and evaluation module is used to use the trained face enhancement network model to predict collected face image test samples to obtain high-quality clear face images, and evaluate them using evaluation indicators.

[0092] The model construction and training unit includes a data processing module, an encoding module and a training module. The data processing module normalizes the input low-resolution dark light face image I and performs multi-channel feature extraction to obtain a multi-granularity enhanced feature map F'; the encoding module constructs several series-connected deep feature extraction modules to perform deep feature extraction on the enhanced feature map F' to obtain a deep feature map F d , and then the depth feature map F d It is fused with the corresponding upsampled low-resolution dark-light face image I to obtain a high-quality feature map F h , the high-quality feature map Fh is converted into a high-quality clear face image Y through convolution layer and sub-pixel convolution pred ; The training module minimizes Y pred and the corresponding real clear face gold standard Y gt The Euclidean distance between them is used to complete the training of the face enhancement network model; the test evaluation unit enhances the dark light face image according to the prediction results of the test sample and evaluates the face enhancement results.

[0093] The face enhancement network model includes: a data enhancement module, a deep feature extraction module and a high-quality feature fusion module. First, the data enhancement module extracts features from the input low-resolution dark-light face image I to obtain multiple feature maps and splice them. Then, an enhanced feature map F' is obtained by fusion through a 1*1 convolutional layer; a deep feature extraction module is constructed in series to perform deep feature extraction on the enhanced feature map F' to obtain a deep feature map F d ; High-quality feature fusion module for deep feature map F d Further processing, the specific operation is: through an information fusion module to transform the deep feature map F d It is fused with the corresponding upsampled low-resolution dark-light face image I, and further obtained into a high-quality feature map F through a 1*1 convolution kernel, an activation function ReLU and a 3*3 convolution kernel. h , for high-quality feature maps Fh Perform sub-pixel convolution to obtain a high-quality clear face image Y pred ; For high-quality clear face image Y pred and the corresponding real clear face gold standard Y gt Calculates the Euclidean distance.

[0094] Based on the same inventive concept, the present invention provides an ultra-lightweight, low-resolution, low-light face enhancement device, including a memory and a processor, wherein:

[0095] a memory for storing computer programs capable of running on the processor;

[0096] The processor is configured to execute the steps of the above-mentioned ultra-lightweight low-resolution low-light face enhancement method when running the computer program.

[0097] Based on the same inventive concept, the present invention provides a storage medium having a computer program stored thereon, which, when executed by at least one processor, implements the steps of the above-mentioned ultra-lightweight, low-resolution, low-light face enhancement method.

Claims

1. An ultra-lightweight low-resolution dark light face enhancement method, characterized by: The following steps are involved: Construct a face enhancement network model for low-resolution low-light face enhancement and train it using a face image dataset; use the trained face enhancement network model to predict the collected face image test samples to obtain high-quality clear face images, which are then evaluated using evaluation indicators; The face enhancement network model includes: a data enhancement module, several series-connected deep feature extraction modules and a high-quality feature fusion module. First, the data enhancement module performs parallel convolution feature extraction on the input low-resolution dark light face image I, and the obtained multiple feature maps are spliced ​​and convolutionally fused to obtain an enhanced feature map F'; the enhanced feature map F' is subjected to deep feature extraction by several series-connected deep feature extraction modules to obtain a deep feature map F d ; Through the high-quality feature fusion module, the deep feature map F d Further processing, the specific operation is: through an information fusion module to transform the deep feature map F d It is fused with the corresponding upsampled low-resolution dark-light face image I, and then further passed through a convolution layer to obtain a high-quality feature map F h , for high-quality feature maps F h Perform sub-pixel convolution to obtain a high-quality clear face image Y pred .

2. The ultra-lightweight low-resolution dark-light face enhancement method according to claim 1, characterized in that: Before extracting features from the low-resolution dark-light face image I, it is also necessary to perform a normalization operation on the data samples in the face image dataset, normalizing the data range of the data samples to between 0 and 1.

3. The ultra-lightweight low-resolution dark-light face enhancement method according to claim 1, characterized in that: The data enhancement module includes n parallel convolutional layers, which use n parallel convolutional layers to extract features from the low-resolution dark light face image I to obtain n feature maps, and then a feature map matrix F composed of n feature maps is obtained through a splicing operation. s ={F1,F2,...,F n }, F1, F2, ..., F n These are the feature maps obtained by extracting features from the 1st to the nth convolutional layers respectively; Then, a convolutional layer is used to transform the feature map matrix F s After fusion, the enhanced feature map F' is obtained as follows: F'=CNN 3*3 (F s ) Among them, CNN 3*3 Is to deal with F s The convolutional layer of , F' represents the enhanced feature map.

4. The ultra-lightweight low-resolution low-light face enhancement method according to claim 1, characterized in that: Several series-connected deep feature extraction modules each include a channel separation module, m parallel convolution layers and a convolution fusion module connected in sequence. Several series-connected deep feature extraction modules are used to extract deep features from the enhanced feature map F' in sequence to obtain a deep feature map F d , specifically: The first deep feature extraction module extracts deep features from the enhanced feature map F': first, the channel separation module performs channel separation on the enhanced feature map F' to enhance the features of facial images in different channel domains; then, the m separated feature maps are respectively subjected to 3*3 convolution and activation through m parallel convolutional layers to achieve deep feature extraction; finally, the feature maps after 3*3 convolution are spliced ​​to achieve channel fusion, and fused using a 1*1 convolution kernel to obtain a fused feature map; The remaining series-connected deep feature extraction modules perform channel separation, deep feature extraction, and convolution fusion on the output results of the previous deep feature extraction module in turn, and finally obtain a deep feature map F d : F d =CNN 1*1 (CON(CNN 3*3 (SP(F′))); Among them, F' is the enhanced feature map, SP is the channel separation operation, CNN 3*3 and CNN 1*1 All are convolutional layers, and CON represents the channel fusion layer.

5. The ultra-lightweight low-resolution dark-light face enhancement method according to claim 1, characterized in that: After the face enhancement network model training is completed, the trained face enhancement network model is further quantized through a dynamic quantization algorithm to obtain a quantized face enhancement network model; and the quantized face enhancement network model is used to predict the collected face image test samples.

6. The ultra-lightweight low-resolution dark-light face enhancement method according to claim 1, characterized in that: The Euclidean distance is used to train the face enhancement network model, specifically: High-quality clear face image Y pred and the corresponding real clear face gold standard Y gt The Euclidean distance between them is: Among them, Y pred Represents the face image predicted by the network, Y gt Represents the face image Y predicted by the network pred The corresponding real clear face gold standard, Dis(Y pred ,Y gt ) represents the face image Y pred and Y gt The Euclidean distance, Y i pr*d and Y i gt They respectively represent the faces belonging to the face image Y pr*d and Y gt Pixels of Use the loss function to express the objective function of face enhancement and complete the training of the face enhancement network model; The loss function calculated by Euclidean distance is: L(Y pred ,Y gt )=Dis(Y pref ,Y gt ); Among them, Dis(Y pred ,Y gt ) represents the high-quality clear face image Y output by the model pred and the corresponding real clear face gold standard Y gt The Euclidean distance between .

7. The ultra-lightweight low-resolution low-light face enhancement method according to claim 1, characterized in that: Evaluation indicators include peak signal-to-noise ratio (PSNR) and structural similarity (SSIM); The peak signal-to-noise ratio (PSNR) is calculated as follows: Among them, MAX I Indicates the maximum value of the image point color; The calculation method of structural similarity SSIM is: in, is the high-quality clear face image Y output by the model pr*d The average value of Yes and Y pr*d The corresponding real clear face gold standard Y gt The average value of It's Y pred The variance of It's Y gt The variance of Indicates Y pred Standard deviation and Y gt Standard deviation The product of , c1 and c2 are smoothing parameters.

8. An ultra-lightweight low-resolution dark light face enhancement system, characterized by: include: Model building and training unit: builds a face enhancement network model for low-resolution low-light face enhancement and trains it using a face image dataset; And a test evaluation module, which is used to use the trained face enhancement network model to predict the collected face image test samples, obtain high-quality clear face images, and evaluate them using evaluation indicators; The face enhancement network model includes: a data enhancement module, several series-connected deep feature extraction modules and a high-quality feature fusion module. First, the data enhancement module performs parallel convolution feature extraction on the input low-resolution dark light face image I, and the obtained multiple feature maps are spliced ​​and convolutionally fused to obtain an enhanced feature map F'; the enhanced feature map F' is subjected to deep feature extraction by several series-connected deep feature extraction modules to obtain a deep feature map F d ; Through the high-quality feature fusion module, the deep feature map F d Further processing, the specific operation is: through an information fusion module to transform the deep feature map F d It is fused with the corresponding upsampled low-resolution dark-light face image I, and then further passed through a convolution layer to obtain a high-quality feature map F h , for high-quality feature maps F h Perform sub-pixel convolution to obtain a high-quality clear face image Y pred .

9. An ultra-lightweight, low-resolution, low-light face enhancement device, characterized by: comprising a memory and a processor, wherein: a memory for storing computer programs capable of running on the processor; A processor, configured to execute the steps of an ultra-lightweight, low-resolution, low-light face enhancement method as described in any one of claims 1 to 7 when running the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by at least one processor, implements the steps of an ultra-lightweight, low-resolution, low-light face enhancement method as described in any one of claims 1 to 7.

Images