Robust face super-resolution processing method and system based on empirical relationship deviation correction

By constructing an augmented library and feature extraction, the correlation and offset of image blocks are calculated, and the super-resolution processing of faces is guided, which solves the problem of insufficient robustness in facial image recovery in complex degraded environments, and significantly improves recovery Effectiveness and robustness.

CN114549323BActive Publication Date: 2024-11-08FUJIAN NORMAL UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210183572.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-28
Publication Date
2024-11-08
Estimated Expiration
2042-02-28

AI Technical Summary

Technical Problem

The existing facial super-resolution technology is difficult to effectively restore high-quality facial images in complex and degraded environments, and is not robust enough, especially in monitoring environments under multiple lighting and distance conditions. The robustness of existing algorithms is and less universal.

Method used

Using a robust face super-resolution processing method based on empirical relationship bias correction, the correlation degree of each image block is calculated by constructing an augmented library containing high-resolution and low-resolution image library using local block scoring and feature extraction. and offsets, generate an augmented matrix and weight vectors, guiding the reconstruction process of low-resolution images.

Benefits of technology

The recovery effect of low-quality face images is improved, the accuracy and noise robustness of image expression are enhanced, and the objective quality and similarity of the recovery results are significantly improved in the monitoring environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114549323B_ABST
    Figure CN114549323B_ABST
Patent Text Reader

Abstract

The present invention discloses a robust face super-resolution processing method and system based on empirical relationship deviation correction, constructs a training library including a high-resolution face image library and its corresponding multiple degraded low-resolution face image libraries; constructs a low-resolution image for supplementing information based on the original training library model and the input image; establishes two models of correlation experience value and individual value through high-resolution and low-resolution channels in the original training library respectively; calculates individual offsets through the correlation using the nearest neighbors of the input image as a bridge, and obtains the expression coefficient of the low-resolution augmented library based on the individual deviation calculation; obtains the reconstructed high-resolution image block based on the expression coefficient and the block corresponding to the high-resolution augmented library; and obtains the high-resolution face image by splicing the high-resolution face image blocks according to the position i. The present invention can significantly improve the visual experience of the restored image, and is particularly suitable for the restoration of face images in low-quality monitoring environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and image restoration, and in particular to a robust face super-resolution processing method and a system thereof based on empirical relationship deviation correction. Background Art

[0002] Face super-resolution technology is to provide effective constraints and guidance for the process from low resolution to high resolution by learning the high-low resolution correspondence between training pairs in the prepared high- and low-resolution image library, and estimate the high-resolution face image for machine or human eye recognition. This technology is widely used in various visual processing technologies, such as pre- and post-processing of face recognition, face image enhancement in surveillance video, etc. However, due to the complex and changeable imaging environment, the degradation conditions of imaging are even more complicated and diverse, and difficult to unify. As a result, the difficulty coefficient of face image enhancement is greatly increased, so face super-resolution technology is needed to increase the effective size and effective resolution of the image, so as to achieve the purpose of restoring from low-resolution image to high-resolution image. However, existing algorithms are often designed for a single degradation. Faced with a variety of low-quality face images generated under complex and numerous degradation conditions, it is difficult to effectively restore recognizable face images.

[0003] In order to solve the problem of restoring low-quality facial images with various complex degradations, traditional algorithms often use various prior information to assist in analyzing, selecting or improving algorithms from a single type of degraded database. The specific approach is to introduce effective computing tools and data utilization forms. In this approach, the neglect of a posteriori knowledge leads to defects in the effectiveness and universality of the algorithm. That is, when encountering a complex degradation process, the algorithm has low robustness and it is difficult to effectively enhance the facial image, which in turn results in low ease of use and effectiveness of the algorithm. A classic category of algorithms is the category based on manifold learning.

[0004] Manifold learning is one of the classic algorithms for face super-resolution. Its core idea is to use manifold expression as a tool to explain the quality improvement process from low resolution to high resolution. Specifically, for the two sample spaces of high and low resolution, find the local properties around each low-resolution image data point, then nonlinearly map the manifold local properties of the low-resolution image to the manifold space of the high-resolution image, and project it on the high-resolution corresponding space to synthesize the high-resolution image. The following are representative methods: In 2004, Chang et al. [1] first introduced manifold learning into image super-resolution reconstruction and proposed a neighborhood embedding image super-resolution reconstruction method. Sung Won Park [2] proposed an adaptive manifold learning method based on local preservation projection, which analyzes the intrinsic features of the face from the local sub-manifold and reconstructs the high-frequency components missing from the low-resolution image. In 2010, Huang [4] proposed a method based on CCA (Canonical Correlation Analysis, CCA), which further solved this problem by extending the PCA (Principal Component Analysis, PCA) space to the CCA space. In the same year, Lan[5] proposed a face super-resolution method based on shape constraints to address the problem of severe image pixel damage caused by severe blur and noise in surveillance environments. Shape constraints were added as a similarity measurement criterion in the traditional PCA architecture, and the robustness of the human eye to interference when recognizing shapes was used to artificially add shape feature points as constraints to optimize the reconstruction results of low-quality images.

[0005] Most of these existing methods only follow traditional technical ideas, focus on the extraction of prior knowledge, focus on the preparation of high- and low-resolution image libraries with fixed and single degradation types, and use prior knowledge, ignoring the fact that the input image may have a complex degradation process that is heterogeneous with the image library, resulting in the algorithm's reconstruction effect for heterogeneous images with different degradation processes is not ideal, especially in complex and diverse degradation environments such as surveillance that contain a variety of lighting and distances. Since it is impossible to strictly control the uniformity of the actual degradation conditions and the single degradation conditions in the library, the captured low-quality face images and the face images in the sample library have large differences in degradation processes, and this difference has obvious diversity. The huge difference and the diversity of differences cause complex and diverse reconstruction errors, which cannot be uniformly handled by the existing prior knowledge and model tools for single degradation, resulting in insufficient robustness and unsatisfactory results. Summary of the invention

[0006] The object of the present invention is to provide a robust face super-resolution processing method and system based on empirical relationship deviation correction.

[0007] The technical solution adopted by the present invention is:

[0008] A robust face super-resolution processing method based on empirical relationship deviation correction comprises the following steps:

[0009] S1: Build an original training library containing a high-resolution face image library and its corresponding low-resolution face image library, as well as two augmented libraries with extremely low quality degradation.

[0010] S2: Based on the backup training library X1, X2, ..., X in the augmented library n And the input face image x to be processed in , a new augmented library is constructed by scoring local blocks;

[0011] S3: Preprocess the high-resolution face image library and the low-resolution face image library on a block basis. Calculate the empirical correlation between each position and other position blocks and the correlation to be tested of the input image at the two levels of the high-resolution image library and the low-resolution input image, so as to prepare data for the subsequent calculation of individual deviations based on the correlation.

[0012] S4: Based on the calculation in the previous step and id, which is the input image block The offset vector is calculated for each neighboring block to prepare data for the subsequent reconstruction process guided by individual deviations.

[0013] S5: Based on the individual bias as a guide, the input low-resolution image x in Project it into the classification feature space and calculate the super-resolution expression coefficient of each block position.

[0014] S6: Project the super-resolution expression coefficients and weight vectors back to the high-resolution space to obtain the high-resolution spatial re-expression result, which can be expressed as Splice high-resolution face image blocks according to position variable i Get high-resolution face image y out .

[0015] Furthermore, in step S1, the positions of high-resolution face images in the high-resolution face image library are aligned, and several different degradation processes are performed to obtain corresponding low-resolution face image libraries. The high-resolution face image library and the three low-resolution face image libraries constitute three training libraries.

[0016] Furthermore, in step S1, the low-resolution face image to be processed is made to have the same size as the image in the training library and to be aligned in position, and the position alignment is performed using an affine transformation method.

[0017] Furthermore, the specific method of generating a low-resolution face image library by downgrading the high-resolution face image library in S1 is as follows: suppose n pairs of well-aligned high-resolution and low-resolution face sample pairs Y and X, and the low-resolution face sample X is generated from the high-resolution face sample Y by a conventional downgrading process D, where X = {x1, x2, ..., x n}, Y={y1,Y2,...,y n}, then X and Y constitute the original training library; n different degradation processes D1, D2, ..., D n , construct n kinds of extremely low-quality backup training libraries X1, X2, ..., X n .

[0018] Furthermore, in S2, when the low-resolution image to be processed is input, in When reconstructing a high-resolution block, take out the image block to be processed corresponding to position i And the set of low-resolution spare training library blocks corresponding to position i The specific steps of block division are as follows:

[0019] S2-1, for x in Each face image in the image is uniformly divided into blocks without any difference. The block window size (block size) is recorded as bb, and the number of overlapping pixel sizes is dd. Therefore, the window sliding step size is bb-dd.

[0020] S2-2, assuming that the length and width of each face image are imrow and imcol pixels respectively, each image block is calculated to be M = U × V, and the calculation formula is as follows:

[0021] U = ceil ((imrow-dd) / (bb-dd))

[0022] V = ceil ((imcol-dd) / (bb-dd))

[0023] Among them, U represents the number of blocks in the length direction, and V represents the number of blocks in the width direction.

[0024] S2-3, let i represent the position number, i=1,2,3,...,M are respectively selected from the low-resolution backup training libraries X1,X2,...,X n get of blocks.

[0025] S2-4, calculate the block at position i in the feature space The degradation type label label, that is, according to Calculate to The closest distance determines the corresponding degradation type as the label of position i, express The gradient direction eigenvalue of ; Respectively The discriminant feature F dis .

[0026] Furthermore, in step S2-4, a feature library for determining degradation categories is constructed in the new feature space, and the feature extraction method is as follows: and a low-resolution alternate training library block set Each image block in the image is extracted based on its discriminant features to form a new feature library in The specific steps are as follows:

[0027] by For example, j=1,2,...n, the extraction formula is: DegradationDis(·) represents the operation for extracting features. Its specific operation includes three steps:

[0028] S2-4-1, Gaussian difference iterative processing: By looping m_i times Gaussian difference, the contour enhancement information is extracted and summed up to obtain the enhanced Gaussian difference information in F G (·) represents Gaussian filter, m1,...,m m_i and n1,...,n m_i Indicates the strength of Gaussian filtering, and when the subscripts are the same, m≤n, is the result of Gaussian difference, p=1,2,…,m_i.

[0029] Specifically, As an example, we obtain the enhanced Gaussian difference iterative feature The process is as follows:

[0030]

[0031]

[0032] …

[0033]

[0034] Add the above contents together to obtain enhanced Gaussian difference information

[0035] S2-4-2, image to be processed The following formula is used for gamma correction,

[0036]

[0037] in, represents the pixel at the i-th row and j-th column after gamma processing, Represents the image to be processed For the pixel in the i-th row and j-th column, gama represents the intensity of the gamma processing;

[0038] S2-4-3, the gamma correction result is sent to S2-4-1 for another Gaussian difference iterative processing to obtain enhanced Gaussian difference information renew For addition and The results afterwards;

[0039] S2-4-4, Extract the gradient direction eigenvalue to obtain

[0040] S2-4-5, respectively Get

[0041] The specific steps for label finalization are as follows:

[0042] S2-4-6, calculation arrive The sum of the distances of each feature at position i in .

[0043]

[0044] …

[0045]

[0046] S2-4-7, selecting the degraded label represented by the minimum value as the degraded type label of the corresponding position through comparison;

[0047] S2-4-8, traverse the i value to calculate the degradation labels corresponding to all positions to obtain an n-dimensional vector L in ;

[0048] Statistics in The frequency of each degradation in the subsequent generation of the augmented matrix is ​​calculated as follows:

[0049]

[0050] Among them, s h Represents the weight corresponding to the h-th degradation, h = 1, 2, ..., n.

[0051] For example, the second degradation D2 in L in The calculation formula for the frequency s2 in is:

[0052]

[0053] S2-4-9, based on weight s h Generate augmented matrix X aug , the calculation formula is as follows

[0054]

[0055] Among them, X aug represents the augmented matrix, s h represents the weight corresponding to the h-th degradation, X h represents the h-th degraded backup training library.

[0056] Further, in S2-4-7, the minimum distance value FinalDis at position i is selected i :

[0057]

[0058] For example, suppose The degradation type of position i is D2, and the label representing position i that has selected label D2 is Store into vector L in .

[0059] Furthermore, the calculation steps of the correlation between the image block at the i-th position in S3 and all other positions are as follows:

[0060] S3-1, let i represent the position number, i = 1, 2, 3, ..., M, then there are n high-resolution sample blocks at position i, that is, j represents the number of samples;

[0061] S3-2, for any j=1,2,3,...,n,calculate the jth face sample image Distance vector to all other patches The calculation formula is as follows:

[0062]

[0063] in, r=1,2,3,...,M; Indicates block fetch The texture features of Indicates the texture feature extraction operation.

[0064] S3-3, after traversing the n image blocks at position i, n The average distance from position i to all other positions is recorded as an M-dimensional vector R. i,Right now:

[0065] S3-4, after traversing all the positions of i, a fully connected network between the positions is obtained. The association degree in the fully connected network is represented as a vector set.

[0066] S3-5, all the connections with weak correlation in the correlation vector are removed, and a specified number of connections with high correlation are left.

[0067] Furthermore, the specific steps of S3-5 are:

[0068] S3-5-1, for each R i Only the smallest K values ​​are retained, and only values ​​less than the threshold t are retained by filtering, which is expressed as take out The index (position number) in is recorded as id i ;

[0069] S3-5-2, after traversing all i values, the simplified network association vector set is recorded as: The corresponding index vector set is recorded as

[0070] S3-5-3, calculate the image block set X at position i corresponding to the low-quality training sample library X according to the association network i The correlation between each image block and the face block in which it is located

[0071] Specifically, take image x in Taking the correlation as an example, the calculation process is explained as follows:

[0072] Calculate the face sample image x input at low resolution in On, calculate Distance vector to all other patches in r=1,2,3,...,M. For example, when r=M, in Indicates block fetch texture features. Indicates the texture feature operation. So for the block at position i After traversing its M positions, we get the distance vector The corresponding label set is combined into a K-dimensional vector according to the first K labels with the smallest distance.

[0073] Traverse all face positions for the input image x inCalculate a vector matrix (M×K dimension) Record the association between its blocks.

[0074] Furthermore, in S4, the input image block is calculated Individual relationships and IDs of neighboring blocks at the same location i The overlap degree of the input image block is calculated Relationship id with in-stock experience i The offset (bias) vector is used as a guide to calculate the candidate samples that should be emphasized in the subsequent reconstruction process.

[0075] Furthermore, the specific calculation steps of S4 are as follows:

[0076] S4-1, first The image block set X at position i corresponding to the low-quality training sample library X i In the above example, we select the K image blocks with the closest distance as the neighbor block set, and then Select the global relationship label vector of the K blocks at their locations; then calculate the global relationship label vector and the experience value id i The number of overlaps is denoted by v k,i ;

[0077] Taking the k=1, 2, ..., K nearest neighbors as an example, according to the number of images in X (j value) and the position (i value), Pick out the corresponding vector And record it as Where k = 1, 2, ..., K.

[0078] Then calculate its value and the experience value id i The number of overlaps is denoted by v k,i , the formula is as follows:

[0079]

[0080] Where ∩ represents the intersection, || 0 It represents taking the 0 norm, that is, calculating the number of intersection elements and assigning it to v k,i .

[0081] S4-2, traverse all value elements of k to obtain the deviation vector The deviation vector V i Placed on the diagonal of the K times K zero matrix to get the matrix V' i =diag(V i ), where diag(·) represents the operation of placing the vector on the diagonal position of the zero matrix of the corresponding size.

[0082] Furthermore, in S5, the image block According to library X and X aug The image block library X corresponding to position i i and The corresponding expression coefficient ω is calculated through the following objective function i :

[0083]

[0084] in, is a diagonal matrix, The elements on the diagonal are calculated as: λ represents the balance coefficient, which is set by experience.

[0085] A robust face super-resolution processing system based on empirical relationship deviation correction, comprising:

[0086] Original training library construction model: used to construct a training library containing a high-resolution face image library and its corresponding low-resolution face image library containing seven degraded ones (one original and six as components for generating X aug of);

[0087] Specific degraded image library generation module: based on the original training library model and input image and x in , through feature extraction, comparison, statistical scoring, and then weighted construction of low-resolution image X for supplementary information aug ;

[0088] Relevance calculation module: establishes two models of relevance experience value and individual value through high-resolution and low-resolution channels in the original training library;

[0089] Offset calculation model: Through the correlation in the previous step, the nearest neighbors of the input image are used as a bridge to calculate the individual offset, and prepare data for the subsequent reconstruction process guided by individual deviations;

[0090] Coefficient retrieval module: Based on individual deviation as a guide, according to the augmented library X aug Based on the original library X, the expression coefficient of the low-resolution augmented library is calculated.

[0091] High-resolution image block reconstruction module: reconstruct high-resolution image blocks based on the expression coefficients and the blocks corresponding to the high-resolution augmentation library;

[0092] Stitching module: used to stitch high-resolution face image blocks according to position i Get high-resolution face images.

[0093] The present invention adopts the above technical solution. Compared with the prior art, the present invention has the following advantages and positive effects:

[0094] First, using the posterior knowledge of the image distribution of the input low-quality image, the ratio and composition of the degradation components are estimated, and an image library based on the posterior information and the input image degradation that is closest to the input image is generated as a knowledge supplement to the pixel density distribution pattern. Secondly, using the prior knowledge of the image library, the expression deviation of each expression neighbor is determined, and used to guide the weight correction work in the subsequent reconstruction process. Through the posterior-based degradation estimation and the prior-based individual deviation estimation, a training library supplement library is adaptively constructed for low-quality images. Through the prior-based image library knowledge, weight correction guidance is provided for the expression of the input image on the training library. The combination of the two can restore a relatively satisfactory reconstructed face image.

[0095] The present invention has universal applicability and achieves good restoration effects for general low-quality face images; in particular, the effect is more obvious for the restoration of face images in low-quality monitoring environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments;

[0097] Figure 1 A schematic diagram of the architecture of a robust face super-resolution processing system based on empirical relationship deviation correction according to the present invention;

[0098] Figure 2 This is a schematic diagram of facial image segmentation based on position according to the present invention. DETAILED DESCRIPTION

[0099] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0100] Unlike traditional methods that focus on the use of fixed single-degraded high- and low-resolution image libraries, the present invention focuses on how to use effective computing tools to study the impact of the complexity of the degradation model on the algorithm. By analyzing the type of degradation of the input image, the degradation process that is homogeneous with the input image is simulated as much as possible, and applied to the preparation of the image library to provide supplementary knowledge for the original image library. At the same time, the original prior knowledge of the image library is used to provide a deviation correction prior for each sample. By combining prior knowledge with a posteriori knowledge, by proposing new a posteriori knowledge of image degradation and applying it to the preparation of the adaptive image library, combined with the newly proposed prior knowledge, this problem is solved based on the feature decomposition of the degradation knowledge.

[0101] Specifically, the present invention uses the posterior knowledge of the image distribution of the input low-quality image to estimate the ratio and composition of the degradation components therein, and generates an image library based on the posterior information and the input image degradation that is closest to the knowledge supplement of the pixel density distribution pattern. Secondly, the present invention uses the prior knowledge of the image library to determine the expression deviation of each expression neighbor, and uses it to guide the weight correction work in the subsequent reconstruction process. Through the posterior-based degradation estimation and the prior-based individual deviation estimation, a training library supplement library is adaptively constructed for low-quality images. Through the prior-based image library knowledge, weight correction guidance is provided for the expression of the input image on the training library. The combination of the two can restore a reconstructed face image with a more satisfactory effect.

[0102] like Figure 1 As shown, the present invention discloses a robust face super-resolution processing method based on empirical relationship deviation correction. The present invention uses a high- and low-resolution image library with multiple degradation processes, and under the guidance of a local feature scoring model, determines the type of degradation process and its corresponding proportion for the input low-resolution image, and constructs a set of low-resolution training libraries whose degradation processes are close to the input image according to the determination results, as a knowledge base supplement of the image pixel distribution pattern, so as to reduce the degradation gap between the input image and the image in the library, enhance the accuracy and noise robustness of the image expression, and improve the objective quality and similarity of the restored result. In addition, by learning the statistical characteristics of the sample individuals and the sample library, this relationship from the individual to the whole, and using the deviation correction to quantify the distribution of individual characteristics, the weight of the individual sample in the subsequent expression is optimized to avoid emphasizing the wrong neighboring samples in the expression and weakening the correct samples, so as to enhance the accuracy of the image expression by using the prior knowledge base, and improve the objective quality and similarity of the restored result.

[0103] The present invention will be further described below in conjunction with specific embodiments and drawings.

[0104] The present invention is aimed at extremely low-quality face images in monitoring environments and adopts a robust face super-resolution processing method based on empirical relationship deviation correction. In specific implementation, the technical solution of the present invention can use computer software technology to realize automatic operation process.

[0105] See also Figure 1 , the specific steps of the present invention are as follows:

[0106] S1: Construct an original training library containing a high-resolution face image library and its corresponding low-resolution face image library, as well as two extremely low-degraded augmented libraries, in which the low-resolution face image library is generated by degrading the quality of the high-resolution face image library.

[0107] Suppose there are n pairs of well-aligned high-resolution and low-resolution face samples X and Y, where X is generated by Y using a conventional degradation process D, where X = {x1, x2, ..., x n}, Y={y1,y2,...,y n Then X and Y constitute the original training library. Select n different degradation processes D1, D2, ..., D n , construct n kinds of extremely low-quality backup training libraries X1, X2, ..., X n .

[0108] In the specific implementation, first, the eyes and mouth positions of the high-resolution face image are aligned to obtain the high-resolution face library Y; then, the high-resolution face image is downsampled, fuzzy window filtered, and upsampled in sequence to obtain the low-resolution face image library X corresponding to the high-resolution face image. n different degradation processes D1, D2, ..., D n They are: (taking n=6 as an example)

[0109] D1 is a low-resolution face image library X1 corresponding to the high-resolution face image, which is obtained by downsampling, fuzzy window filtering, adding 0.015 Gaussian white noise, and upsampling the high-resolution face image in sequence;

[0110] D2 is to downsample, filter with blur window, and upsample the high-resolution face image in sequence to obtain a low-resolution face image library X2 corresponding to the high-resolution face image, where the downsampling of D2 is twice the size of X and D1.

[0111] D3 is to downsample, filter with blur window, and upsample the high-resolution face image in sequence to obtain a low-resolution face image library X3 corresponding to the high-resolution face image, where the downsampling of D3 is four times the size of X and D1.

[0112] D4 is to sequentially downsample, filter with blur window, add Poisson noise, and upsample the high-resolution face image to obtain a low-resolution face image library X4 corresponding to the high-resolution face image;

[0113] D5 is to downsample, filter with a blur window, and upsample the high-resolution face image in sequence to obtain a low-resolution face image library X4 corresponding to the high-resolution face image, where the downsampling of D5 is twice the size of D3.

[0114] D6 is to sequentially downsample, filter with a fuzzy window, add salt and pepper noise, and upsample the high-resolution face image to obtain a low-resolution face image library X6 corresponding to the high-resolution face image.

[0115] Specifically, for ease of implementation reference, the specific process of using the affine transformation method to achieve face image alignment is provided below:

[0116] The high-resolution face image is annotated with feature points, which are the edge points of the facial features, such as the corners of the eyes, the tip of the nose, the corners of the mouth, etc. Then, the feature points are aligned using the affine transformation method. The specific affine transformation method is:

[0117] Add all the face images in the high-resolution face image library Y and divide by the number of samples to get the average face. i ,y′ i ) is the coordinate of the i-th feature point on the average face, (x i ,y i ) is the coordinate of the i-th feature point on the high-resolution face image to be aligned. Let the affine matrix Among them, a, b, c, d, e, and f are affine transformation coefficients. Represents the coordinates of the i-th feature point on the average face and the high-resolution face image to be aligned (x′ i ,y′ i ) and (x i ,y i ), the direct linear transformation method is used to solve the affine transformation matrix M. The coordinates obtained by multiplying all coordinate points of the high-resolution face image to be aligned with the affine matrix M are the coordinates of the aligned high-resolution face image.

[0118] The aligned high-resolution face images are degraded. For example, the high-resolution face images are downsampled by 4 times, filtered by a blur window of 3*3, and upsampled by 4 times in sequence to obtain low-resolution face images corresponding to the high-resolution face images, thereby obtaining a low-resolution face image library X.

[0119] The facial images in the high-resolution facial image library Y and the low-resolution facial image library X correspond one to one to form a pair of high-resolution and low-resolution facial images. The low-resolution facial images to be processed are made to have the same size as the images in the training library and aligned in position.

[0120] Furthermore, the present invention is to process a low-resolution face image x in Process it and estimate the corresponding high-resolution face image, and record the estimated high-resolution face image as the high-resolution face image to be estimated y out .

[0121] Low-resolution face image x to be processed in It is usually a low-resolution face image obtained in a noisy environment. For the low-resolution face image to be processed as input, it is generally pre-processed, including cutting out the face part that meets the unified regulations, that is, the low-resolution face image to be processed x in Upsample to make it the same size as the face image in the training library. inThe feature points are marked, and finally the affine transformation method recorded in step S1 is used to transform the low-resolution face image x to be processed. in Align with the average face position. In this way, the face images in the training library and the low-resolution face images to be processed x in In terms of size and eyebrow height, they are at the same level. If the low-resolution face image x in If the light is insufficient during acquisition, the low-resolution face image x to be processed can be aligned. in Automatically adjust the brightness and contrast to make it at a similar brightness level to the low-resolution face images in the training library.

[0122] S2: Based on the backup training library X1, X2, ..., X in the augmented library n And the input face image x to be processed in , based on the scoring of local blocks, a new augmented library is constructed.

[0123] Specifically, when the low-resolution input image to be processed x in When reconstructing a high-resolution block, take out the image block to be processed corresponding to position i Take out the low-resolution spare training library block set corresponding to position i

[0124] x in Take x as an example to illustrate the block method: in Each face image in is divided into blocks uniformly without distinction. The block window size (block size) is recorded as bb, and the number of overlapping pixel sizes is dd. Therefore, the window sliding step is bb-dd. Assuming that the length and width of each face image are imrow and imcol pixels respectively, then it is calculated that each image is divided into M=U×V blocks, where U represents the number of blocks in the length direction and V represents the number of blocks in the width direction. The formula for obtaining the two is:

[0125] U = ceil ((imrow-dd) / (bb-dd))

[0126] V = ceil ((imcol-dd) / (bb-dd))

[0127] Let i represent the position label, i = 1, 2, 3, ..., M. From the low-resolution backup training library X1, X2, ..., X n get The block division method is the same as above.

[0128] Furthermore, a feature library for degradation category determination is constructed in the new feature space, and the feature extraction method is as follows:

[0129] Will and a low-resolution alternate training library block set For each image block in the image, its discriminant-based features are extracted to form a new feature library in by For example, j=1,2,...n, the extraction formula is: DegradationDis(·) represents the operation for extracting features, which consists of four steps:

[0130] (1) The first step is Gaussian difference iterative processing. The contour enhancement information is extracted by looping m_i times of Gaussian difference and then summed.

[0131] by Take the enhanced Gaussian difference iterative feature as an example. The process is as follows:

[0132]

[0133]

[0134] …

[0135]

[0136] Add the above contents to obtain enhanced Gaussian difference information in

[0137] Among them, F G () represents Gaussian filter, m1,...,m m_i and n1,...,n m_i Indicates the strength of Gaussian filtering, and when the subscripts are the same, m≤n, is the result of Gaussian difference, p=1,2,…,m_i.

[0138] (2) Image to be processed The following formula is used for gamma correction,

[0139]

[0140] in, represents the pixel at the i-th row and j-th column after gamma processing, Represents the image to be processed For the pixel in the i-th row and j-th column, gama represents the intensity of the gamma processing;

[0141] (3) Send the result of (2) to (1) for processing to obtain enhanced Gaussian difference information renew For addition and The result afterwards.

[0142] (4) Extract the gradient direction eigenvalue to obtain In this step, the gradient direction feature value adopts HoG feature [6].

[0143] Similarly, from Get the corresponding discriminant features

[0144] Furthermore, the block at position i is calculated in the feature space The degradation category label label, that is, according to Calculate to The closest distance is determined, and the corresponding degradation type is determined as the label of position i. The specific method is as follows:

[0145] (1) Calculation arrive The closest distance: The calculation method is to calculate arrive The sum of the distances of each feature at position i in .

[0146]

[0147] …

[0148]

[0149] (2) Select the minimum value by comparison and label it. The specific process is as follows:

[0150] Select the minimum distance value FinalDis at position i i :

[0151]

[0152] The downgraded label represented by the selected distance value is used as the downgraded category label of the location. The degradation type of position i is D2, and the label representing position i that has selected label D2 is Store into vector L in .

[0153] Traversing the i value and calculating the degradation labels corresponding to all positions, we will get an n-dimensional vector L in .

[0154] In L in In the above, by counting the frequency of each degradation, the proportion of the degradation in the subsequent generation of the augmented matrix is ​​calculated, and the weight corresponding to the hth degradation is recorded as s h , Where h = 1, 2, ..., n.

[0155] Specifically, for example, the second degradation D2 in L in Frequency of occurrence When the weight s is obtained h After that, the augmented matrix is ​​generated by the following formula:

[0156] S3: Based on the block division, preprocess the high-resolution face image library and the low-resolution face image library respectively. The empirical correlation between each position and other position blocks and the tested correlation of the input image are calculated at the two levels of the high-resolution image library and the low-resolution input image respectively, so as to prepare data for the subsequent calculation of individual deviations based on the correlation.

[0157] Let i represent the position number, i = 1, 2, 3, ..., M. For the image block at the i-th position, the process of obtaining the correlation between the block and all other positions is as follows:

[0158] There are n high-resolution sample blocks at position i j represents the number of samples. j=1,2,3,...,n, calculated on the jth face sample image Distance vector to all other patches in r=1,2,3,...,M, Indicates block fetch texture features. Indicates the texture feature extraction operation.

[0159] After traversing the n image blocks at position i, we can get n Vectors of the same size. By averaging these vectors, we can get the average distance from position i to all other positions, recorded as M-dimensional vector R i ,Right now:

[0160] After traversing all the positions of i, a fully connected network between the positions is obtained. The association degree in the network is represented as a vector set

[0161] Furthermore, the correlation vector obtained in the above process is pruned to remove all connections with weak correlation, leaving only a specified number of connections with high correlation. Specifically:

[0162] For each R i Only the smallest K values ​​are retained, and then only the values ​​less than the threshold t are retained through screening, which is expressed as take out The index (position number) in is denoted as id i .

[0163] After traversing all i values, we get a simplified set of network association vectors, recorded as: The corresponding index vector set is recorded as

[0164] According to the above association network, the image block set X at position i corresponding to the low-quality training sample library X is calculated. i The correlation between each image block and the face block in which it is located

[0165] Specifically, take image x in Taking the correlation as an example, the calculation process is explained as follows:

[0166] Calculate the face sample image x input at low resolution in , calculate the face sample image block Distance vector to all other patches in r=1,2,3,...,M. For example, when r=M, in Indicates block fetch texture features. Indicates the texture feature extraction operation.

[0167] So for the block at position i After traversing its M positions, we get the distance vector The corresponding label set is combined into a K-dimensional vector according to the first K labels with the smallest distance.

[0168] Traverse all face positions, for the input image x in Calculate a vector matrix (M×K dimension) Record the association between its blocks.

[0169] S4: Based on the calculation in the previous step and id, which is the input image block The offset vector is calculated for each neighboring block to prepare data for the subsequent reconstruction process guided by individual deviations.

[0170] By calculating the individual relationship and ID of its neighboring blocks at the same location i The degree of overlap is calculated Relationship id with in-stock experience iThe offset (bias) vector of . Using this deviation as a guide, the candidate samples that should be emphasized in the subsequent reconstruction process are calculated. The specific calculation steps are as follows:

[0171] First of all The image block set X at position i corresponding to the low-quality training sample library X i In the above example, we select the K image blocks with the closest distance as the neighbor block set, and then In , we select the global relationship label vector of the K blocks. And record it as Where k = 1, 2, ..., K; then calculate its value and the empirical value id i The number of overlaps is recorded as Where ∩ represents the intersection, || 0 Represents taking the 0 norm, that is, calculating the number of intersection elements and assigning it to v k,i .

[0172] Traverse all the value elements of k and get the deviation vector By placing this vector on the diagonal of the K times K zero matrix, we get the matrix V i ′=diag(V i ), where diag() represents the operation: placing the vector on the diagonal position of the zero matrix of corresponding size.

[0173] Specifically, taking the k=1, 2, ..., Kth nearest neighbor as an example, according to the number of images in X (j value) and the position (i value), Pick out the corresponding vector And record it as Where k = 1, 2, ..., K.

[0174] S5: Based on the individual bias as a guide, the input low-resolution image x in Project it into the classification feature space and calculate the super-resolution expression coefficient of each block position.

[0175] Specifically: Taking position i as an example, for the image block According to library X and X aug The image block library X corresponding to position i i and The corresponding expression coefficient ω is calculated through the following objective function i :

[0176]

[0177] in is a diagonal matrix, The elements on the diagonal are calculated as: λ represents the balance coefficient, which is set by experience.

[0178] S6: After obtaining the representation coefficients, project the coefficients and weight vectors back to the high-resolution space to obtain the high-resolution spatial re-expression result. The formula is expressed as: Splice high-resolution face image blocks according to position variable i Get high-resolution face image y out .

[0179] Specifically, when the low-resolution input image to be processed x in When reconstructing a high-resolution block, take out the image block to be processed corresponding to position i Take out the low-resolution reconstructed image library block set corresponding to position i The set of high-resolution reconstructed image library blocks corresponding to position i

[0180] X F Take X as an example to illustrate the block method: F Each face image in is divided into blocks uniformly without distinction. The block window size (block size) is recorded as bb, and the number of overlapping pixel sizes is dd. Therefore, the window sliding step is bb-dd. Assuming that the length and width of each face image are imrow and imcol pixels respectively, then it is calculated that each image is divided into M=U×V blocks, where U represents the number of blocks in the length direction and V represents the number of blocks in the width direction. The formula for obtaining the two is:

[0181] U = ceil ((imrow-dd) / (bb-dd))

[0182] V = ceil ((imcol-dd) / (bb-dd))

[0183] Let i represent the position number, i = 1, 2, 3, ..., M. From the low-resolution input image x in get and a set of patches Y reconstructed from the high-resolution image library F get The block division method is the same as above.

[0184] The low-resolution face image to be processed and the images in the training library are divided into image blocks with overlapping parts by the same block division method, wherein the image blocks are square image blocks with a side length of bb; in the present invention, each image in the training library is divided into M square image blocks; at the same time, the low-resolution face image to be processed x in It is also divided into M image blocks. The corresponding face image is represented by the image block set, and the low-resolution face image x is processed. in The image block is restored.

[0185] like Figure 2 As shown in the figure, the main basis for dividing the face image into blocks is the idea of ​​local manifold, that is, the face image is a special type of image, which has a specific structural meaning. For example, all the small blocks at a certain position are eyes, or all the small blocks at a certain position are noses. In other words, the local small blocks at each position in the image are in a specific local geometric manifold. To ensure this local manifold, the image needs to be divided into several square image blocks. The size of the image block needs to be of appropriate size. If the block is too large, it will cause ghosting due to minor alignment problems; if the block is too small, it will blur and weaken the position characteristics of each small block. In addition, it is also necessary to select the size of the overlapping blocks between image blocks. Because if the image is simply divided into several square small blocks without overlapping blocks, there will be a grid effect between these square blocks due to incompatibility issues. Moreover, face images are not always square, so the size of the overlapping blocks needs to be selected to make the image as fully divided as possible.

[0186] In order to verify the technical effect of the present invention, the Chinese face database CAS-PEAL was used for verification. 540 face samples were selected from them with a resolution of 112*96, and the faces were aligned using the affine transformation method. 500 images were randomly selected from the face samples as training samples, and the remaining 40 images were downsampled 4 times (resolution of 24*28) and then added with 0.015 Gaussian noise as test images. The remaining images of the face samples were used as the training library, and the test images were enlarged 4 times using the bicubic interpolation method to obtain subjective images; the traditional local face super-resolution method (method 1), method Lan[4] (method 2), and the robust face super-resolution processing method based on contour prior[5] (method 3) were used to obtain subjective images.

[0187] From the experimental results, it can be seen that although methods 1 to 3 have improved resolution compared to the interpolation method, they have serious errors and are very low in similarity to the original image. The results of method 2 are based on the global face architecture. Global-based methods often have shortcomings in detail recovery, so they are slightly inferior to the method of the present invention in this respect. The quality of the image restored by the method of the present invention is significantly improved compared to methods 1 to 3 and the bicubic interpolation method.

[0188] Table 1 shows the objective quality of each image, including PSNR (peak signal-to-noise ratio) and SSIM value (structural similarity criterion). As can be seen from Table 1, the method of the present invention also has a relatively obvious and stable improvement in the objective quality of restored images.

[0189] Table 1 Comparison of objective quality of restored images

[0190]

[0191]

[0192] The method of the present invention determines the type and proportion of degradation through feature extraction, and based on this, adaptively constructs a training library for low-resolution input images as a supplement to the pixel pattern distribution, providing a possible pixel distribution sample library for the input low-resolution image, thereby enhancing the robustness of the algorithm. On the other hand, by calculating the deviation of the two statistical results of the individual and the whole, the degree of deviation of the individual sample from the overall distribution is quantified, and the weight of the individual sample block in the reconstruction process is optimized, thereby improving the expression accuracy of the low-resolution input image and restoring the low-quality face image. The experimental results prove the effectiveness of the present invention from both subjective quality and objective quality, that is, the introduction of adaptive training library construction and deviation correction strategy effectively reduces the impact of severe noise on super-resolution reconstruction, thereby improving the face super-resolution processing results.

[0193] The present invention adopts the above technical scheme, and uses the posterior knowledge of the image distribution of the input low-quality image to estimate the ratio and composition of the degradation components therein, and generates an image library based on the posterior information and the input image degradation that is closest to the knowledge supplement of the pixel density distribution pattern. The prior knowledge of the image library is used to determine the expression deviation of each expression neighbor, and is used to guide the weight correction work in the subsequent reconstruction process. Through the posterior-based degradation estimation and the prior-based individual deviation estimation, a training library supplement library is adaptively constructed for low-quality images. Through the prior-based image library knowledge, weight correction guidance is provided for the expression of the input image on the training library. The combination of the two can restore a reconstructed face image with a more satisfactory effect. The present invention has universal applicability and achieves good restoration effects for general low-quality face images; especially for the restoration of face images in low-quality monitoring environments, the effect is more obvious.

[0194] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments are combined with each other. The components of the embodiments of the present application described and shown in the drawings are usually arranged and designed in various configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0195] References:

[0196] [1]H.Chang,D.-Y.Yeung,and Y.Xiong,“Super-resolution through neighborembedding,”in Proc.IEEE Conf.Comput.Vis.Pattern Recog.,Jul.2004,pp.275–282.

[0197] [2]Sung Won Park,Savvides,M."Breaking the Limitation of ManifoldAnalysis for Super-Resolution of Facial Images",ICASSP,pp:573-576,2007.

[0198] [3]Xiaogang Wang and Xiaoou Tang,“Hallucinating face byeigentransformation,”Systems,Man,and Cybernetics,Part C:Applications andReviews,IEEE Transactions on,vol.35,no.3,pp.425–434,2005.

[0199] [4]Hua Huang,Huiting He,Xin Fan,and Junping Zhang,“Super-resolutionof human face image using canonical correlation analysis,”PatternRecognition,vol.43,no.7,pp.2532–2543,2010.”

[0200] [5]C Lan,R Hu,Z Han,A face super-resolution approach using shapesemantic mode regularization.IEEE International Conference on ImageProcessing(ICIP),2021–2024,26-29 Sept.2010.

[0201] [6]Navneet Dalal and Bill Triggs.Histograms of oriented gradients forhuman detection.In CVPR,pages 886–893,2005。

Claims

1. A robust face super-resolution processing method based on empirical relationship deviation correction, characterized by: It includes the following steps: S1: Construct an original training library containing a high-resolution face image library and its corresponding low-resolution face image library, as well as two degraded augmented libraries. S2: Based on the backup training library X1, X2, ..., X in the augmented library N And the input face image x to be processed in , a new augmented library is constructed by scoring local blocks; in S2, a low-resolution input image to be processed x is in When reconstructing a high-resolution block, take out the image block to be processed corresponding to position i And the set of low-resolution spare training library blocks corresponding to position i The specific steps of block division are as follows: S2-1, for x in Each face image in is divided into blocks uniformly without any difference, the block window size is bb, and the number of overlapping pixel sizes is dd. S2-2, assuming that the length and width of each face image are imrow and imcol pixels respectively, each image block is calculated to be M = U × V, and the calculation formula is as follows: U = ceil ((imrow-dd) / (bb-dd)) V = ceil ((imcol-dd) / (bb-dd)) Among them, U represents the number of blocks in the length direction, and V represents the number of blocks in the width direction. S2-3, let i represent the position number, i=1,2,3,...,M are respectively selected from the low-resolution backup training libraries X1,X2,...,X N get of blocks; S2-4, calculate the block at position i in the feature space The degradation type label label, that is, according to Calculate to The closest distance determines the corresponding degradation type as the label of position i. express The gradient direction eigenvalue of ; Respectively The discriminant features of and a low-resolution alternate training library block set Each image block in the image is extracted based on its discriminant features to form a new feature library in In order to finalize the label; the specific steps of feature extraction and label finalization are as follows: S2-4-1, Gaussian difference iterative processing: By looping m_i times Gaussian difference, the contour enhancement information is extracted and summed up to obtain the enhanced Gaussian difference information in F G (·) represents Gaussian filter, m1,...,m m_i and n1,...,n m_i Indicates the strength of the Gaussian filter, and when the subscripts are the same, m m_i ≤n m_i , is the result of Gaussian difference, p=1,2,…,m_i; S2-4-2, image to be processed The following formula is used for gamma correction, in, represents the pixel at row I and column J after gamma processing, Represents the image to be processed For the pixel in row I and column J, gama represents the intensity of gamma processing; S2-4-3, the gamma correction result is sent to S2-4-1 for another Gaussian difference iterative processing to obtain enhanced Gaussian difference information plus and on As new S2-4-4, the new Extract the gradient direction eigenvalue to obtain S2-4-5, respectively Get S2-4-6, calculation arrive The sum of the distances of each feature at position i in is as follows: … S2-4-7, select the degradation label represented by the minimum value as the degradation type label of the corresponding position; select the minimum distance value FinalDis at position i i : S2-4-8, traverse the i value to calculate the degradation labels corresponding to all positions to obtain an N-dimensional vector L in ; Statistics L in The frequency of each degradation in the subsequent generation of the augmented matrix is ​​calculated as follows: Among them, s k represents the weight corresponding to the k-th degradation, k = 1, 2, ..., N; S2-4-9, based on weight s k Generate augmented matrix X aug , the calculation formula is as follows Among them, X aug represents the augmented matrix, s k represents the weight corresponding to the kth degradation, X k represents the k-th degraded backup training library; S3: Preprocess the high-resolution face image library and the low-resolution face image library on a block basis. Obtain the empirical correlation between each position and other position blocks and the correlation to be tested of the input image at the two levels of the high-resolution image library and the low-resolution input image, so as to prepare data for the subsequent calculation of individual deviations based on the correlation. The calculation steps of the correlation between the image block at the i-th position and all other positions in S3 are as follows: S3-1, let i represent the position number, i = 1, 2, 3, ..., M, then there are n high-resolution sample blocks at position i, that is, j represents the number of samples; S3-2, for any Calculate the jth face sample image Distance vector to all other patches The calculation formula is as follows: in, express The rth element of , r = 1, 2, 3, ..., M; Indicates block fetch The texture features of Indicates the operation of taking texture features; S3-3, after traversing the n high-resolution sample blocks at position i, n The average distance from position i to all other positions is recorded as an M-dimensional vector R. i ,Right now: S3-4, after traversing all the positions of i, a fully connected network between the positions is obtained. The association degree in the fully connected network is represented as a vector set S3-5, removing all the connections with weak correlation in the correlation vector, leaving a specified number of connections with high correlation; the specific steps of S3-5 are: S3-5-1, for each R i Only the smallest K values ​​are retained, and only values ​​less than the threshold t are retained by filtering, which is expressed as take out The index in is denoted as id i ; S3-5-2, after traversing all i values, the simplified network association vector set is recorded as: The corresponding index vector set is recorded as S3-5-3, calculate the image block set X at position i corresponding to the low-quality training sample library X according to the association network i The correlation between each image block and the face block in which it is located S4: Input image block Each neighboring block of calculates the offset vector, S5: Based on individual deviations, the input low-resolution image x is processed in Project it into the classification feature space and calculate the super-resolution expression coefficient of each block position; S6: Project the super-resolution expression coefficients and weight vectors back to the high-resolution space to obtain the high-resolution spatial re-expression result, which can be expressed as Splice high-resolution face image blocks according to position variable i Get high-resolution face image y out ;in, Represents the restored high-resolution face image y out The image patch at the i-th position, ω i Represents the weight coefficient vector of the i-th image block; Y i Represents a set of image blocks consisting of image blocks at position i in all images in the high-resolution face library.

2. The robust face super-resolution processing method based on empirical relationship deviation correction according to claim 1, characterized in that: In step S1, the high-resolution face images in the high-resolution face image library are aligned, and several different degradation processes are performed to obtain corresponding low-resolution face image libraries. The high-resolution face image library and n kinds of low-resolution face image libraries constitute n+1 kinds of training libraries. At the same time, the low-resolution face images to be processed are made to have the same size and aligned with the images in the training library. The alignment is performed by using an affine transformation method. The specific method of generating a low-resolution face image library by degrading the quality of a high-resolution face image library is as follows: suppose n pairs of well-aligned high-resolution and low-resolution face sample pairs Y and X, and the low-resolution face sample X is generated by the high-resolution face sample Y using a conventional degradation process D, where X = {x1, x2, ..., x n }, Y={y1,y2,...,y n }, then X and Y constitute the original training library; select N different degradation processes D1, D2, ..., D N , construct N kinds of degraded backup training libraries X1, X2, ..., X N .

3. The robust face super-resolution processing method based on empirical relationship deviation correction according to claim 1, characterized in that: In S4, the input image block is calculated Individual relationships and IDs of neighboring blocks at the same location i The overlap degree of the input image block is calculated Relationship id with in-stock experience i The specific calculation steps of S4 are as follows: S4-1, first The image block set X at position i corresponding to the low-quality training sample library X i In the above example, we select the K image blocks with the closest distance as the neighbor block set, and then Select the global relationship label vector of the K blocks at their locations; then calculate the global relationship label vector and the experience value id i The number of overlaps is denoted by v q,i ; S4-2, traverse all the value elements of q to obtain the deviation vector The deviation vector V i Placed on the diagonal of the K times K zero matrix to get the matrix V i ′=diag(V i ), where diag(·) represents the operation of placing the vector on the diagonal position of the zero matrix of the corresponding size.

4. The robust face super-resolution processing method based on empirical relationship deviation correction according to claim 3, characterized in that: In S5, the image blocks According to library X and X aug The image block library X corresponding to position i i and The corresponding expression coefficient ω is calculated through the following objective function i : in, is a diagonal matrix, The elements on the diagonal are calculated as: λ represents the balance coefficient, which is set by experience.

5. A robust face super-resolution processing system based on empirical relationship deviation correction, adopting a robust face super-resolution processing method based on empirical relationship deviation correction according to any one of claims 1 to 4, characterized in that: The system includes: Original training library construction model: used to construct a training library containing a high-resolution face image library and its corresponding low-resolution face image library containing multiple degradations; Specific degraded image library generation module: based on the original training library model and input image x in , through feature extraction, comparison, statistical scoring, and then weighted construction of low-resolution image X for supplementary information aug ; Relevance calculation module: establishes two models of relevance experience value and individual value through high-resolution and low-resolution channels in the original training library; Offset calculation model: The individual offset is calculated by using the nearest neighbors of the input image through the correlation degree, so as to prepare data for the subsequent reconstruction process guided by individual deviations; Coefficient retrieval module: Based on individual deviation as a guide and according to the augmented library X aug The expression coefficient of the low-resolution augmented library is calculated with the original library X. High-resolution image block reconstruction module: reconstruct high-resolution image blocks based on the expression coefficients and the blocks corresponding to the high-resolution augmentation library; Stitching module: used to stitch high-resolution face image blocks according to position i Get high-resolution face images.