Degraded image restoration and recognition method based on sparse low-rank prior

Abstract

The present invention relates to a degraded image restoration and recognition method based on sparse low-rank prior. Degraded face images are analyzed, and the degraded face images are restored and recognized at the same time through the sparse low-rank prior of the face images, which not only improves the efficiency of the face restoration algorithm, but also ensures the recognition accuracy, solving the challenging problem of degraded face recognition. An iterative algorithm is proposed, which combines degraded face image restoration and face recognition. The face image restoration part aligns the face, and the recognition part recognizes the face image. Using the sparse low-rank prior, the face image is restored and aligned, and then the restored and aligned face image is input into the face recognition, the efficiency of the restoration and alignment algorithm is improved through the recognition effect, and the recognition accuracy is guaranteed through the restoration and alignment. The algorithm solves the problems of image alignment, image de-occlusion and image recognition of degraded face images, and verifies its effectiveness and advancement.

Description

Technical Field

[0001] The invention relates to an image restoration and recognition method, in particular to a degraded image restoration and recognition method based on sparse low-rank prior. Background Art

[0002] Image is the basis of human vision, an objective reflection of natural scenery, and the main source of human perception of the objective world and external information. Improving image quality and enhancing the application value of images have always been one of the most basic and important research topics in the field of image processing and computer vision, and have received extensive attention from scholars in the fields of mathematics, computer vision and signal processing. This is because images play the most important role in human production and life, and are an important tool for humans to obtain information and understand the world, especially in today's highly developed science and technology. From taking photos in daily life, calligraphy and painting in artistic creation, film and television, to image data for medical diagnosis, satellite launches, manned moon landings, and Mars exploration in scientific exploration, images are all involved. Although there are many ways to obtain images and the technology for obtaining images is constantly updated, images are inevitably affected by equipment, environment and scenes during the process of acquisition, conversion, transmission and display, resulting in degradation of image quality, which in turn affects the subsequent processing and application of images. Therefore, in order to improve the quality of degraded images and obtain more real and reliable information, degraded images must be processed.

[0003] Image recognition is one of the most important uses of image processing. Its purpose is to allow a trained computer to extract information from the image to be detected and identify it. The reason why image recognition has been deeply studied and developed rapidly is because of its wide and important application value. At present, image recognition technology is mature and widely used in the recognition of faces, numbers and other objects. The basic process of image recognition is to preprocess the image first, then extract features, and finally use a classifier for recognition (i.e. classification). Usually, in order to solve the problem of degraded image recognition, the method that comes to mind is to restore the image first to obtain a better quality image, and then input the restored result into the recognition system. The problem with this direct method is that many restoration algorithms are designed to improve human visual perception, not machine perception, so there is no guarantee of recognition improvement. Worse, when the degradation model is unknown, general restoration algorithms (such as deblurring) perform poorly on some real images that do not show strong edge structures (such as faces), and usually introduce serious artifacts, which actually affect the recognition performance. Moreover, separating the restoration task from the recognition task will lead to a lack of information interaction between them, that is, the restoration task execution process does not take into account the recognition requirements, and the recognition task does not necessarily require the restoration task to provide excessively high global image clarity. Summary of the invention

[0004] In view of the above deficiencies in the prior art, the purpose of the present invention is to provide a method for restoration and recognition of degraded images based on sparse low-rank prior. The sparsity prior can characterize that the degraded image can be expressed as a sparse combination of input image data under effective restoration, and the coefficients of the sparse expression can be used to identify degraded image data. In order to solve the problem of recognition of degraded images, we intend to use data-driven sparsity prior knowledge transfer to realize cross-task algorithm research for closed-loop collaborative processing of restoration and recognition. We will combine the two algorithms to achieve the purpose of mutual promotion, and align and recognize facial images with occlusion noise and misalignment. Thereby improving image quality, enhancing the application value of images, and ensuring accurate recognition of degraded images.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a degraded image restoration and recognition method based on sparse low-rank prior, which is used to simultaneously restore and recognize degraded images, maintain image recognition accuracy, and improve image restoration speed; including:

[0006] Step 1: Collect images, preprocess and normalize them, and create a training sample set;

[0007] Step 2: a. Establish a degraded image restoration and recognition model based on sparse low-rank prior:

[0008]

[0009] Among them, I is the input face image, I 0 is the face image after restoration of I, let column vector I x is to restore the aligned face image I 0 Resize to m*1, m=w×h, E is the error caused by partial occlusion or image damage, is the Jacobian matrix, τ is the alignment factor of the domain affine and projective transformation, Δτ is the increment of τ each time it is updated, D is the face dictionary, α is the sparse representation of the face dictionary, μ, λ, η, γ are weights, where Y∈R m×n is the Lagrange multiplier matrix;

[0010] b. Training: Input the image to the model, and train the four unknown numbers I in the model. 0 ,E,τ,α is solved iteratively, and the recognition and classification results are output according to the sparse index SCI, and the optimization model that meets the solution convergence conditions is obtained;

[0011] Step 3: Input the image to be recognized into the optimization model, automatically output the recognition and classification results, and realize the alignment, restoration and recognition of the input face image I.

[0012] The preprocessing and normalization include:

[0013] Resize each image to a column vector of size w*h, manually label the category i, and put the column vectors with the same category into the same class set D i In the above example, the column vectors of all images are combined into a face dictionary matrix D = [D 1 ,D 2 ,…,D k ] There are k classes in total, and the image set of the i-th class is D i ; Normalize each column to unit vector.

[0014] The four unknowns in the model I 0 ,E,τ,α,is solved iteratively using the Admm method, which minimizes one function at a time: select an unknown number, retain the items related to the unknown number in the model, minimize them, and determine the unknown number.

[0015] Determine the category of the current input image:

[0016]

[0017] where δ i To describe the function, Through For each element in the matrix expression, keep the elements related to the target class i and set the other elements to zero.

[0018] calculate The sparse index SCI is calculated as follows:

[0019]

[0020] Among them, k is the number of face image categories, and i represents the category sequence number;

[0021] if Then output the recognition and classification results Otherwise, the image is considered invalid and a message "Invalid image" is displayed.

[0022] The solution convergence condition is that the iteration stops when E converges.

[0023] The present invention has the following beneficial effects and advantages:

[0024] (1) The method of the present invention establishes an algorithm that can restore, correct and recognize simultaneously. Compared with the algorithm that first restores and corrects the face image and then recognizes the face image, the restoration and re-recognition process is shortened, the number of restoration and correction iterations is reduced, and the restoration and correction are more efficient while ensuring recognition accuracy.

[0025] (2) The method of the present invention is superior to existing algorithms in the restoration and recognition of degraded images and has stable performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of restoration and recognition of degraded images; DETAILED DESCRIPTION

[0027] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation method of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention, so the present invention is not limited by the specific implementation disclosed below.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0029] like Figure 1 ,Degraded image restoration and recognition based on sparse low-rank prior,consists of two parts: the alignment operation, in planar regions with significant affine or projective deformation, our method can accurately restore the inherent low-rank texture and precise domain transformation, thereby restoring the 3D geometry and appearance of the planar region.

[0030] The formula of the face image restoration and alignment algorithm based on sparse low-rank prior is as follows:

[0031]

[0032] Among them, the image I∈R w×h , initial transformation τ∈G (affine or projective) I 0 ∈R w×h is the output image.

[0033] Recognition operation, describes the recognition problem as a classification problem in a multi-linear regression model, and believes that sparse signal representation provides a solution to this type of problem. The errors caused by occlusion and corrosion are usually sparse with respect to the standard (pixel) orthogonal basis, so this framework can consistently handle these errors.

[0034] The formula of the face recognition algorithm based on sparse low-rank prior is as follows:

[0035]

[0036] Where x is the input image and D is the dictionary constructed from the face image.

[0037] The two algorithms are iterated simultaneously, and face alignment and face recognition operations are completed by optimizing a model. Using sparse low-rank prior, face images are restored and aligned while face recognition is performed. The efficiency of the restoration and alignment algorithm is improved through the recognition effect, and the recognition accuracy is guaranteed through restoration and alignment.

[0038] 1. Degraded image restoration and recognition model

[0039] The technical solution adopted by the present invention to solve the technical problem is: a sparse low-rank

[0040] Priori degraded image restoration and recognition, using sparse low-rank prior to simultaneously restore and correct degraded images and recognize them, restoration and correction improve recognition accuracy, recognition improves restoration and correction efficiency, the algorithm includes the following steps:

[0041] Training sample set:

[0042] (1) Resize each image to w*h (the size needs to be consistent), let m = w*h, resize each image into an m*1 column vector, manually label the category i, and put the column vectors with the same category into the same class set D i In the above example, the column vectors of all images are combined into a matrix D = [D 1 ,D 2 ,…,D k ] There are k classes in total, and the image set of the i-th class is D i , normalize each column to a unit vector.

[0043] (2) Input a w*h face image I and a threshold τ. Use the following formula to complete the alignment, restoration and recognition of the input face image I:

[0044]

[0045] Among them, I is the input face image, I 0 is the face image after restoration of I, column vector I x is to restore the aligned face image I 0 Resize to m*1, m=w×h, E is the error caused by partial occlusion or image damage, is the Jacobian matrix, τ is the domain transformation (affine, projective) positive factor, Δτ is the increment of τ each time it is updated, D is the face dictionary, α is the sparse representation of the face dictionary, μ, λ, η, γ are weights, where Y∈R m×n is the Lagrange multiplier matrix;

[0046] (3) Perform local optimization on the combined algorithm to perform calculations:

[0047] Formula (1) is difficult to solve directly. Therefore, people usually choose to use an alternating strategy to approximately minimize the Lagrangian function: for the four unknowns I 0 ,E,τ,α minimizes one function at a time:

[0048] The input face image I is restored, corrected and recognized at the same time, that is, I in formula (1) 0 , E, and α are optimized respectively. First, E is optimized, that is, the error of the input face image I is calculated, such as occlusion, noise, etc.

[0049]

[0050] Formula (2) can be transformed into

[0051]

[0052] Aligned face image I 0 And optimize the positive factor τ:

[0053]

[0054] Finally, face recognition is performed. Since the error E has been calculated in the optimization alignment part, the error does not need to be considered in recognition. x Identification, that is, optimizing α, is obtained

[0055]

[0056] Use this method to make classification decisions:

[0057]

[0058] where δ i is a descriptive function that selects coefficients that are only associated with the i-th class. Through For each element in the matrix expression, keep the elements related to the target class i and set the other elements to zero.

[0059] (4) In order to prevent the appearance of face images that are not in the dictionary set but are in the dictionary set, we introduce the sparsity concentration index (SCI). The calculation formula of SCI is as follows:

[0060]

[0061] in, Through For each element in the matrix expression, keep the elements related to the target class i and set the other elements to zero.

[0062] calculate If Then output the classification result Otherwise, the image is considered invalid and a message "Invalid image" is displayed.

[0063] (5) Output the corrected and restored image I after iteration 0 And the classification results

[0064] Note:

[0065] ① The threshold ρ is manually entered at the beginning, with a range of (0.5-1) and a recommended value of 0.85.

[0066] ②Invalid images: faces that are not in the dictionary (strangers), images that are not faces.

[0067] The two parts of the above algorithm are iteratively operated to finally obtain a restored and corrected face image and an accurate classification result.

[0068] 3. The theoretical basis of the algorithm model. The algorithms of the prior art are described in 3.1 and 3.2 below. By comparing the following prior art methods with this method, the differences in the models are explained:

[0069] The algorithm is mainly based on sparse low-rank prior, and iteratively performs recovery correction and recognition. It uses recovery correction to improve recognition accuracy, and uses recognition to improve recovery correction efficiency.

[0070] 3.1. Degraded Image Restoration

[0071] In terms of restoring alignment, low-rank textures are used to capture geometrically meaningful structures in images, which include traditional local features such as edges and corners, as well as various regular, symmetric patterns that are prevalent in urban environments and man-made objects. These low-rank texture methods utilize convex optimization to enable robust recovery of high-dimensional low-rank matrices despite severe sparsity errors. In planar regions with significant affine or projective deformations, the restoration method can accurately restore the inherent low-rank texture and precise domain transformation, thereby restoring the 3D geometry and appearance of the planar region. This method can effectively handle many regular and nearly regular patterns or low-rank objects, such as symmetrical patterns, building facades, printed text, and faces. The relevant algorithm formula is as follows:

[0072] (1) First, the input image of the algorithm is I∈R w×h , initial transformation τ∈G (affine or projective), weight λ>0.

[0073]

[0074] Among them I 0 ∈R w×h is the output image

[0075] (2) For images taken in the real world, partial occlusion and other types of damage often occur. Therefore, the error E

[0076]

[0077] (3) But constraints The nonlinearity is due to This constraint arises from the complex dependence of the transformation on τ. When the change in τ is small, we can approximate this constraint by linearizing the current estimate of τ.

[0078]

[0079] Through the above algorithm cycle operation, the image can be restored and aligned. However, due to the large number of algorithm cycles, the efficiency of the recognition operation is low.

[0080] 3.2 Image Recognition

[0081] On the recognition side, we consider the problem of automatically recognizing frontal faces with different expressions, illumination, occlusions, and disguises. We formulate this type of recognition problem as a classification problem in a multilinear regression model, and argue that a new theory of sparse signal representation provides a solution to this problem. We propose a general classification algorithm for object recognition (image-based) based on a sparse representation computed by l1 minimization. Errors due to occlusion and corrosion are usually sparse with respect to a standard (pixel) orthogonal basis, and thus this framework can handle these errors consistently. The theory of sparse representation can help predict how large an occlusion the recognition algorithm can handle and how to choose training images to maximize robustness under occlusion.

[0082] Recognize faces with errors (noise, blur, etc.):

[0083] (1) Input:

[0084] ρ (only used in step 5)

[0085] Training set samples: Each image has a size of w*h (the size needs to be consistent), let m = w*h, resize each image into an m*1 column vector, and combine the column vectors of all images into a matrix D = [D 1 ,D 2 ,…,D k ] There are k classes in total, and the image set of the i-th class is D i .

[0086] Test sample: y∈R m , y is an m*1 column vector (the same size as the image in A)

[0087] (2) Normalize each column of A to a unit vector

[0088] (3) In order to increase the robustness to occlusion, the equation is expanded:

[0089] y=y 0 +e 0 =Dx 0 +e 0 (11) is transformed into:

[0090] The D matrix is ​​expanded into the B matrix, and the x0 matrix is ​​expanded into w 0 . I is the identity matrix, w 0 To be requested.

[0091] Solve the l1 norm minimization problem:

[0092]

[0093] Get the solution:

[0094] Restore unobstructed face images:

[0095] (4) Calculation and The dimensions are the same, only The element corresponding to the i-th category is 0, and the other elements are 0.

[0096] Compute the residual for each class:

[0097]

[0098] Wherein, i=1, 2, 3, ..., k;

[0099] Find the i with the smallest residual, i is the final classification of y.

[0100] The above algorithm can be used to recognize the restored and corrected face image, but the algorithm that first restores and corrects and then performs the recognition operation is less efficient. The iterative algorithm formed by the two algorithms is superior to the former in terms of efficiency and accuracy.

[0101] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should be regarded as within the scope of protection of the present invention.

Claims

1. A degraded image restoration and recognition method based on sparse low-rank prior, characterized in that: Used for simultaneous restoration and recognition of degraded images; including: Step 1: Collect images, preprocess and normalize them, and create a training sample set; Step 2: a. Establish a degraded image restoration and recognition model based on sparse low-rank prior: Among them, I is the input face image, I 0 is the face image after restoration of I, let column vector I x is to restore the aligned face image I 0 Resize to m*1, m=w×h, E is the error caused by partial occlusion or image damage, is the Jacobian matrix, τ is the alignment factor of the domain affine and projective transformation, Δτ is the increment of τ each time it is updated, D is the face dictionary, α is the sparse representation of the face dictionary, μ, λ, η, γ are weights, where Y∈R m×n is the Lagrange multiplier matrix; b. Training: Input the image to the model, and train the four unknown numbers I in the model. 0 ,E,τ,α is solved iteratively, and the recognition and classification results are output according to the sparse index SCI, and the optimization model that meets the solution convergence conditions is obtained; Step 3: Input the image to be recognized into the optimization model, automatically output the recognition and classification results, and realize the alignment, restoration and recognition of the input face image I.

2. The degraded image restoration and recognition method based on sparse low-rank prior according to claim 1, characterized in that: The preprocessing and normalization include: Resize each image to a column vector of size w*h, manually label the category i, and put the column vectors with the same category into the same class set D i In the above example, the column vectors of all images are combined into a face dictionary matrix D = [D1, D2, ..., D k ] There are k classes in total, and the image set of the i-th class is D i ; Normalize each column to unit vector.

3. The degraded image restoration and recognition method based on sparse low-rank prior according to claim 1, characterized in that: The four unknowns in the model I 0 ,E,τ,α,is solved iteratively using the Admm method, which minimizes one function at a time: select an unknown number, retain the items related to the unknown number in the model, minimize them, and determine the unknown number.

4. The degraded image restoration and recognition method based on sparse low-rank prior according to claim 1, characterized in that: Determine the category of the current input image: where δ i To describe the function, Through Each element in the matrix expression retains the elements related to the target class i, and the other elements are set to zero.

5. The degraded image restoration and recognition method based on sparse low-rank prior according to claim 4, characterized in that: calculate The sparse index SCI is calculated as follows: Among them, k is the number of face image categories, and i represents the category sequence number; if Then output the recognition and classification results Otherwise, the image is considered invalid and a message "Invalid image" is displayed.

6. The degraded image restoration and recognition method based on sparse low-rank prior according to claim 1, characterized in that: The solution convergence condition is that the iteration stops when E converges.

Images