Image restoration processor utilizing framework to generate sparse regularizers for image restoration

Abstract

The image restoration processor includes an image receiver, a matrix converter, a frame memory, a modifier, a matrix decomposer, and an image reconstructor. An image receiver receives one or more degraded images. A matrix converter shapes image data of the one or more degraded images into one or more target matrices. A framework memory stores a framework for generating one or more sparse facilitated regularizers. A modifier reads the framework memory and applies the framework to modify the M estimators and accordingly outputs hybrid M estimators and corresponding sparse regularizers. A matrix decomposer receives one or more target matrices and a sparse regularizer and applies the sparse regularizer to the RPCA by decomposing one of the one or more target matrices into a sum of a low-rank matrix and a sparse matrix. The image reconstructor integrates the decomposition results from the matrix decomposer to form one or more restored images.

Description

Technical Field

[0001] The present invention generally relates to image restoration techniques. More specifically, the present invention relates to an image restoration processor that applies a framework to generate a sparse regularizer for image restoration. Background Art

[0002] Principal component analysis (PCA) refers to finding potential low-dimensional linear subspaces from high-dimensional data. This is widely used for dimensionality reduction and is widely used in many applications such as computer vision, bioinformatics, and signal and image processing. Although PCA works well in the presence of zero-mean Gaussian noise, it is easily affected by outliers. To overcome this shortcoming, robust principal component analysis (RPCA) was proposed to decompose the data contaminated by outliers into a superposition of a low-rank matrix and a sparse matrix. RPCA has been applied to fields such as hyperspectral image restoration, shadow removal, and video separation. The low-rank plus sparse decomposition model can be viewed as a weighted linear combination of the rank and l0-norm minimization problem, which is an NP-hard problem. To solve this problem, a common approach is to replace the rank function and l0-norm with the nuclear norm and l1-norm, respectively, resulting in a convex optimization formula, which is called principal component pursuit (PCP). In addition, it has been shown that, under the condition of satisfying the incoherence condition, the minimization of the weighted linear combination of the nuclear norm and the l1-norm can achieve accurate low-rank sparse component recovery with high probability. To solve PCP, many efficient algorithms based on singular value thresholding (SVT), accelerated proximal gradient and enhanced Lagrange multipliers have been developed.

[0003] Although convex relaxation is relatively easy to perform in optimization, its estimation is biased. For example, when the nuclear norm is used to find low-rank components, all non-zero singular values ​​will be underestimated and shrunk by the same constant. It has been shown that shrinking large singular values ​​by smaller values ​​will lead to better recovery performance in image denoising and inpainting as well as background subtraction. There are two main strategies to mitigate the bias caused by the nuclear norm. The first is to weight the singular values ​​differently by updating the weights at each iteration, which is similar to reweighted l1 minimization. A related work has extended this technique to low-rank matrix approximation and adopted weighted nuclear norm minimization (WNNM) as an alternative to rank minimization and adopted l1-norm to resist outliers. In addition, to facilitate weight adaptation, a weighted MCP (weighted minimax-concave penalty; WMCP) is developed, which uses a clever trick to unfold the MCP and exploits the alternating direction method of multipliers (ADMM) to find solutions.

[0004] The second strategy is to use nonconvex sparse regularizers to reduce the estimation bias. There have been many studies showing that nonconvex regularizers outperform convex relaxation methods. One study developed a fast proximal algorithm with nonconvex regularizers (such as smoothly clipped absolute deviation (SCAD) and MCP) for low-rank matrix learning. One study suggested that by using l p -norm (0<p≤1) is used as a regularizer to impose constraints on low-rank and sparse terms to achieve robust PCA, but l p -norm in addition to Except for the special case of , there is no closed-form expression for its proximal operator. There is also a design improvement l p -norm to impose low-rank and sparse constraints to achieve low-rank and sparse decomposition, thereby achieving good performance in background separation. In addition, other non-convex regularizers, including exponential-type penalty (ETP) and Laplace function, are applied to low-rank matrix recovery by iteratively reweighting the nuclear norm. However, none of them have explicit expressions for the corresponding proximal operators.

[0005] Although many non-convex regularizers have been proposed, they still have the above limitations. The field of low-rank matrix recovery faces the challenges of dealing with large-scale datasets, complex noise models, and incomplete or damaged data. Traditional methods may not be sufficient to cope with these challenges, so it is necessary to develop new methods that can effectively handle these problems and provide accurate recovery results. Summary of the invention

[0006] The object of the present invention is to provide a device and a method to solve the above-mentioned problems in the prior art.

[0007] According to one aspect of the present invention, an image restoration processor is provided. The image restoration processor includes an image receiver, a matrix converter, a framework memory, a modifier, a matrix decomposer and an image reconstructor. The image receiver is configured to receive one or more degraded images. The matrix converter is configured to reshape the image data of one or more degraded images into one or more target matrices. The framework memory is configured to store a framework for generating one or more sparse promotion regularizers, which can generate one or more sparse promotion regularizers and derive their theoretical properties and closed form neighbor operators. The modifier is configured to read the framework memory and apply the framework to modify the M estimator, and output the mixed M estimator and the corresponding sparse regularizer accordingly. The matrix decomposer is configured to receive one or more target matrices and sparse regularizers, and to apply the sparse regularizer to a robust principal component analysis (RPCA) method by decomposing one of the one or more target matrices into a sum of low rank and sparse matrices. The image reconstructor integrates the decomposition results from the matrix decomposer to form one or more restored images from the one or more degraded images using a low-rank sparse matrix.

[0008] According to one aspect of the present invention, an image restoration processor is provided. The image restoration processor includes an image receiver, a matrix converter, a framework memory, a modifier, a matrix decomposer, an image reconstructor and an image selector. The image receiver is configured to receive a degraded image. The matrix converter is configured to reshape the image data of the degraded image into a target matrix. The framework memory is configured to store a framework for generating one or more sparse promotion regularizers, which can generate one or more sparse promotion regularizers and derive their theoretical properties and closed form neighbor operators. The modifier is configured to read the framework memory and apply the framework to modify more than one M estimator, and accordingly output more than one mixed M estimator and more than one sparse regularizer corresponding to more than one mixed M estimator. The matrix decomposer is configured to receive a target matrix and different sparse regularizers, and to apply different sparse regularizers to a robust principal component analysis (RPCA) method by decomposing the matrix into a sum of a low-rank matrix and a sparse matrix. The image reconstructor integrates the decomposition results from the matrix decomposer, uses different sparse regularizers, and utilizes low-rank and sparse matrices to form more than one restored image from the degraded image. The image selector is configured to receive the restored images from the reconstructor and determine which of the restored images to output based on image evaluation metrics of all the restored images.

[0009] In one embodiment, a framework is designed to generate different non-convex sparsity-promoting regularizers under some relatively mild conditions. Although the obtained regularizers may be non-convex, their Moreau envelopes can be shown to be convex, and their analytical solutions are given. In addition, the framework is also applied to three popular M-estimators, namely Welsch, Cauchy, and German-McClure, to generate corresponding sparse regularizers, which are due to their considerable success in image restoration, compressed sensing, and subspace clustering. Although these M-estimators provide implicit regularizers (IR) through semi-quadratic optimization, sparsity cannot be achieved (see Appendix A). Sparsity-promoting regularizers related to the three M-estimators are generated in this work. In addition, an effective hyperparameter selection strategy is derived and their physical meanings are explained.

[0010] In the framework of the present invention, the generated sparse regularizer can be considered as a non-convex proxy to implement low-rank plus sparse decomposition. Unlike PCP and WNNM-RPCA, both of which use l1-norm to combat sparse outliers, the present invention uses a designed non-convex regularizer. The resulting optimization problem is solved by ADMM with convergence guarantee. In short, the contributions provided by the above configuration are: (1) We introduce a framework for generating sparsity regularizers and derive their theoretical properties as well as closed-form neighbor operators. (2) We apply the framework to the Welsch, Cauchy, and German-McClure functions, generate three different sparsity-promoting regularizers, and analyze their parameter choices. (3) Then, a sparse regularizer is used to achieve low-rank plus sparse decomposition, thereby obtaining an ADMM-based algorithm with theoretical guarantees. (4) Extensive experiments are conducted on synthetic and real data to verify the effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings, in which:

[0012] Figure 1 The curves of Huber and truncated quadratic functions are shown;

[0013] Figure 2 Shows a convex function f(x) and its conjugate function f at point y * (y), where f*(y) is the maximum gap between yx and f(x), where and includes parts (a) and (b);

[0014] Figure 3 Draw and The numerical calculation curve of

[0015] Figure 4 It shows that according to various embodiments of the present invention, under different regularizers, when λ=0.5

[0016] Figure 5 Algorithm 1 describes the previous optimization steps according to various embodiments of the present invention;

[0017] Figure 6 Different rank and outlier ratio settings {p r , p s} algorithm phase transition diagram, including parts (a), (b), (c), (d), (e), (f), (g) and (h);

[0018] Figure 7 Convergence curves of different techniques are shown, including parts (a), (b), (c), (d), (e), (f), and (g);

[0019] Figure 8shows the restored image of the National Mall in Washington, D.C. with 126-73-12 as the RGB band, including parts (a), (b), (c), (d), (e), (f), (g), (h), (i), and (j);

[0020] Fig. 9 The restored images of HYDICE city data with 6-188-36 as RGB band are shown, including parts (a), (b), (c), (d), (e), (f), (g), (h), (i) and (j);

[0021] Fig.10 Images from two datasets are shown, with the first and second rows of images coming from the CAVE and Harvard datasets, respectively;

[0022] Fig.11 The average PSNR curves of different algorithms as a function of the band are shown, including parts (a) and (b);

[0023] Fig.12 The restored image with 23-13-4 as the RGB band is shown, in which one area is magnified 6 times, including parts (a), (b), (c), (d), (e), (f), (g), (h), (i) and (j);

[0024] Fig.13 The restored image with 25-18-8 as the RGB band is shown, in which one area is magnified 8 times, including parts (a), (b), (c), (d), (e), (f), (g), (h), (i) and (j);

[0025] Fig.14 A schematic diagram of an image restoration processor according to an embodiment of the present invention is shown, which is used to perform image restoration using a framework for generating a sparse regularizer;

[0026] Fig.15 A schematic diagram showing an image restoration processor according to another embodiment of the present invention, which is used to perform image restoration using a framework for generating a sparse regularizer;

[0027] Fig.16 P'(x) curves of different regularizers are shown;

[0028] Fig.17 The relationship between RRE and λ in logarithmic scale is shown, including parts (a), (b), (c), (d), (e), (f), (g), (h), and (i). DETAILED DESCRIPTION

[0029] In the following description, an image restoration processor using a framework for generating a sparse-promoting regularizer, etc. will be described as a preferred example. It will be apparent to those skilled in the art that modifications, including additions and / or substitutions, may be made without departing from the scope and spirit of the invention. Specific details may be omitted in order not to obscure the invention; however, this disclosure is written to enable those skilled in the art to practice the teachings herein without undue experimentation.

[0030] In order to more clearly illustrate the embodiments of the present invention, symbols and basic operations are explained before describing the embodiments.

[0031] A) Symbols:

[0032] Scalars and matrices are denoted by italic and bold capital letters, respectively. The (i, j) entry of a matrix A is denoted by A ij express,(·) T is the transpose operator. In addition, yes The Frobenius norm of , and unless otherwise specified, the matrix norm refers to the Frobenius norm, that is, ||A||=||A| F The first and second derivatives of a differentiable function g(x) are represented by g'(x) and g"(x), respectively. In addition, |α| represents the absolute value of α, refers to the collection of subgradients which reduce to the derivatives of a differentiable function.

[0033] Related Work:

[0034] (1) Robust PCA: In order to make PCA robust against outliers, RPCA is developed by decomposing the degenerate matrix into the sum of a low-rank matrix and a sparse matrix. Many studies have shown that this decomposition can be achieved accurately with high probability by solving the following convex relaxation formula: where X, M, and E are the contaminated matrix, low-rank matrix, and sparse matrix, respectively, and the nuclear norm ||M|| * is the sum of the singular values ​​of M. The l1-norm of E, i.e. ||E||1, is a regularizer to handle sparse outliers, and λ is a regularization parameter that controls the relative weight between the norm and the regularizer. However, the nuclear norm underestimates the non-zero singular values, especially the large singular values ​​that contain the main information of the matrix. To mitigate this effect, it is recommended to use the weighted nuclear norm, which is extended from the weighted l1-norm of the vector, resulting in:

[0035] The weighted nuclear norm is the weighted l1-norm of the vector of singular values. Nevertheless, compared with the non-convex penalty, the l1-norm in (2), i.e., ||E||1, is still susceptible to large outliers. In addition, it is well known that non-convex sparse regularization of low-rank matrices can reduce the bias of singular value estimates, so the following double non-convex penalty for the low-rank sparse component model is developed as: in and is a non-convex penalty function.

[0036] (2) Proximity operator: For an appropriate lower semi-continuous function Regularizer The Moreau envelope of is multiplied by a scalar λ>0 and is defined as: Its solution can be found by the operator: This is called In fact, the neighboring operator has been widely used in signal denoising, image restoration and sparse representation. When is convex, (4) is strictly convex and has an optimal solution. When φ(·) is non-convex, (4) is usually a non-convex problem. Recent studies have shown that even when φ(·) is non-convex, the solution of (4) can be obtained, but it requires iteration, that is, an explicit neighbor operator cannot be found. In addition, a related work generated a series of non-convex IRs to achieve robustness using semi-quadratic optimization, but the IRs they generated are not sparsity-promoting regularizers, which limits their application. A common sparsity-promoting regularizer is the l1-norm, whose neighbor operator can be found in (10) discussed below. However, as mentioned earlier, this suffers from the bias problem.

[0037] One of the purposes of the present invention is to solve the problem of recovering a matrix with a low rank constraint in the presence of sparse outliers. Non-convex regularizers may not have closed-form neighbor operators, so iterations are required to find their expressions, which increases the computational load. To solve this problem, a framework according to various embodiments is designed to generate sparse-boosted regularizers with closed-form neighbor operators. Although the resulting regularizers may be non-convex, their Moreau envelopes are convex.

[0038] Then, the framework is applied to three popular M-estimators, namely Welsch, Cauchy and German-McClure functions, and firstly, the sparsity-promoting regularizers associated with them are generated, thus enriching the variety of regularizers. Next, the parameter selection strategy for the obtained regularizers is also proposed. In addition, the regularizers are used for low-rank and sparse decomposition applications, resulting in three algorithms based on the alternating direction method of multipliers with convergence guarantees. The property that any of the generated limit points are critical points is also proved. Finally, extensive numerical experiments are performed on synthetic and real data to demonstrate the effectiveness of the developed method.

[0039] First, the framework for generating sparsity-enhanced regularizers is discussed. Figure 1 The Huber curve and the curve of the truncated quadratic function are shown. Review the definitions of the Huber and truncated quadratic functions.

[0040] Definition 1: Huber function Defined as:

[0041] Definition 2: Truncated quadratic function Defined as:

[0042] l h and l t The curve is as Figure 1 In addition, both can be written as: For the Huber function, And for the truncated quadratic function:

[0043] for and The solution of (8), which is called the proximity operator, is: and They respectively mean and is a sparsity-enhancing regularizer. It is worth noting that the neighbor operator of the l0-norm is the same as (11).

[0044] In this invention, a framework is provided to generate different sparsity-enhancing regularizers by generalizing (6) and (7) for |x|>λ, such as Figure 1In various embodiments, the provided loss function has the following form: Where g(x) is a continuous function when x>0, and g′(x)≥0, and a and b are constants that make l(x) continuously differentiable at x=λ. Therefore, a=λ / g, (λ)>0(g′(λ)≠0), and b=λ 2 / 2-ag(λ). It can be easily seen that equations (6) and (7) are special cases of equation (12).

[0045] Theorem 1: Definition For x>λ, if h′(x)>0 and h″(x)>0, that is, when x>λ, h(x) is strictly convex, then equation (12) can be used to generate a sparse-enhancing regularizer.

[0046] Proof: Prove that a function can be transformed into a sparse regularizer through Legendre-Fenchel transformation. Given a function f(x), its conjugate f * (y) is: If f(x) is convex, then f * The conjugate of (x) is f(x), that is,

[0047] Note that sup is maximum when f(x) is a convex and continuous function. Since h(x) is a strictly convex function, the convex function f(x) can be defined as:

[0048] therefore, in

[0049] Since f(x) is convex, we can get:

[0050] Combining (15) and (18), we can easily obtain:

[0051] Similar to the regularizer in related work, The expression of is usually unknown, which is called an implicit regularizer (IR). Before solving (19), the following lemma is introduced.

[0052] Lemma 1: (Inversion rule of subgradient relation) For any suitable lower semicontinuous convex function f(x), we have:

[0053] Since the solution of (19) is the same as that of (18), according to Lemma 1, the solution of y in (19) is: This means that g，λ (x) The generated IR can make the solution sparse, and is nondecreasing, since h(x) is strictly convex for |x|>λ.

[0054] Figure 2 It shows that the convex function f(x) and its conjugate function f at y * (y), f * (y) is the maximum gap between yx and f(x), where Part (a) is an arbitrary convex function, while part (b) is a convex function generated by (12).

[0055] Next, we explain why sparsity promotes the generation of regularizers through (12), and compare Figure 2 Compare with other convex functions in . It can be seen that except Figure 2 Except for x=0 in (a), is not zero, but for Figure 2 The convex function in (b) is And |x|≤λ, and when |x|>λ, is not equal to 0 and is unique, because f(x) is convex. In addition, the following proposition describes the properties of IR, namely, The proof process can be found in Appendix B.

[0056] Proposition 1: Although may be non-convex, but problem (19) is convex, and It has the following properties: 1) is symmetrical, that is 2) is continuous, but not smooth at y=0. 3) is non-negative and increasing when y>0. 4) If g(x) is concave when x>λ, then is concave and satisfies the triangle inequality, namely For any

[0057] Secondly, generalization by using g(x) will be discussed. It is worth mentioning that g(x) can represent many convex or non-convex functions. In various embodiments, a non-convex g(x) is considered, and three different sparsity-promoting regularizers are generated by using g(x) for three M estimators, namely Welsch, Cauchy, and German-McClure. Compared with the case when x>0 and g(x) as a convex function or a concave function, it is difficult to analyze these M estimators because they are non-convex and non-concave.

[0058] Regarding the modification of the Welsch function, the Welsch function is an M estimator, and its expression is:

[0059] This is because, when a Gaussian kernel is used as the correlation entropy function and σ is the kernel parameter, minimizing equation (22) has been shown to be equivalent to maximizing the correlation entropy criterion. Therefore, the theory provided by the present invention is applied to the Welsch function. When g(x) = ρ welsch (x), according to (12), the mixed Welsch function is: It can be rewritten through formula (19) as:

[0060] The solution to this is:

[0061] Regarding the modification of the Cauchy function, unlike the Welsch M estimator which is bounded from above, the Cauchy function is unbounded from above and its expression is: where γ is the scale parameter of the Cauchy distribution. Therefore, when g(x) = ρ cauchy (x), the mixed Cauchy function can be obtained as: in In addition, there are: According to formula (21), the solution is:

[0062] Regarding the modification of the German_McClure function, the expression of the German_McClure (GMC) M estimator is: where τ>0 is the scale parameter. Therefore, when g(x)=ρgmc (x), there are: is equivalent to: in is with l τ，λ (x) related sparse regularizer. When (21) is adopted, the solution of (32) is:

[0063] Figure 3 Shows and The numerical calculation curves of , and its neighboring operators are summarized in Table 1. Figure 3 The IR of the three hybrid loss functions has λ=0.5, γ=λ and And when y>0, the three IRs are concave. Table 1: Different loss functions and neighbor operators used for the sparsity-boosting regularizer.

[0064] In various embodiments, parameter selection strategies may be applied to the modified functions described above. Figure 4 The results of different regulators when λ=0.5 according to various embodiments of the present invention are shown.

[0065] There are four parameters provided to l σ，λ (x), l γ，λ (x), l τ，λ (x), namely, λ, σ, γ and τ. Obviously, understanding the physical meaning of the parameters can help choose the parameters that are suitable for their values. First, the Huber function l h (x) can be rewritten through (8) as: where λ is a positive weight parameter that controls sparsity, i.e., y = 0 when |x| ≤ λ. Similarly, in (24), (28), and (32) and The preceding λ also controls sparsity. That is, y = 0 for |x| ≤ λ. However, unlike the Huber function with an explicit λ|y|, and The expression for is usually unknown. However, they all have smaller deviations than λ|y|, which can be expressed as Figure 4 As shown, and its theoretical analysis is shown in Proposition 2.

[0066] Furthermore, the proximal operator of λ|y| in (10) treats all values ​​of x equally and scales x down by the same bias λ as shown in (10). In fact, for many real-world applications, the bias should not be the same for different x. For example, when solving low-rank matrix recovery, the larger singular values ​​of the observation matrix will correspond to the dominant information, so it is better to slightly reduce their values. Mathematically, when x>λ, the bias can be described by the difference between y=x and the proximal operators of different regularizers: Δd=x-(xa·g′(x))=a·g′(x) (35) And it is required that Δd does not increase with the increase of x, which means g″(x)≤0, that is, g(x) is concave when x>λ. For the mixed Welsch function, when x>λ, Thus we get Since x>λ, then:

[0067] Similarly, for l γ，λ (x), when x>λ, It can be obtained that γ≤x. Therefore, for l γ，x (x) is: γ≤λ (37)

[0068] Finally, for the mixed GMC function, we have

[0069] The above can be summarized as the following proposition.

[0070] Proposition 2. When applying the following problem, if γ≤λ and and and The deviation ratio Smaller:

[0071] Next, we discuss the applications of low-rank matrix recovery, including the mathematical foundations, algorithms and complexity of low-rank plus sparse matrix decomposition.

[0072] Regarding the mathematical preparation, the theory derived above applies to the scalar case. When it is extended to vectors, matrices, and singular values ​​of matrices, corresponding definitions need to be given. In addition, the proofs of the following propositions can be found in Appendix D, E, and F, respectively.

[0073] Definition 3: Let Defined as: in Can be and And σ, γ and τ are positive constants.

[0074] Proposition 3: Given the vector operator defined in (40), its neighboring operator Defined as:

[0075] The solution is:

[0076] in is an element-by-element operator. That is, In the present invention, Can be and In addition, when γ≤λ and When , we can get:

[0077] Definition 4: Let Defined as: in Can be and σ, γ and τ are all positive constants.

[0078] Proposition 4: Given the matrix operator defined in (44), its neighbor operator Defined as: The solution is: in is an element-by-element operator, Can be as well as In addition, when γ≤λ and When , we can get:

[0079] Definition 5. Let X = U Diag(s)V T is a matrix of rank r SVD, where s = [s1, s2, ..., s γ ] T are the singular values ​​of the vector. Norm ||X||* Defined as: This is the l1-norm of s.

[0080] However, using the nuclear norm to achieve low-rank recovery will lead to biased solutions because the nuclear norm is based on the l1-norm of singular values. To solve this problem, the non-convex regularizer of the present invention is used to replace the l1-norm. Compared with other non-convex regularizers, our regularizer has a closed-form neighbor operator.

[0081] Definition 6: Let x = U Diag(s)V T is a matrix of rank r SVD, where s = [s1, s2, ..., s γ ] T is a vector of singular values. The norm is expressed as Defined as:

[0082] Lemma 2: Let X = u Diag(s)V T is a matrix of rank r SVD of s=[s1,s2,...,s γ ] T is the vector of singular values, and defines:

[0083] If the neighboring operator is monotonic, then the solution of (50) is: M=U Diag(s * )V T where s * satisfy And i = 1, 2, ..., r, which is determined as:

[0084] Proposition 5: For the solution of (50), when γ≤λ and λ=0.5 and When , it is:

[0085] Regarding the algorithm of low-rank plus sparse matrix decomposition, the regularizer of the present invention decomposes the target matrix into the sum of a low-rank matrix and a sparse matrix and applies it to RPCA, which can be listed as a formula: in and is the regularization term included in Table II. Table 2: Selection of different sparsity regularizers.

[0086] Problem (52) can be solved using ADMM, and its augmented Lagrangian is: is equivalent to: Where Λ contains the Lagrange multiplier, and the last term is the augmented term, and ρ>0 is the augmented Lagrangian parameter. ADMM updates the primal and dual variables in the following way at the (k+1)th iteration: Λ k+1 =Λ k +ρ k (XM k+1 -E k+1 ) (55)

[0087] The exact expression at the (k+1)th iteration, namely {M k+1 , E k+1 , Λ k+1}, which is derived as follows.

[0088] Update of f: E can be updated by: The constant term is ignored because it does not affect E k+1 The solution. Calling (21) will yield:

[0089] Update of M: Given E k+1 and Λ k , the low-rank matrix M can be updated by:

[0090] According to Lemma 2, the solution is:

[0091] Figure 5Algorithm 1 is shown, which describes the aforementioned optimization steps according to various embodiments of the present invention. The aforementioned optimization steps are described in Algorithm 1, and its convergence is analyzed in Theorem 2. In addition, it is worth mentioning that Algorithm 1 gives a framework for generating low-rank and sparse components obtained by different sparsity-promoting regularizers in Table 2. Although, as shown in Table 2, different combinations of and Nine algorithms are available, but we consider and This will produce three different algorithms for the same loss function. and When generated by a mixed Welsch function, that is, and The resulting algorithm is called SIR-HW. The abbreviations of the remaining algorithms can be found in Table 2. In addition, the sparse regularizer generated by Huber and truncated quadratic functions can be used to obtain low-rank and sparse decomposition, resulting in two ADMM-based algorithms, called SIR-HU and SIR-HT.

[0092] Theorem 2: The {M k , E k Satisfies the following properties: 1). 2). 3).

[0093] make is the {M k , E k , Λ k}, so that In this way, then, {M * , E * , Λ *} is the critical point.

[0094] Regarding complexity, similar to PCP and WNNM-RPCA, the proposed algorithm involves computing SVD at each iteration when finding a low-rank matrix, and its complexity is Where m and n are the row and column lengths of the degenerate matrix respectively. In addition, the complexity of calculating the sparse matrix is Therefore, solving for the low-rank components dominates the computation time of each iteration.

[0095] To validate the image restoration processor of the present invention, extensive experiments were conducted on synthetic and real data. In the experimental results, SIR-HW, SIR-HC and SIR-HG, as well as SIR-HU and SIR-HT were evaluated. The recommended parameter settings are γ=λ and In addition, three benchmark techniques, namely PCP, WNNM-RPCA and DPRPCA, are also adopted. The experimental results include two parts, (1) synthetic data and (2) real-world image restoration.

[0096] For synthetic data, the data matrix From the low-rank matrix M = UV T and the sparse matrix E, where and r is the rank. For convenience, n=m is set. The elements of matrices U and V satisfy the standard Gaussian distribution, while for the location of the sparse outlier E, it has S non-zero elements, which are uniformly distributed in the interval [500, 500] and independently drawn from the Bernoulli distribution. Similar to some related works, r=p r ×m and S = p s ×m 2 In the experiment, we set m=400 and p r and p s It varies between 0.01 and 0.05 with a step size of 0.02. In addition, two evaluation indicators are used, namely the low-rank matrix The relative reconstruction error is is the estimated low-rank matrix, is the estimated ranking. All competing methods, including PCP and WNNM-RPCA, set the same parameters and stopping conditions. That is, and μ = 1.05, and ||XM k -E k || F / ||X|| F <10 -7 and the maximum number of iterations 1000 were used as the termination conditions for all algorithms. In addition, the performance of all methods was evaluated using the average results of 100 independent runs.

[0097] Figure 6 Depicts different levels and outlier ratio settings {p r , p s}, where parts (a), (b), (c), (d), (e), (f), (g), and (h) correspond to PCP, WNNM-RPCA, DPRPCA (p = 0.6), SIR-Hc, SIR-HU, SIR-HW, SIR-HG, and SIR-HT. r , p s The logarithmic scale REE of various settings of} can be Figure 6 As shown. It can be seen that PCP and SIR-HU have the same phase transition graph because the regularizer generated by the Huber function is l1-norm, and PCP also adopts l1-norm to achieve low rank and sparse decomposition. In addition, compared with PCP, WNNM-RPCA and DPRPCA (p = 0.6), SIR-HW, SIR-HC and SIR-HG can recover more cases, and among all the techniques, SIR-HG has the largest success area (if REE < 10 -4 , it indicates that the recovery is successful). Table III lists the average estimated rankings of different algorithms. It is observed that for different rank parameters, SIR-HW and SIR-HG give more accurate rank estimates than the other methods. Table 3: Average estimated rankings of different methods, where p s =0.15.

[0098] in addition, Figure 7 Convergence curves for different techniques are shown. Figure 7 Figures (a)-(d) of the graph plot different p r and p s The logarithmic scale RRE of the value is the relationship between iterations. It is easy to see that when p r or p s When p increased, the performance of PCP, WNNM-RPCA, DPRPCA (p = 0.6), SIR-HU, and SIR-HT deteriorated, while SIR-HW, SIR-HC, and SIR-HG remained the same even when p r = 0.3 and p s = 0.3, it can also run normally. Due to the discontinuity of the neighboring operator in (11), the SIR-HT curve oscillates. In addition, to verify the theoretical results of Theorem 2, we first define M at the kth iteration k , E k and XM k -E k The relative error is:

[0099] Figure 7 Parts (e)-(g) show the r= 0.1 and p s =0.1, Convergence curve with the number of iterations. It is observed that after a sufficient number of iterations, the relative error is close to zero, which verifies Theorem 2. In addition, the influence of λ on the proposed algorithm is studied in Appendix H.

[0100] Regarding hyperspectral image restoration, Figure 8 Depicts the restored image of the Washington DC shopping mall, with band 126-73-12 in RGB, where: (a) is a degraded image corrupted by 10 dB impulse noise; (b), (c), (d), (e) correspond to PCP, WNNM-RPCA, DPRPCA (p = 0.9), DPRPCA (p = 0.6); (f) is the original noise-free part; (g), (h), (i), (j) correspond to SIR-HC, SIR-HW, SIR-HG, SIR-HT. These images are the restoration results of different algorithms, and the demarcated area is enlarged 4 times. Fig. 9 Depicted are the restored images of the HYDICE city data, where band 618836 is RGB, where: (a) part is the degraded image, and the remaining images (b) (j) are the restoration results of different algorithms, and the demarcated area is magnified 4 times.

[0101] Hyperspectral imaging has many applications, such as environmental monitoring, mineral exploration, and urban planning. However, they may be contaminated by noise during the acquisition process. In order to restore hyperspectral images (HSI), low-rank modeling is very useful because there is a strong correlation along the spectral direction. The above characteristics are applicable to the restoration of hyperspectral images. Three quantitative indicators can be used, namely peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and root mean square error (RMSE). The PSNR, SSIM, and RMSE calculation formulas for HSI data are: in and Represents the true value and estimation matrix of HSI, M j and They are M and In addition, the higher the PSNR and SSIM, the smaller the RMSE, and the better the restoration quality.

[0102] The sub-image of the Washington DC Mall contains 191 spectral bands, each spectral band is 256*256 pixels. The grayscale value of each band is normalized to [0, 1], and then each band is vectorized into a vector, which is then stacked to construct a pure matrix. The impulse noise is generated by the built-in command “imnoise(I, ‘salt&pepper’, ρ)’ in MATLAB, where I is the original matrix and ρ is the normalized noise intensity. The relationship between ρ and signal-to-noise ratio (SNR) is ρ = 1 / SNR. In addition, in order to compare different algorithms fairly, the optimal weight parameter λ for each method is selected based on the lowest RRE. Impulse noise with different ρ is added to M, and the average denoising results after 10 independent runs are listed in Table IV. It is observed that SIR-HC, SIR-HW and SIR-HG all achieve good restoration results, among which SIR-HG is the best method overall because it achieves better restoration performance in all cases. Although DPRPCA (p = 0.6) has a smaller SSIM in one of the cases, the choice of its ρ is crucial because this is the method using l p -norm to achieve sparsity. To provide a visual comparison, three HSI bands were selected to form Figure 8 Pseudo-color image shown. It can be seen that SIR-HG achieves the best denoising effect in terms of PSNR value compared with competing methods.

[0103] In addition, all methods are evaluated using the real HYDICE urban dataset. Similar to related work, bands 104-108, 139-151, and 207-210 are removed due to severe noise pollution, so a sub-data with dimension 256×256×188 is used. Fig. 9 The image restored by setting band 6-188-36 to RGB is shown. It can be seen that SIR-HW, SIR-HG, and SIR-HT outperform the rest of the schemes. Table 4: Average restoration results under different noise levels, where the best three results are indicated in bold.

[0104] Regarding multispectral image restoration, Fig.10 Depicted are images from two datasets, where the images in the first and second rows are from the CAVE and Harvard datasets, respectively. Fig.11 The average PSNR curves of different algorithms as a function of spectral bands are shown, which include two parts, (a) CAVE dataset and (b) Harvard dataset. Fig.12The restored image of band 23-13-4 is depicted as RGB, where part (a) is the degraded image, part (f) is the original noise-free image, and the images in the remaining parts are the restoration results of different algorithms, and the marked area is magnified 6 times; and Fig.13 The restored image of band 25-18-8 is depicted as RGB, where part (a) is the degraded image, part (f) is the original noise-free image, and the images in the remaining parts are the restoration results of different algorithms. The marked area is enlarged 8 times.

[0105] Multispectral images (MSI) are different from HSI because they have a higher spatial resolution. In this section, two datasets are used, namely CAVE and Harvard. The CAVE dataset consists of 32 different objects, each with 31 spectral bands and a size of 512×512. The Harvard dataset contains 50 MSIs of real indoor and outdoor scenes. For each set of MSI, it has 31 spectral bands and a spatial resolution of 1392×1040. Fig.10 Figures of 16 MSIs from the CAVE and Harvard datasets are shown, which are used to evaluate all methods. In addition, for the MSIs in the Harvard dataset, each band is resized to 696×520. In addition, 10dB impulse noise is added to the MSIs. Fig.10 The average results for all MSIs shown in are listed in Table V. It can be observed that although PCP requires less computation time, SIR-HW, SIR-HC, and SIR-HG achieve better recovery in terms of PSNR compared to other algorithms. Again, it can be concluded that SIR-HG generally outperforms the rest of the methods as it achieves comparable recovery results in terms of PSNR, SSIM, and RMSE for both datasets. To further compare the performance of all techniques in each frequency band, Fig.11 A plot of average PSNR versus number of spectral bands is shown.

[0106] It can be seen that SIR-HC, SIR-HW, and SIR-HG have high PSNR values ​​for most of the frequency bands in both datasets. It is important to note here that all methods perform poorly in the first few frequency bands due to the presence of blur in these frequency bands. To provide a visual comparison, Fig.12 and Fig.13 The restored feather image and Img-7 are shown respectively. Fig.12 and Fig.13 There is still obvious noise in the images restored by PCP, WNNM-RPCA, DPRPCA (p=0.9) and DPRPCA (p=0.6). Table 5: Comparison of average restoration performance in terms of PSNR, SSIM, RMSE, and running time (in seconds).

[0107] Through the above derivation and verification, in various embodiments, an image restoration processor is provided for modifying the function and restoring the damaged image.

[0108] Fig.14 A schematic diagram of an image restoration processor 100 is depicted, which may utilize a framework that can generate a sparse regularizer for image restoration according to various embodiments of the present invention. The image restoration processor 100 includes an image receiver 102, a matrix converter 104, a modifier 106, a matrix decomposer 108, a framework memory 110, a parameter selector 120, a calculation module 122, a first cache 130, a second cache 132, and an image reconstructor 140. These components may be arranged to be in electrical communication with other components, so that they can send at least one information signal to or receive at least one information signal from other components.

[0109] The image receiver 102 is configured to receive at least one degraded image to be restored. The matrix converter 104 is electrically coupled to the image receiver 102 and configured to reshape the image data of the degraded image into at least one target matrix. The degraded image or the target matrix can be used as input data for processing by the image restoration processor 100. The matrix converter 104 is electrically coupled to the matrix decomposer 108 and is also configured to send the target matrix to the matrix decomposer 108.

[0110] The modifier 106 is electrically coupled to the framework memory 110. The framework memory 110 may be configured to store a framework as described above (e.g., Proposition 1 as described above) for generating one or more sparse promotion regularizers, i.e., it is capable of generating one or more sparse promotion regularizers and deriving their theoretical properties and closed form neighboring operators. The modifier 106 may receive an M estimator from an external source. For example, a user may optionally input an M estimator to the modifier 106 for further modification. In one embodiment, the image restoration processor 100 may further include a user interface configured to receive an M estimator from an external source and send an information signal to notify the modifier 106 of the received M estimator. The modifier 106 may be configured to read the framework memory 110 and apply the framework to modify the input M estimator. As described above, the input M estimator may be modified by the modifier 106, and then the modifier 106 may output a hybrid M estimator accordingly and generate a corresponding sparse regularizer. In one embodiment, the sparse regularizer is generated by a hybrid M-estimator in modifier 106 and then output by modifier 106. In some embodiments, the input M-estimator is selected from Welsch function, Cauchy function and German-McClure function. In various embodiments, the generated sparse regularizer is non-convex and has a closed form neighbor operator.

[0111] The parameter selector 120 is electrically coupled to the modifier 106 and is configured to execute the parameter selection strategy as described above. Specifically, the parameter selector 120 may determine the relationship between the scale parameter of the sparse regularizer and the positive weighting parameter controlling the sparsity.

[0112] The matrix decomposer 108 is configured to receive a target matrix and a sparse regularizer. In various embodiments, the matrix decomposer 108 can receive the target matrix and the sparse regularizer by accessing the first cache 130 and the second cache 132. For example, the first cache 130 is configured to receive the target matrix from the matrix converter 104 and store the target matrix, and the second cache 132 is used to receive the sparse regularizer from the modifier 106 and store the sparse regularizer. When the matrix decomposer 108 is triggered to decompose, the matrix decomposer 108 can obtain the target matrix and the sparse regularizer in the following manner: access the first cache 130 and the second cache 132 respectively. After obtaining, the matrix decomposer 108 can be configured to decompose the target matrix into the sum of a low-rank matrix and a sparse matrix, and apply the sparse regularizer to the RPCA method, as described above. The computing module 122 can participate in the work to decompose the target matrix and solve the aforementioned equation (52) with the matrix decomposer 108. The computation module 122 may be configured to perform the ADMM method after the matrix decomposer 108 performs the RPCA method.

[0113] The image reconstructor 140 is electrically coupled to the matrix decomposer 108 and is configured to integrate the decomposition results from the matrix decomposer 108 using a low-rank sparse matrix to form at least one restored image from the original degraded image. In one embodiment, the image restoration processor 100 may also include a display that is electrically in communication with the image reconstructor 140 and is configured to display the restored image, and may also display the difference between the image before and after restoration.

[0114] The image restoration processor 100 can be used as a visual restoration pipeline or a digital image reconstruction framework that combines specific steps or processes to restore an image to its original visual state. In this regard, the use scenario of the image restoration processor 100 includes significantly reducing image capture resources by deliberately undersampling the target image to be captured. This allows undersampled images to be transmitted in a shorter time and consumes fewer hardware resources; for example, images transmitted wirelessly will require less spectrum bandwidth; images transmitted electronically will use less RAM / cache when transmitted on the network. Therefore, the image restoration processor 100 of the present invention improves the efficiency of network operation and is particularly suitable for networks that transmit large amounts of images, videos and / or radar data.

[0115] For example, hyperspectral imaging is a popular image acquisition method that captures a large number of two-dimensional images at different frequencies to generate more information from a scene. If the image is spectrally undersampled, image recovery according to the present invention can be performed to reconstruct the complete image. Similarly, other types of images, such as radar and MIMO images associated with antenna technology used in wireless communications, may also be damaged during transmission. In this case, the image recovery processor 100 can be effectively applied to restore these images, thereby mitigating any potential misunderstandings or misinterpretations caused by the damage. This will help ensure that image processing for radar and MIMO applications is accurate and reliable, thereby improving the overall performance and reliability of these systems. Therefore, in various embodiments, the image data can be from one or more degraded hyperspectral images or from one or more degraded multispectral images.

[0116] Fig.15 A schematic diagram of an image restoration processor 200 is depicted that utilizes a framework that can generate a sparse regularizer for image restoration according to various embodiments of the present invention. Fig.15 The architecture of the image restoration processor 200 shown is similar to Fig.14 The architecture of the illustrated image restoration processor 100 is similar, except that the image restoration processor 200 further includes an image selector 260 electrically coupled between the image reconstructor 240 and the display 250 .

[0117] exist Fig.15 In the image restoration processor 200 shown, more than one M estimator is input to the modifier 206, and therefore the modifier 206 will be able to output more than one sparse regularizer. For example, if two different M estimators are input to the modifier 206, there will be two different sparse regularizers generated from the modifier 206, both of which are modified by the same framework. In this way, through the matrix decomposer 208, the matrix can be decomposed into the sum of different groups of low-rank and sparse matrices, where different sparse regularizers can be applied to the RPCA method. Thereafter, the image reconstructor 240 can integrate the results of the decomposition and form more than one restored image. For example, if two different M estimators are input to the modifier 206, the result is two restored images formed from a degraded image using different sparse regularizers.

[0118] The image selector 260 is configured to receive the restored image from the image reconstructor 240 and determine which of the restored images is to be output based on the image evaluation metrics of all the restored images. In one embodiment, the image evaluation metrics include SNR (signal-to-noise ratio), SSIM (structural similarity index) and RMSE (root mean square error). The restored image with the best image evaluation index can be transmitted from the image selector 260 to the display 250 to display the image. In various embodiments, the display 250 in electrical communication with the image selector 260 is also configured to display which function is introduced as a sparse regularizer for outputting the restored image, so as to inform the user which sparse regularizer is finally applied to output. In various embodiments, the image restoration processor 200 may also include an image comparator 262, which is configured to compare the restored images and quantify their restoration performance, such as the above table (e.g., Tables 1 to 5), so as to inform the user of the quantitative comparison results of the restored images.

[0119] As described above, according to various embodiments of the present invention, a framework for generating sparsity-promoting regularizers and theoretical analysis is provided. The framework of the present invention can be applied to three M-estimators, namely Welsch, Cauchy and GMC, and generate IRs of these functions. This is the first time that sparse regularizers associated with these three functions are generated. In addition, the obtained regularizers can be applied to low-rank recovery, and three corresponding algorithms with convergence guarantees are proposed herein. Numerical examples based on simulation and real data show that the image restoration method of the present invention can consistently achieve excellent restoration performance. As a result, the image restoration processor of the present invention can be used to perform image restoration.

[0120] The aforementioned appendix is ​​provided below.

[0121] Appendix A:

[0122] Fig.16Depicts the performance of different regularizers The M estimator f(x) can be reformulated by semi-quadratic optimization as: in is the associated implicit regularizer (IR). The IRs of the Welsch, Cauchy, and German-McClure M estimators are shown in Table VI, but they cannot achieve sparsity. When , the solution of (61) is expressed as and is zero for |x|≤λ. That is, when the size of x is less than the threshold, the l1-norm can achieve sparsity and set y to zero. However, when the penalty When it is considered as the IR of these M estimators, sparsity cannot be achieved because the solution is zero if and only if x = 0. In addition, to make it clear that the regularizers listed in Table VI can only achieve sparsity with the l1-norm, The curve is as Fig.16 shown. Table 6: Different regularizers and their corresponding neighbor operators

[0123] Appendix B:

[0124] Proof of Proposition 1: Since f in formula (16) * (y) is convex, so we know that in equation (18) and is concave with respect to y, so m(y) is convex with respect to y. That is, (19) is a convex problem. 1) According to formula (17), it is: The second to last equation is because l g，λ (x) is an even function. Therefore, is symmetrical. 2) According to the inversion rule of the sub-gradient relationship and formula (16), we can get:

[0125] In addition, through (21) we can get:

[0126] Then, we divide it into two cases, y>0 and y=0, and discuss the solution x in (63): ＊In addition, we define q(x) = x·yf(x), which is a concave function because f(x) is a convex function. Therefore, the solution x of (63) is ＊ satisfy Then we can get y = f'(x ＊ ). 3) According to formula (63), when y>0, There is a unique solution that satisfies: y=x ＊ -a·g′(|x ＊ |)sign(x ＊ ) (65) This implies that y < x * , the reason is that g′(|x|)sign(x)>0, because g(x) is a monotonically increasing function, and by (63)( Note that Replace with because For |y|>0), we can obtain Therefore, when y>0, increases as y increases, and is non-negative, because 4) Verification For any The discussion provided is for the following three cases. First, if y1·y2=0, and assuming y1≠0 and y2≠0, we get This is because

[0127] If y1·y2>0, first assume that y1>0 and y2>0, then for any y>0, we have (65), and from (16) we know that y will change with x. ＊ Combined with And (65), we can get Furthermore, assuming that g(x) is a concave function for x>λ, g′(x) will decrease as x increases. Therefore, will decrease as y increases. That is, is a concave function. Therefore,

[0128] Then, we can get On the other hand, if y1·y2>0, y1<0 and y2<0, then (62) can be easily obtained:

[0129] Finally, if y1·y2<0, and assuming y1>0 and y2<0, then Conclusions can be drawn immediately.

[0130] Appendix C:

[0131] Proof of Proposition 2: When γ≤λ and When Δd is known to and is non-increasing. When |x|=λ, Δd takes the maximum value, that is, λ, so for the above three regularizers, it will be Δd≤λ. , Δd=λ, so the proof is complete.

[0132] Appendix D:

[0133] Proof of Proposition 3: According to Definition 3, is divisible, so we have: The solution is: therefore, in is a pointwise operator.

[0134] In addition, when γ≤λ and When , we know from Proposition 2 that: Therefore, it will be known

[0135] Appendix E:

[0136] Proof of Proposition 4: This proof is exactly the same as that of Proposition 3, because Definition 4 shows that is divisible.

[0137] In addition, when γ≤λ and Then, through Proposition 2, we can know that: Therefore, there will be

[0138] Appendix F:

[0139] Proof of Proposition 5: Let x = U Diag(s)V T is a matrix of rank r and SVD, where s = [s1, s2, ..., s γ ] T is a singular value vector, and Then we have: The last inequality is due to Proposition 3.

[0140] Therefore, we can get

[0141] Appendix G:

[0142] Before the proof, the definition of critical point and the following proposition are stated first.

[0143] Definition 7. Assume that f is a function of M, E and Λ. If M, E and Λ satisfy: Then, {M, E, A} will be the critical points of f.

[0144] Proposition 6: Definition and Then when y>0, h1(σ,λ), h2(γ,λ) and h3(τ,λ) increase with the increase of λ and its corresponding parameter σ, and are σ, γ and τ respectively. prove:

[0145] According to (17), we can have Through (21) or Figure Figure 2 In part (b), when y>0, we know that x>λ, so we only need to verify that when x ＊ >λ, we can check whether h(σ,λ) increases with λ and σ. and Therefore, h1(σ, λ) increases with the increase of λ and σ. Similarly, it can be verified whether h2(γ, λ) and h3(τ, λ) increase with the increase of λ and its corresponding parameters γ and τ respectively.

[0146] Proof of Theorem 2:

[0147] Let U k Diag(s k )(V k ) T For the moment in is a singular value vector, r is rank, and r << min(m, n)). Thus, {Λ k} is bounded because: where a is attributed to Proposition 3.

[0148] According to the update equations of E k+1 and Λ k we have: and

[0149] Thus, it will be obtained that: where the last inequality is initiated by Proposition 4.

[0150] Similarly, according to the update of M k+1 and Λ k we get: and

[0151] Therefore, it will be:

[0152] It is easy to obtain:

[0153] However, the above Frobenius norm condition does not guarantee the boundedness of M k and E k Next, its boundedness will be established by the boundedness of in each iteration. Since the updates of E k and M k are the minimizations of their corresponding optimization problems, we have:

[0154] After updating Λ^(k + 1) and ρ^(k + 1), there is also: where b and c are obtained from Proposition 6 and (66) respectively, and Q is a constant with respect to μ. Combining equations (67) and (68), we can have: Thus, it can also be obtained that:

[0155] Given M 0 , E 0 and Λ 0 is bounded because From the above we can know is bounded. Therefore, and It can be seen from the above (67) that it is bounded, which means that each iteration E k+1 or M k+1 The update of is bounded. This is because if or but or

[0156] According to the Bolzano-Weierstrass theorem, {M k , E k , Λ k} will show that it is possible to k , E k , Λ k}There is at least one accumulation point {M * , E * , Λ *}. That is, there exists a subsequence So that:

[0157] Since the proximal operator is a closed solution of the Moreau envelope of the sparse-boosting regularizer, namely, E k+1 and M k+1 are the minimum values ​​of (56) and (58) respectively. Although the regularization term is non-convex, we will have: also, Therefore, combining equation (69) and equation (70), we can obtain: Therefore, any accumulation point {M * , E * , Λ *} is a critical point.

[0158] Appendix H:

[0159] Fig.17 The corresponding results are plotted in logarithmic scale RRE vs. c value, where (a)-(c), (d)-(f), and (g)-(i) show the effects of matrix rank, outlier ratio, and matrix dimension, respectively. Specifically, Fig.17 The relationship between RRE and c-value in logarithmic scale is plotted, where parts (a)-(b) plot the curves of different methods with different ranks only, and p s = 0.1 and m = n = 400 are kept fixed; parts (e)-(g) show different outlier ratios p s And p r = 0.05 and m = n = 400; (g)-(i) plots curves at different sizes, and p r = 0.05 and p s =0.1.

[0160] Inspired by PCP, the parameter λ is chosen as Right now where c is a constant. The value of c is studied in three cases, i.e. different matrix ranks, proportions of outliers, and matrix dimensions.

[0161] It can be seen that the suitable range of c for our algorithm is larger than that for PCP, and the rank and dimension of the matrix have little effect on the choice of c for our algorithm compared to PCP. In addition, the value range of all techniques decreases with the increase of the proportion of outliers, so for convenience, it is set in the synthetic data because although λ is not the current optimal value setting, all methods including PCP have comparable recovery performance.

[0162] The functional units and modules of the image recovery processor and method according to the embodiments disclosed herein can be implemented by hardware or software. That is, the image recovery processor claimed for protection can be fully implemented as a combination of machine instructions or machine instructions and hardware components. Hardware components include, but are not limited to, computing devices, computer processors, or electronic circuits, including, but not limited to, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), microcontrollers, and other programmable logic devices configured or programmed according to the teachings of the present disclosure. Based on the teachings of the present disclosure, a person skilled in the art of software or electronics can easily prepare computer instructions or software codes that run in a computing device, a computer processor, or a programmable logic device.

[0163] The image recovery processor may include a computer storage medium, a transient and non-transient memory device having computer instructions or software codes stored therein, which may be used to program or configure a computing device, a computer processor or an electronic circuit to perform any of the methods of the present invention. The storage medium, transient and non-transient memory devices may include, but are not limited to, floppy disks, optical disks, Blu-ray disks, DVDs, CD-ROMs and magneto-optical disks, ROMs, RAMs, flash memory devices, or any type of medium or device suitable for storing instructions, codes and / or data.

[0164] The image recovery processor can also be configured as a distributed computing environment and / or a cloud computing environment, in which all or part of the machine instructions are executed in a distributed manner by one or more processing devices interconnected by a communication network, such as: an intranet, a wide area network (WAN), a local area network (LAN), the Internet and other forms of data transmission media.

[0165] The foregoing description of the present invention is provided for the purpose of illustration and description. It is not intended to be exhaustive or to limit the present invention to the precise form disclosed. Many modifications and variations will be apparent to those skilled in the art.

[0166] The embodiment was chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to understand the invention for various embodiments and with various modifications as are suited to the particular use contemplated.

Claims

1. An image restoration processor, characterized in that: include: an image receiver configured to receive one or more degraded images; a matrix converter configured to reshape the image data of the one or more degraded images into one or more target matrices; a framework memory configured to store a framework for generating one or more sparsity-promoting regularizers to enable generating the one or more sparsity-promoting regularizers and deriving theoretical properties and closed-form neighbor operators thereof; A modifier configured to read the framework memory, apply the framework to modify the M-estimator, and accordingly output a hybrid M-estimator and a sparse regularizer corresponding to the hybrid M-estimator; a matrix decomposer configured to receive the one or more target matrices and the sparse regularizer, and apply the sparse regularizer to a robust principal component analysis (RPCA) method by decomposing one of the one or more target matrices into a sum of a low-rank and a sparse matrix; and An image reconstructor integrates the decomposition results from the matrix decomposer to form one or more restored images from the one or more degraded images using the low-rank and sparse matrix.

2. The image restoration processor according to claim 1, characterized in that Also includes: a first cache configured to receive the one or more target matrices from the matrix converter and store the one or more target matrices, wherein the matrix decomposer receives the one or more target matrices by accessing the first cache; and A second cache is configured to receive the sparse regularizer and store the sparse regularizer, wherein the matrix decomposer receives the sparse regularizer by accessing the second cache.

3. The image restoration processor according to claim 1, characterized in that Also includes: A parameter selector is configured to determine a relationship between a scale parameter of the sparse regularizer and a positive weight parameter for controlling sparsity.

4. The image restoration processor according to claim 1, wherein: The image data is from one or more degraded hyperspectral images.

5. The image restoration processor according to claim 1, characterized in that: The image data is from one or more degraded multispectral images.

6. The image restoration processor according to claim 1, characterized in that The M-estimator is selected from the group consisting of a Welsch function, a Cauchy function, and a German-McClure function.

7. The image restoration processor according to claim 1, characterized in that: Also includes: A user interface is configured to receive the M-estimator from an external source and send an information signal to inform the modifier of the received M-estimator.

8. The image restoration processor according to claim 1, wherein: Also includes: A computing module is configured to perform an alternating direction method of multipliers (ADMM) method after the matrix decomposer performs the RPCA method.

9. The image restoration processor according to claim 1, characterized in that: The sparse regularizer is non-convex and has a closed form neighbor operator.

10. The image restoration processor according to claim 1, characterized in that Also includes: A display is in electrical communication with the image reconstructor and is configured to display the restored one or more images.

11. An image restoration processor, characterized in that: include: an image receiver configured to receive a degraded image; a matrix converter configured to reshape the image data of the degraded image into a target matrix; a framework memory configured to store a framework for generating one or more sparsity-promoting regularizers, enabling generation of the one or more sparsity-promoting regularizers and derivation of theoretical properties and closed-form neighbor operators thereof; A modifier configured to read the framework memory and apply the framework to modify more than one M-estimator, and accordingly output more than one hybrid M-estimator and more than one sparse regularizer corresponding to the more than one hybrid M-estimator; A matrix decomposer configured to receive the target matrix and different sparse regularizers, and to apply different sparse regularizers to a robust principal component analysis (RPCA) method by decomposing the matrix into a sum of low-rank and sparse matrices; an image reconstructor that integrates the decomposition results from the matrix decomposer to form more than one restored image from the degraded image using the low-rank and sparse matrix using different of the sparse regularizers; and An image selector is configured to receive the restored images from the image reconstructor and determine which one of the restored images to output based on image evaluation metrics of all the restored images.

12. The image restoration processor according to claim 11, characterized in that Also includes: a first cache configured to receive the target matrix from the matrix converter and store the target matrix, wherein the matrix decomposer receives the target matrix by accessing the first cache; and A second cache is configured to receive the sparse regularizer and store the sparse regularizer, wherein the matrix decomposer receives the sparse regularizer by accessing the second cache.

13. The image restoration processor according to claim 11, characterized in that: Also includes: A parameter selector is configured to determine a relationship between a scale parameter of the sparse regularizer and a positive weighting parameter for controlling sparsity.

14. The image restoration processor according to claim 11, wherein: The image data is from a degraded hyperspectral image.

15. The image restoration processor according to claim 11, characterized in that The image data is from a degraded multispectral image.

16. The image restoration processor according to claim 11, characterized in that The image evaluation metrics include SNR (Signal to Noise Ratio), SSIM (Structural Similarity Index) and RMSE (Root Mean Square Error).

17. The image restoration processor according to claim 11, characterized in that: The M-estimator is selected from the group consisting of a Welsch function, a Cauchy function, and a German-McClure function.

18. The image restoration processor according to claim 17, wherein: Also includes: A display is in electrical communication with the image selector and is configured to display which function is introduced as the sparse regularizer of the output restored image.

19. The image restoration processor according to claim 11, characterized in that: Also includes: A display is in electrical communication with the image reconstructor and is configured to display the restored image from the degraded image.

20. The image restoration processor according to claim 11, characterized in that Also includes: An image comparator is configured to compare the restored images and quantify restoration performance of the restored images.

Images