A multi-view image representation method based on low-rank correlation analysis

Abstract

The present invention discloses a multi-view image representation method based on low-rank association analysis, which explores the association structure of multi-view image variables through typical association analysis; obtains the common part of the multi-view image and its projection vector in the sample space through factor decomposition; and reduces the dimension and noise of the data through low-rank representation, thereby suppressing the influence of outliers. The advantage of the present invention is that when the noise density is high, the association information in the multi-view image is extracted through matrix factor decomposition, and according to the extracted common part, a better low-rank structure is obtained through low-rank alignment, thereby achieving a better noise reduction effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular to a multi-view image representation method based on low-rank correlation analysis. Background Art

[0002] With the continuous development of information acquisition technology, data in many fields such as machine learning and computer vision not only have the characteristics of high latitude and low value density, but also have multi-view characteristics according to different perspectives. For multi-view data, its reasonable data representation has always been one of the key points and difficulties of machine learning, and its learning effect is often affected by the data representation method; in real life, multi-view data often has noise or missing data. This situation brings great inconvenience to the analysis and processing of image information. In order to effectively deal with this problem, low-rank models have been proposed and widely used in the representation of graphics, appearance or motion in computer vision problems, subspace segmentation, subspace clustering and image classification and other practical applications.

[0003] In various practical applications, low-rank representation has good robustness to noisy or missing data, which makes low-rank representation a very competitive technology. Low-rank representation can obtain the representation of the underlying data structure in the case of multiple subspaces by applying a dictionary that linearly spans the data space. Therefore, many computer vision tasks can be represented as low-dimensional linear models. The low-rank representation methods involved in the present invention are: low-rank matrix factorization (LRMF).

[0004] However, although the influence of noise and outliers can be suppressed to a large extent by only using low-rank models for multi-view data, the correlation information between multi-view data is not well preserved. In recent years, methods such as canonical association analysis and principal component analysis have been proposed one after another. They measure the closeness of the relationship between samples by calculating covariance or correlation coefficients and then find the similarities and differences. There are also some limitations: when the observation data is insufficient or there is severe noise, the clustering or classification accuracy of the samples will drop sharply. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention proposes a multi-view image representation method based on low-rank correlation analysis, which obtains the lowest rank representation of the common part of the multi-view images through low-rank constraints, thereby achieving a better denoising effect.

[0006] The technical solution adopted by the present invention is as follows:

[0007] Step 1: Collect a multi-view data set {X, Y}, where X and Y are the data matrices of the data sets corresponding to the two views, and initialize the number of loops;

[0008] Step 2: Decompose the data matrices X and Y into X=UP, Y=VQ by factor decomposition; where U and V are the unique feature parts of the data matrices X and Y obtained by factor decomposition, and P and Q are the common parts of the data matrices X and Y obtained by factor decomposition. Apply low-rank constraints to the common parts P and Q to obtain and Get about and The objective function of

[0009] Step 3: Use the Lagrangian function to process the objective function and update and solve each variable;

[0010] Step 4: Set the end judgment condition of the iterative update. If the above judgment conditions are met, the loop is jumped out and the optimal solutions P and Q are output; otherwise, the loop is continued;

[0011] Step 5: Based on the obtained optimal low-rank matrices P and Q, a low-dimensional representation of the denoised multi-view association data is finally obtained.

[0012] Furthermore, in step 2, the common parts P and Q are constrained to have a low rank by using the nuclear norm to obtain and Then the objective function is obtained, which is expressed as:

[0013]

[0014] in, and They are the common parts P and Q obtained by applying low-rank constraints, E is the residual term, λ=0.1 is the loss function and L 2，1 The equilibrium parameter of the norm, ||·|| * represents the nuclear norm, ||·|| 2，1 Indicates L 2，1 Norm.

[0015] Furthermore, in step 3, the objective function is Lagrange transformed to obtain:

[0016]

[0017] Among them, L is the Lagrangian function, E is the residual term, λ=0.1 is the loss function and L 2，1 The equilibrium parameter of the norm, and are the common parts P and Q to which the low-rank constraint is imposed, μ is the penalty parameter μ>0, ||·|| * represents the nuclear norm, ||·|| F represents the F-norm, tr(*) represents the trace of the matrix; M1, M2, M3, M4 and M5 are the Lagrange multipliers respectively.

[0018] Furthermore, based on the Lagrangian function, the process of updating and solving each variable is as follows:

[0019] Step 1: According to the Lagrangian function, It can be obtained by optimizing and solving the following formula:

[0020]

[0021]

[0022] Step 2: By taking the partial derivatives of the Lagrangian function P and Q and setting them to 0, we can get:

[0023]

[0024]

[0025] Step 3: By taking the partial derivatives of the Lagrangian function U and V and setting them to 0, we can calculate:

[0026]

[0027]

[0028] Step 4: Based on P and Q calculated in the above steps, the residual term E can be obtained by optimizing the following formula:

[0029]

[0030] Where I represents the identity matrix, U T 、V T are the transpose of U and V respectively.

[0031] Furthermore, the end determination condition is: the overall error of the current iteration is less than a threshold or the number of iterations is greater than Maxiter times, where Maxiter is the maximum number of iterations.

[0032] Further, the overall error is expressed as: stopC=sqrt(sum(diag(leq′*leq))); wherein leq is the error term, expressed as leq=PQE, leq′ is the transposed representation of leq, diag(*) represents extracting the diagonal elements of the matrix, sum(*) is the summation function, and sqrt(*) is the square root function of a non-negative real number.

[0033] Beneficial effects of the present invention:

[0034] 1. The present invention measures the correlation between multi-view data through typical correlation analysis, and extracts the correlation information between multi-view data through matrix factorization. Matrix factorization can extract the rich information behind each view and provide better and deeper feature representation.

[0035] 2. By imposing low-rank constraints on the common parts of the factorized multi-view dataset, the influence of noise on the correlation information of multi-view data can be further suppressed, thereby obtaining a better low-rank structure.

[0036] 3. Use matrix factor decomposition method to reduce the difficulty of calculation, thereby increasing the calculation speed and reducing the time complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the method of the present invention;

[0038] Figure 2 The dataset images are under different levels of noise;

[0039] Figure 3 It is a line graph of the accuracy of the method of the present invention and other comparative methods. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] like Figure 1 A multi-view image representation method based on low-rank correlation analysis is shown, comprising the following steps:

[0042] Step 1: Collect a multi-view dataset {X, Y} = {(x1, y1), (x2, y2), ..., (x n ,y n )} as the required data set, X, Y are the data matrices of the data sets corresponding to the two views, x i ,y i are the i-th samples in datasets X and Y, respectively, x i ,y i ∈R dxn , d and n represent the dimension and number of images respectively. And initialize the number of loops iter = 0.

[0043] Step 2: Decompose the data matrices X and Y into X=UP, Y=VQ by factor decomposition, where U and V are the unique feature parts of the data matrices X and Y obtained by factor decomposition, and P and Q are the common parts of the data matrices X and Y obtained by factor decomposition. At the same time, a low-rank constraint is imposed on the common parts P and Q to obtain and The specific process is:

[0044] For the common parts P and Q, the low rank of the matrix is ​​constrained by the nuclear norm to obtain and The objective function is expressed as:

[0045]

[0046] in, and They are the common parts P and Q obtained by applying low-rank constraints, E is the residual term, λ=0.1 is the loss function and L 2，1 The equilibrium parameter of the norm, ||·|| * represents the nuclear norm, ||·|| 2，1 Indicates L 2，1 Norm.

[0047] Step 3: Use the Lagrangian function to process the objective function, which can be expressed as:

[0048]

[0049] Among them, L is the Lagrangian function, E is the residual term, λ=0.1 is the loss function and L 2，1 The equilibrium parameter of the norm, and are the common parts P and Q to which the low-rank constraint is imposed, μ is the penalty parameter μ>0, ||·|| * represents the nuclear norm, ||·|| F represents the F-norm, tr(*) represents the trace of the matrix; M1, M2, M3, M4 and M5 are the Lagrange multipliers respectively.

[0050] Based on the Lagrangian function, each variable is updated and solved. The specific method is as follows:

[0051] Step 1: According to the Lagrangian function, It can be obtained by optimizing and solving the following formula:

[0052]

[0053]

[0054] Step 2: By taking the partial derivatives of the Lagrangian function P and Q and setting them to 0, we can get:

[0055]

[0056]

[0057] Where I represents the identity matrix, U T 、V T are the transpose of U and V respectively.

[0058] Step 3: By taking the partial derivatives of the Lagrangian function U and V and setting them to 0, we can calculate:

[0059]

[0060]

[0061] Step 4: Based on P and Q calculated in the above steps, the residual term E can be obtained by optimizing the following formula:

[0062]

[0063] Step 4, set the end judgment condition of the iterative update. If the above judgment conditions are met, then jump out of the loop; otherwise, continue to execute the loop, that is, the process of updating and solving each variable in the iterative step 2. When the loop exits, it means that the optimal solution P and Q have been found. The end judgment condition is: until the overall error is less than the threshold or the number of iterations is greater than Maxiter times, Maxiter = 200 is the maximum number of iterations. Calculate the overall error: stopC = sqrt(sum(diag(leq′*leq)))

[0064] Among them, leq=PQE is the error term, leq′ is the transposed representation of leq, diag(*) represents extracting the diagonal elements of the matrix, sum(*) is the summation function, and sqrt(*) is the square root function of non-negative real numbers.

[0065] Step 5: Based on the obtained optimal low-rank matrices P and Q, a low-dimensional representation of the denoised multi-view association data is finally obtained.

[0066] In this implementation example, the CMU face dataset is selected as the multi-view dataset {X, Y}. The dataset contains different views of 68 people in 13 different postures, 42 lighting conditions, and 4 expressions, where the dataset {X, Y} is processed by adding noise. Figure 2 shown.

[0067] The present invention measures the correlation between multi-view data through typical correlation analysis, and extracts the correlation information between multi-view data through matrix factorization; at the same time, when the noise density is high, the influence of noise on the correlation information of multi-view data can be well suppressed by applying low-rank constraints, thereby obtaining a lower-rank matrix to achieve better noise reduction effect. In order to more clearly illustrate the effect of the present invention, as shown in Figure 2 Different degrees of noise are applied to the multi-view dataset images, and the multi-view dataset images with different degrees of noise are input into the scheme of the present invention, non-convex matrix low rank decomposition (LRSC), multi-view non-negative matrix decomposition (MultiNMF), robust multi-view dual low rank decomposition (RMSL), and multi-view low rank sparse decomposition (MLRSSC) for processing, and the clustering accuracy of the images is calculated respectively. Figure 3 It can be seen that under different degrees of noise, the clustering accuracy processed by the method proposed in the present invention is always higher than that of other comparison algorithms.

[0068] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A multi-view image representation method based on low-rank correlation analysis, characterized in that: The following steps are involved: Step 1: Collect a multi-view data set {X, Y}, where X and Y are the data matrices of the data sets corresponding to the two views, and initialize the number of loops; Step 2: Decompose the data matrices X and Y into X=UP, Y=VQ by factor decomposition; where U and V are the unique feature parts of the data matrices X and Y obtained by factor decomposition, and P and Q are the common parts of the data matrices X and Y obtained by factor decomposition. Apply low-rank constraints to the common parts P and Q to obtain and Get about and The objective function of Step 3: Use the Lagrangian function to process the objective function and update and solve each variable; Step 4: Set the end judgment condition of the iterative update. If the above judgment conditions are met, the loop is jumped out and the optimal solutions P and Q are output; otherwise, the loop is continued; Step 5: finally obtaining a low-dimensional representation of the denoised multi-view correlation data based on the obtained optimal low-rank matrices P and Q; In step 2, the common parts P and Q are constrained by the nuclear norm to obtain the low rank matrix and Then the objective function is obtained, which is expressed as: stP=Q+E,X=UP,Y=VQ, in, and They are the common parts P and Q obtained by applying low-rank constraints, E is the residual term, λ=0.1 is the loss function and L 2,1 The equilibrium parameter of the norm, ||·|| * represents the nuclear norm, ||·|| 2,1 Indicates L 2,1 norm; In step 3, the objective function is Lagrange transformed to obtain: Among them, L is the Lagrangian function, E is the residual term, λ=0.1 is the loss function and L 21 The equilibrium parameter of the norm, and are the common parts P and Q to which the low-rank constraint is imposed, μ is the penalty parameter μ>0, ||·|| * represents the nuclear norm, ||·|| F represents the F-norm, tr(*) represents the trace of the matrix; M1, M2, M3, M4 and M5 are the Lagrange multipliers respectively; Based on the Lagrangian function, the process of updating and solving each variable is: Step 1: According to the Lagrangian function, It can be obtained by optimizing and solving the following formula: Step 2: By taking the partial derivatives of the Lagrangian function P and Q and setting them to 0, we can get: Step 3: By taking the partial derivatives of the Lagrangian function U and V and setting them to 0, we can calculate: Step 4: Based on P and Q calculated in the above steps, the residual term E can be obtained by optimizing the following formula: Where I represents the identity matrix, U T 、V T are the transpose of U and V respectively.

2. The multi-view image representation method based on low-rank correlation analysis according to claim 1, characterized in that: The end determination condition is: the overall error of the current iteration is less than the threshold or the number of iterations is greater than Maxit.

3. The multi-view image representation method based on low-rank correlation analysis according to claim 1, characterized in that: The overall error is expressed as: stopC=sqrt(sum(diag(leq'*leq))); where leq is the error term, expressed as leq=PQE, leq′ is the transposed representation of leq, diag(*) means extracting the diagonal elements of the matrix, sum(*) is the summation function, and sqrt(*) is the square root function of a non-negative real number.

Images