Unsupervised image feature dimension reduction method

Abstract

The invention belongs to the field of image feature selection, and particularly relates to an unsupervised image feature dimension reduction method. The method comprises the following steps: performing iterative optimization by taking minimization of a total loss function as a target to obtain an optimized projection matrix corresponding to a data matrix formed by sample vectors containing more than two features of each image to be subjected to feature dimension reduction; selecting a set number of row or column vectors according to the module length of each row or column vector corresponding to different features of the sample vector in the data matrix in the optimized projection matrix; the features in the sample vectors corresponding to the selected row or column vectors are used for constructing a data matrix of the image after feature dimension reduction; the total loss function comprises sparse constraints and also comprises an orthogonal basis clustering loss item: # imgabs0 #, wherein zi comprises a difference value of a product of a row or column vector xi of the ith feature of a sample vector and a projection matrix and a product of a row or column vector ui of the ith feature of a corresponding sample vector in a basis matrix of orthogonal basis clustering and a coding matrix of the orthogonal basis clustering; l () is a set logarithmic function.

Description

Technical Field

[0001] The invention belongs to the field of image feature selection, and in particular relates to an unsupervised image feature dimensionality reduction method. Background Art

[0002] In the era of rapid development of the Internet, the advancement of information technology such as storage technology has led to an increasing number of data dimensions faced when identifying or classifying images. These high-dimensional data bring a large number of redundant or noisy features, which not only significantly increase the processing time, but also have a negative impact on the clustering effect. Therefore, how to remove these useless features has become one of the research hotspots in the current field of machine learning. At present, data dimensionality reduction is a powerful tool to solve the above problems. Data dimensionality reduction technology is mainly divided into two types: feature extraction and feature selection. Feature extraction is to project the original high-dimensional features into a new low-dimensional feature space. The newly constructed feature space is usually a linear or nonlinear combination of the original features; while feature selection directly selects a subset of relevant features for model construction. When the original data does not directly contain features available to the learning algorithm, people will choose to extract features to make the data more suitable for the learning algorithm. Since feature extraction creates a set of new features, we cannot retain the physical meaning of these features, which affects further analysis. In contrast, feature selection retains the physical meaning of the original features and gives the model better readability and interpretability by retaining some of the original features.

[0003] According to the relationship with the learning algorithm, feature selection is usually divided into three methods: filtering, encapsulation and embedded. The filtering method is independent of the subsequent learning algorithm and only evaluates the features according to specific criteria, making it have a faster calculation speed; the encapsulation method directly uses the performance of the learning algorithm as the evaluation criterion of the feature subset to adapt to the needs of specific learning tasks. Since the learning algorithm needs to be called multiple times, the encapsulation method has higher computational costs and greater computing resource requirements in comparison, but its computational overhead is large and is usually used in specific applications. The embedded method embeds feature selection directly into the learning algorithm to adapt to specific tasks, inheriting the advantages of filtering and encapsulation. According to the technology used, filtering and embedded methods can be divided into four categories: similarity-based methods, information-theory-based methods, sparse learning-based methods, and statistics-based methods. Similarity-based methods evaluate feature importance by retaining the ability to retain data similarity. For supervised feature selection, data similarity can be derived from label information; for unsupervised feature selection methods, most methods use different distance metrics to obtain data similarity. Representative methods include Laplacian Score, Fisher Score, etc. Unlike similarity-based feature selection algorithms that cannot handle feature redundancy, most feature selection algorithms based on information theory can be unified in a probabilistic framework that considers both “feature relevance” and “feature redundancy”. At the same time, similar to similarity-based methods, feature selection methods based on information theory are independent of any learning algorithm and are therefore generalizable. However, most existing feature selection methods based on information theory can only work in supervised scenarios. Without the guidance of category labels, it is still unclear how to evaluate the importance of features. In addition, these methods can only process discrete data, while continuous numerical variables need to be discretized beforehand. Most statistical feature selection methods rely on predefined statistical measures to filter out unnecessary features, which are simple and direct in nature and the computational cost of these methods is often low. For this reason, they are often used as a preprocessing step before applying other mature feature selection algorithms. In addition, like similarity-based feature selection methods, statistical feature selection methods often evaluate the importance of features separately, so they cannot handle feature redundancy. At the same time, most algorithms in this category can only process discrete data and require traditional data discretization techniques to preprocess numerical and continuous variables.

[0004] The advantage of the feature selection method based on sparse learning is that feature selection is embedded in a typical learning algorithm, so it can often bring very good performance to the underlying learning algorithm. In addition, the sparsity of the feature weights makes the model well interpretable. The feature selection method based on sparse learning usually adopts a two-step strategy: first, pseudo labels are introduced to help discover the importance of each feature, where several evaluation criteria such as Laplace scores and scores calculated on a specified matrix are used to measure the importance of features; then, the top k important dimensions are selected according to the learning order of all features to achieve feature dimensionality reduction; after completing the feature dimensionality reduction, the performance of the selected features (i.e., the reduced features) is evaluated by K-means clustering. Alternatively, since the optimization target is unknown in an unsupervised scenario, the matrix corresponding to the target is usually regarded as a learning variable and incorporated into the learning framework of the feature selection method of sparse learning. However, the existing feature selection methods that incorporate the matrix corresponding to the target into sparse learning often lack effective constraints in the loss function of sparse learning or the constraints are difficult to effectively reduce outliers and noise, resulting in insufficient robustness of feature dimensionality reduction based on feature selection. Summary of the invention

[0005] The purpose of the present invention is to provide an unsupervised image feature dimensionality reduction method to solve the problem of insufficient robustness of feature dimensionality reduction based on feature selection, which occurs in the existing method of image feature dimensionality reduction through feature selection based on sparse learning, because the loss function of sparse learning lacks effective constraints or the constraints are difficult to effectively reduce outliers and noise.

[0006] In order to achieve the above-mentioned object, the present invention provides an unsupervised image feature dimensionality reduction method, which performs iterative optimization with the goal of minimizing the total loss function to obtain an optimized projection matrix; the projection matrix corresponds to a data matrix composed of sample vectors containing more than two features of each image to be feature-reduced; a set number of row or column vectors are selected according to the modulus length of each row or column vector corresponding to different features of the sample vector in the data matrix in the optimized projection matrix; the features in the sample vector corresponding to the selected row or column vector are used to construct the data matrix of the image after feature dimensionality reduction;

[0007] The total loss function contains the following orthogonal basis clustering loss terms:

[0008]

[0009] z i The row or column vector x containing the i-th feature of the sample vector i The product of the projection matrix and the row or column vector u of the i-th feature of the sample vector in the basis matrix of the orthogonal basis clustering iThe difference between the product of the encoding matrix of the orthogonal basis clustering and l() is the set logarithmic function; the constraints of the total loss function include sparse constraints.

[0010] Furthermore, the total loss function also includes an inner product loss term as a regularization term; if the projection matrix is ​​composed of row vectors corresponding to different features of the sample vectors in the data matrix, the inner product loss term includes the sum of the absolute values ​​of the inner products of each row vector in the projection matrix and other row vectors with different sequence numbers in the projection matrix; if the projection matrix is ​​composed of column vectors corresponding to different features of the sample vectors in the data matrix, the inner product loss term includes the sum of the absolute values ​​of the inner products of each column vector in the projection matrix and other column vectors with different sequence numbers in the projection matrix.

[0011] Furthermore, the constraints of the total loss function also include orthogonality constraints and non-negativity constraints on the basis matrix in orthogonal basis clustering and orthogonality constraints on the encoding matrix.

[0012] Furthermore, the set logarithmic function is:

[0013]

[0014] Wherein, l(x) is the set logarithmic function; x is a variable in the logarithmic function, and γ is a set numerical parameter.

[0015] Furthermore, the method of selecting a set number of row or column vectors according to the modulus length of each row or column vector in the optimized projection matrix that respectively corresponds to different characteristics of the sample vector in the data matrix includes selecting the set number of row or column vectors whose modulus length is larger than that of other row or column vectors in the optimized projection matrix that respectively correspond to different characteristics of the sample vector in the data matrix.

[0016] Furthermore, the sample vector containing more than two features of the image to be feature-reduced is obtained by performing pixel-based expansion on a grayscale image composed of pixel values ​​obtained after grayscale processing of the image to be feature-reduced.

[0017] Furthermore, the sparse constraint is obtained by using the projection matrix l as a regularization term in the total loss function. 2,1 The loss term implementation of the norm.

[0018] Furthermore, the method of iteratively optimizing the optimized projection matrix with the goal of minimizing the total loss function includes: taking minimizing the total loss function as the goal, taking the projection matrix, the base matrix of the orthogonal basis clustering and the encoding matrix as iterative variables, performing iterative optimization by the alternating direction multiplication method until the iterative stopping condition is met, and finally obtaining the optimized projection matrix.

[0019] Furthermore, the sum of the absolute values ​​of the inner products of each row vector in the projection matrix and each row vector in the projection matrix with a different sequence number from the row vector, and the sum of the absolute values ​​of the inner products of each column vector in the projection matrix with other column vectors in the projection matrix with different sequence numbers are obtained by calculating the difference between the l1 norm of the product of the projection matrix and itself and the square of the Frobenius norm of the projection matrix.

[0020] Furthermore, the orthogonality constraint on the basis matrix includes:

[0021] U T U=I

[0022] The orthogonality constraints on the encoding matrix include:

[0023] V T V=I

[0024] The non-negativity constraints on the basis matrix include:

[0025] U≥0

[0026] Where U is the basis matrix, V is the encoding matrix, I is the identity matrix, and U T is the transposed matrix of U, V T is the transposed matrix of V.

[0027] The beneficial effects of the technical solution of the unsupervised image feature dimensionality reduction method mentioned above in the present invention include: integrating feature selection and data clustering into a framework through a total loss function that includes an orthogonal basis clustering loss term and has a sparse constraint, so as to ensure the feature selection of unsupervised sparse learning through data clustering; on this basis, the orthogonal basis clustering result is processed by a logarithmic function to obtain an orthogonal basis clustering loss term, considering that the loss function in the form of a logarithmic function has the smallest growth rate for the increase of error compared to other forms of loss functions, and when the error approaches infinity, its derivative approaches 0, thereby being able to effectively limit the influence of noise and outliers, that is, equivalent to using a logarithmic function to perform robustness constraints on the orthogonal basis clustering loss term in the total loss function, thereby utilizing the robustness advantage of the logarithmic function to effectively reduce outliers and noise, etc., and ensuring the robustness of the total loss function, thereby effectively enhancing the robustness of feature dimensionality reduction through feature selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a principle block diagram of the unsupervised image feature dimensionality reduction method in an embodiment of the unsupervised image feature dimensionality reduction method of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0030] Unsupervised image feature dimensionality reduction method embodiment

[0031] This embodiment provides a technical solution for an unsupervised image feature dimensionality reduction method, which enhances the robustness of the model by designing a robust loss function, and combines sparse and low redundancy learning to make the selected feature subset have the characteristics of high sparsity and low redundancy.

[0032] The scheme specifically includes: performing iterative optimization with the goal of minimizing the total loss function to obtain an optimized projection matrix; the projection matrix and the data matrix X=[x1,x2,…,x n ]∈R d×n Correspondingly; selecting a set number of row or column vectors from each row or column vector according to the modulus of each row or column vector in the optimized projection matrix corresponding to different features of the sample vector in the data matrix; the features in the sample vector corresponding to the selected row or column vector are used to construct the data matrix of the image after feature dimensionality reduction.

[0033] Specifically, the sample vector containing more than two features of the image to be feature-reduced is obtained by performing pixel-based expansion on the grayscale image composed of pixel values ​​obtained after grayscale processing of the image to be feature-reduced, that is, Figure 1 The data preprocessing step in the process; since the data matrix X = [x1, x2, ..., x n ]∈R d×n Each sample vector x1,x2,…,x n The method belongs to the prior art and will not be described here.

[0034] Since in this embodiment, the sample vectors of the image to be feature reduced in the data matrix that contain more than two features are arranged in rows, then the row vectors correspond to the different features of the sample vectors in the data matrix in the projection matrix; if the sample vectors of the image to be feature reduced in the data matrix that contain more than two features are arranged in columns, then the column vectors correspond to the different features of the sample vectors in the data matrix in the projection matrix.

[0035] Among them, the total loss function contains the following orthogonal basis clustering loss terms:

[0036]

[0037] Where ‖‖2 is the l2 norm; z i The row or column vector x containing the i-th feature of the sample vector i The product of the projection matrix and the row or column vector u of the i-th feature of the corresponding sample vector in the basis matrix of the orthogonal basis clusteringi The difference between the product of the encoding matrix of the orthogonal basis clustering and l() is a set logarithmic function. In addition, the constraints of the total loss function include sparse constraints.

[0038] Therefore, the unsupervised image feature dimensionality reduction method of this embodiment integrates feature selection and data clustering into a framework through a total loss function that includes an orthogonal basis clustering loss term and has a sparsity constraint to ensure feature selection for unsupervised sparse learning through data clustering. On this basis, the orthogonal basis clustering result is processed by a logarithmic function to obtain an orthogonal basis clustering loss term. Considering that the loss function in the form of a logarithmic function has the smallest growth rate for the increase in error compared to other forms of loss functions, and when the error approaches infinity, its derivative approaches 0, it can effectively limit the influence of noise and outliers, that is, it is equivalent to using a logarithmic function to perform robustness constraints on the orthogonal basis clustering loss term in the total loss function, thereby utilizing the robustness advantage of the logarithmic function to effectively reduce outliers and noise, and ensure the robustness of the total loss function, thereby effectively enhancing the robustness of feature dimensionality reduction through feature selection.

[0039] Specifically, in this embodiment:

[0040]

[0041] In the formula, x i is the row or column vector of the i-th feature of the sample vector; W is the projection matrix; V is the encoding matrix of the orthogonal basis clustering; u i is the row or column vector of the i-th feature of the corresponding sample vector in the basis matrix U of the orthogonal basis clustering; the superscript T represents the transpose of the matrix, which is a common setting for implementing matrix multiplication.

[0042] The set logarithmic function in the orthogonal basis clustering loss term is:

[0043]

[0044] Wherein, l(x) is the set logarithmic function; x is the variable in the logarithmic function, and γ is the set numerical parameter. Therefore, the above orthogonal basis clustering loss term can be expressed as:

[0045]

[0046] In addition, the constraints corresponding to the orthogonal basis clustering loss term include sparse constraints, orthogonality constraints and non-negativity constraints on the basis matrix in orthogonal basis clustering, and orthogonality constraints on the encoding matrix. Among them, the orthogonality constraints on the basis matrix include:

[0047] U T U=I

[0048] The orthogonality constraints on the encoding matrix include:

[0049] V T V=I

[0050] The non-negativity constraints on the basis matrix include:

[0051] U≥0

[0052] Where U is the basis matrix, V is the encoding matrix, I is the identity matrix, and U T is the transposed matrix of U, V T is the transposed matrix of V. In general, the sparse constraints for orthogonal basis clustering usually include:

[0053] st‖W‖ 2,0 ≤r

[0054] Among them, st represents the constraint corresponding to the total loss function, W∈R n×c is the projection matrix (also called the transformation matrix), ‖W‖ 2,0 It's W's l 2,0 norm, which represents the number of non-zero rows of W, r is a given sparsity level, and r>0. However, with ‖W‖ 2,0 The optimization problem of sparse constraints is usually very challenging and NP-hard. Therefore, this embodiment uses the projection matrix l as a regularization term in the total loss function. 2,1 The loss term of the norm realizes the sparse constraint, which transforms the sparse optimization problem into the following unconstrained ‖W‖ 2,1 Regularized optimization problem:

[0055]

[0056] Where, ‖W‖ 2,1 That is, l represents the projection matrix as a regularization term in the total loss function 2,1 norm loss term; α is the projection matrix l as a regularization term in the total loss function 2,1 The loss term of the norm corresponds to a hyperparameter that provides a trade-off between accuracy and sparsity.

[0057] After analysis, it is found that the above regularization term α‖W‖ about W 2,1 The one used is l 2,1norm, it cannot achieve high sparsity and low redundancy at the same time. Therefore, in the unsupervised image feature dimensionality reduction method of this embodiment, the total loss function also includes an inner product loss term as a regularization term; if the projection matrix corresponds to different features of the sample vectors in the data matrix respectively, the inner product loss term includes the sum of the absolute values ​​of the inner products of each row vector in the projection matrix and other row vectors with different serial numbers in the projection matrix; if the projection matrix corresponds to different features of the sample vectors in the data matrix respectively, the inner product loss term includes the sum of the absolute values ​​of the inner products of each column vector in the projection matrix and other column vectors with different serial numbers in the projection matrix. That is, the correlation between features is evaluated by calculating the absolute value of the inner product of the vector of the projection matrix corresponding to the feature, thereby constructing a regularization term that characterizes the independence or significance of the metric variable; generally speaking, if w i and w j The absolute value of the inner product of is large, then the corresponding feature f i and f j Often have a strong correlation and may contain redundant information; on the contrary, if the absolute value of the inner product is small, it indicates that the two features are more independent; taking the sum of the absolute values ​​of the inner products of each row vector in the projection matrix and other column vectors with different serial numbers in the projection matrix as the regularization term of the total loss function, it can be ensured that when the total loss function is minimized, the optimization result has the characteristics of low redundancy. Combined with the regularization term α‖W‖ 2,1 Joint optimization can achieve both high sparsity and low redundancy optimization.

[0058] Specifically, the sum of the absolute values ​​of the inner products of each row vector in the projection matrix and each row vector with different sequence numbers in the projection matrix, and the sum of the absolute values ​​of the inner products of each column vector in the projection matrix with other column vectors with different sequence numbers in the projection matrix can be obtained by calculating the difference between the l1 norm of the product of the projection matrix and itself and the square of the Frobenius norm of the projection matrix. Taking the row vectors in the projection matrix corresponding to different features of the sample vectors in the data matrix as an example, the specific formula is derived as shown in the following formula:

[0059]

[0060] Where Ω(W) is the inner product loss term; w i ,w j are all row vectors in the projection matrix W; ∑ i,j,i≠j | <w i ,w j >| represents the sum of the absolute values ​​of the inner products of each row vector in the projection matrix W and other row vectors with different serial numbers in the projection matrix. ‖W‖ F represents the Frobenius norm of the projection matrix W, ‖WW T‖1 represents the l1 norm of the product of the projection matrix and itself.

[0061] In summary, refer to Figure 1 , the total loss function and its corresponding constraints of this embodiment are specifically:

[0062]

[0063] Us T U=I,U≥0,V T V=I

[0064] Where ‖‖2 is the l2 norm; u i V T -x i T W represents the row or column vector x of the i-th feature of the sample vector i The product of the projection matrix W and the row or column vector u of the i-th feature of the corresponding sample vector in the basis matrix U of the orthogonal basis clustering i The difference between the product of the encoding matrix V of the orthogonal basis clustering; l() is the set logarithmic function; ‖W‖ 2,1 represents the projection matrix l as a regularization term in the total loss function 2,1 The loss term of the norm; represents the inner product loss term as a regularization term in the total loss function, where ‖WW T ‖1 represents the l1 norm of the product of the projection matrix W and itself, represents the square of the Frobenius norm of the projection matrix W; α and are respectively the projection matrix l as the regularization term in the total loss function 2,1 The hyperparameters corresponding to the norm loss term and the inner product loss term are both greater than 0; st represents the constraint corresponding to the total loss function. The overall minimization of the above formula is Figure 1 The objective function to be solved in is: Figure 1 The robust loss function in , That is Figure 1 Orthogonal basis clustering in , α‖W‖ 2,1 That is Figure 1 The sparse term constraints in That is Figure 1 The inner product regularization constraint in .

[0065] Specifically, the method of iteratively optimizing the optimized projection matrix with the goal of minimizing the total loss function includes: with the goal of minimizing the total loss function, the projection matrix, the base matrix of the orthogonal basis clustering and the encoding matrix are used as iterative variables, and the iterative optimization is performed by the alternating direction multiplier method (ADMM) until the iteration stop condition is met, and finally the optimized projection matrix is ​​obtained. Since the optimization of the objective function by the alternating direction multiplier method belongs to the prior art, only the process is described in this embodiment, and no specific principle analysis is performed; then the method of iteratively optimizing by the alternating direction multiplier method until the iteration stop condition is met, and finally the optimized projection matrix is ​​obtained is as follows:

[0066] First, by introducing the auxiliary variable H = X T W-UV T , G=U, rewrite the minimization of the total loss function (the minimization of the total loss function can be called the objective function) as follows:

[0067]

[0068] Us T U=I,V T V=I,H=X T W-UV T ,G=U,G≥0

[0069] The Lagrange multiplier method is used to solve the objective function, and the final function to be optimized is as follows:

[0070]

[0071] Where μ>0 is the penalty parameter. The principle of the alternating direction multiplier method is to iteratively optimize each variable while fixing the remaining variables until the algorithm converges. It should be emphasized that all subproblems have closed-form solutions. Then the kth iteration of the multi-module ADMM that solves the objective function is:

[0072]

[0073] Update W and fix all variables except W to get:

[0074]

[0075] Taking the derivative of the variable W, we get:

[0076] W=(αD+β(I d×d -I d )+μXX T ) -1 (μXUV T+μXH-X∑)

[0077] Among them I d×d and I d They represent a d×d dimensional matrix with all elements set to 1 and a d-dimensional identity matrix. The matrix D is a diagonal matrix, and the elements on its diagonal are defined as:

[0078]

[0079] where ∈>0 is a constant small enough to avoid w i =0. It is worth noting that D in the formula still depends on W. In other words, W cannot be calculated directly. Therefore, the alternating optimization method is used to iteratively calculate W and D. The pseudo code is designed as follows:

[0080]

[0081] Update H and fix all variables except H to get:

[0082]

[0083] make You can get:

[0084]

[0085] Transform the above formula into n independent sub-problems, and the i-th sub-problem is expressed as follows:

[0086]

[0087] For the above formula, we use the following theory to solve it:

[0088] Theorem 1: Assume m∈R n For a given vector, the loss function is For the following questions:

[0089]

[0090] The solution is:

[0091]

[0092] in Proof of Theorem 1: According to the definition of l(·) and the first-order optimality condition, the solution x * Must meet:

[0093]

[0094] in is ‖x‖2 at x * The subgradient at , which is expressed as:

[0095]

[0096] If x * =0, then the above formula is:

[0097]

[0098] According to the above equation, since We can get μ‖m‖2<γ.

[0099] If x * ≠0, then It is expressed as:

[0100]

[0101] The above formula can be rewritten as:

[0102]

[0103] Pair Taking the l2 norm on both sides of the equation, we can get:

[0104]

[0105] According to the formula It can be known that x * for:

[0106]

[0107] In fact, if γ<μ‖m‖2, we can obtain:

[0108] μ 2 (γ-‖m‖2) 2 -4μ(γ 6 -γμ‖m‖2)≥μ 2 (γ-‖m‖2) 2

[0109] This shows that is strictly defined.

[0110] According to the following formula:

[0111]

[0112] The above Theorem 1 can be proved. And the solution of matrix H is as follows:

[0113]

[0114] in

[0115] Then update U and fix all variables except U to get:

[0116]

[0117] According to relevant theories, the above formula can be transformed into the following problem

[0118]

[0119] in The relevant theories and proofs for solving this problem are as follows:

[0120] Theorem 2: Assume that M∈R m×n is a known matrix, P∈R m×m , Q∈R m×n is the left and right singular matrices of the matrix M calculated by singular value decomposition, then PI n×m Q T is the solution to the following problem:

[0121]

[0122] Proof of Theorem 2: Assume P∑Q T is the singular value decomposition of matrix M, P, Q are the left and right singular matrices of M, ∑∈R m ×n is a diagonal matrix with diagonal elements σ i ≥0(i=1,2,…,n), according to the properties of the matrix trace, we can get:

[0123] Tr(MU T )=Tr(P∑Q T U T )=Tr(∑Q T U T P)=Tr(∑Z)

[0124] Where Z = Q T U T P∈R n×m , since P, Q, and U are orthogonal, we can get

[0125] ZZ T =Q T U T PP T UQ=I

[0126] This means that Z has row orthogonality. Therefore, -1≤Z ij ≤1, and:

[0127]

[0128] The above equation holds true if and only if Zii =1(i=1,2,…,m), that is, Z=I n×m is the identity matrix, then

[0129]

[0130] Theorem 2 holds, so Us T The solution of U=I is:

[0131]

[0132] Where P U and Q U M U The left and right singular matrices after singular value decomposition, and

[0133] Then update V and fix all variables except V to get:

[0134]

[0135] SV T V=I

[0136] Using linear algebra problems, the above formula U=P U I n×c Q U T Can be updated to:

[0137] m V axTr(M V V T ),stV T V=I

[0138] in According to Theorem 2, SV T The solution of V = I is as follows:

[0139]

[0140] Where P V and Q V M V The left and right singular matrices after singular value decomposition.

[0141] Then update G and fix all variables except G to get:

[0142]

[0143] Pair Simplifying it gives:

[0144]

[0145] The above formula can be directly solved as:

[0146]

[0147] In summary, the pseudo code of the algorithm described above in this embodiment is designed as follows:

[0148]

[0149] Reference Figure 1 , after iterative optimization of the above algorithm 2, the final W is the optimized projection matrix obtained by iterative optimization with the goal of minimizing the total loss function; a set number of row or column vectors are selected according to the modulus lengths of each row or column vector corresponding to different features of the sample vector in the data matrix in the optimized projection matrix; the features in the sample vector corresponding to the selected row or column vector are used to construct the data matrix of the image after feature dimensionality reduction, thereby realizing image dimensionality reduction.

[0150] In this embodiment, the method of selecting a set number of row or column vectors according to the modulus length of each row or column vector in the optimized projection matrix that corresponds to different features of the sample vector in the data matrix includes selecting the set number of row or column vectors whose modulus length is larger than that of other row or column vectors in the optimized projection matrix that correspond to different features of the sample vector in the data matrix. For example, if the row vectors w in the projection matrix W that correspond to different features of the sample vector in the data matrix are i , then calculate all ‖w i ‖2(i=1,2,...,d) and sort them in order from large to small, select the row vectors with the highest number of rows, and use the features of the sample vectors in the data matrix corresponding to these row vectors as the features of the data matrix for constructing the image after feature dimensionality reduction. In other embodiments, sorting can also be performed from small to large, selecting the row vectors with the lowest number of rows, etc., as long as it can be ensured that the modulus of the selected row vectors with the highest number of rows is larger than the modulus of the other row vectors in the optimized projection matrix that correspond to different features of the sample vectors in the data matrix.

[0151] The present invention has the following characteristics:

[0152] 1) Feature selection and data clustering are integrated into a framework through a total loss function that includes an orthogonal basis clustering loss term and has a sparsity constraint, so as to ensure feature selection for unsupervised sparse learning through data clustering; on this basis, the orthogonal basis clustering result is processed by a logarithmic function to obtain an orthogonal basis clustering loss term. Considering that the loss function in the form of a logarithmic function has the smallest growth rate for the increase of error compared to other forms of loss functions, and when the error approaches infinity, its derivative approaches 0, which can effectively limit the influence of noise and outliers, that is, it is equivalent to using a logarithmic function to impose a robustness constraint on the orthogonal basis clustering loss term in the total loss function, thereby utilizing the robustness advantage of the logarithmic function to effectively reduce outliers and noise, and ensure the robustness of the total loss function, thereby effectively enhancing the robustness of feature dimensionality reduction through feature selection.

[0153] 2) The sum of the absolute values ​​of the inner products of each row vector in the projection matrix and other column vectors with different sequence numbers in the projection matrix is ​​used as the regularization term of the total loss function to ensure that when the optimization is performed with the minimum total loss function as the goal, the optimization result has the characteristic of low redundancy.

[0154] 3) Through the projection matrix l as a regularization term in the total loss function 2,1 The loss term of the norm realizes the sparse constraint, that is, converting the sparse optimization problem into an unconstrained regularized optimization problem, avoiding the situation where the optimization of the total loss function is difficult and NP-hard due to the sparse constraint.

[0155] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, but do not constitute a limitation to the present invention.

Claims

1. An unsupervised image feature dimensionality reduction method, It is characterized in that An optimized projection matrix is ​​obtained by iterative optimization with the goal of minimizing the total loss function; the projection matrix corresponds to a data matrix composed of sample vectors containing more than two features of each image to be feature-reduced; a set number of row or column vectors are selected according to the modulus length of each row or column vector corresponding to different features of the sample vector in the data matrix in the optimized projection matrix; the features in the sample vector corresponding to the selected row or column vector are used to construct a data matrix of the image after feature dimensionality reduction; The total loss function contains the following orthogonal basis clustering loss terms: z i The row or column vector x containing the i-th feature of the sample vector i The product of the projection matrix and the row or column vector u of the i-th feature of the sample vector in the basis matrix of the orthogonal basis clustering i The difference between the product of the encoding matrix of the orthogonal basis clustering and l() is the set logarithmic function; the constraints of the total loss function include sparse constraints.

2. The unsupervised image feature dimensionality reduction method according to claim 1, It is characterized in that The total loss function also includes an inner product loss term as a regularization term; if the projection matrix contains row vectors that correspond to different features of the sample vectors in the data matrix, the inner product loss term includes the sum of the absolute values ​​of the inner products of each row vector in the projection matrix and other row vectors with different serial numbers in the projection matrix; if the projection matrix contains column vectors that correspond to different features of the sample vectors in the data matrix, the inner product loss term includes the sum of the absolute values ​​of the inner products of each column vector in the projection matrix and other column vectors with different serial numbers in the projection matrix.

3. The unsupervised image feature dimensionality reduction method according to claim 1 or 2, It is characterized in that The constraints of the total loss function also include orthogonality constraints and non-negativity constraints on the basis matrix in orthogonal basis clustering and orthogonality constraints on the encoding matrix.

4. The unsupervised image feature dimensionality reduction method according to claim 1 or 2, It is characterized in that The set logarithmic function is: Wherein, l(x) is the set logarithmic function; x is a variable in the logarithmic function, and γ is a set numerical parameter.

5. The unsupervised image feature dimensionality reduction method according to claim 1 or 2, It is characterized in that The method of selecting a set number of row or column vectors according to the modulus length of each row or column vector in the optimized projection matrix that respectively corresponds to different characteristics of the sample vector in the data matrix includes selecting the set number of row or column vectors whose modulus length is larger than that of other row or column vectors in the optimized projection matrix that respectively correspond to different characteristics of the sample vector in the data matrix.

6. The unsupervised image feature dimensionality reduction method according to claim 1 or 2, It is characterized in that The sample vector containing more than two features of the image to be feature-reduced is obtained by performing pixel-based expansion on a grayscale image composed of pixel values ​​obtained after grayscale processing of the image to be feature-reduced.

7. The unsupervised image feature dimensionality reduction method according to claim 3, It is characterized in that The sparse constraint is implemented by using the projection matrix l as a regularization term in the total loss function. 2,1 The loss term implementation of the norm.

8. The unsupervised image feature dimensionality reduction method according to claim 1 or 2, It is characterized in that The method of iteratively optimizing the optimized projection matrix with the goal of minimizing the total loss function includes: taking the minimization of the total loss function as the goal, taking the projection matrix, the base matrix of the orthogonal basis clustering and the encoding matrix as iterative variables, performing iterative optimization by the alternating direction multiplication method until the iterative stop condition is met, and finally obtaining the optimized projection matrix.

9. The unsupervised image feature dimensionality reduction method according to claim 2, It is characterized in that The absolute sum of the inner products of each row vector in the projection matrix and each row vector with a different sequence number in the projection matrix, and the absolute sum of the inner products of each column vector in the projection matrix and other column vectors with different sequence numbers in the projection matrix are calculated by calculating the product of the projection matrix and itself. 1 The difference between the norm and the square of the Frobenius norm of the projection matrix is ​​obtained.

10. The unsupervised image feature dimensionality reduction method according to claim 3, It is characterized in that The orthogonality constraints on the basis matrix include: IN T U=I The orthogonality constraints on the encoding matrix include: V T V=I The non-negativity constraints on the basis matrix include: U≥0 Where U is the basis matrix, V is the encoding matrix, I is the identity matrix, and U T is the transposed matrix of U, V T is the transposed matrix of V.

Images