A robust feature space joint learning method for multi-label image classification

Abstract

The present invention discloses a multi-label image classification method for robust feature space co-learning. First, multi-label data is prepared; a low-rank representation method of feature selection and nuclear norm is introduced in the feature space and the label space to construct a multi-label classification model for robust label and feature space co-learning; a q-dimensional label space is mapped to an r-dimensional label space using a robust low-rank projection matrix V, so that V represents the robust low-rank projection learned in the label space; an augmented Lagrange multiplier method is introduced to solve the matrix P and the matrix V in the objective function of the multi-label classification model; the multi-label data is used as a training data sample to train the proposed multi-label classification model to obtain the optimal robust low-rank coefficient matrix P and the robust low-rank projection matrix V; the training of the multi-label classification model is completed based on the optimal robust low-rank coefficient matrix P and the robust low-rank projection matrix V, and the multi-label classification model is used to perform multi-label image classification.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and in particular to a multi-label image classification method of robust feature space co-learning. Background Art

[0002] In recent years, multi-label image classification is one of the popular technologies in the field of machine learning and has been widely used to solve various real-world application problems. Unlike traditional supervised learning with only one mutually exclusive label, multi-label classification contains multiple different labels. Due to the ubiquitous correlation between the feature space and label space of multi-label classification, the complexity of the training space increases with the number of samples, and there is also label correlation in the multi-label, all of which have a negative impact on the classification performance.

[0003] However, most of the current multi-label image classification methods only study the technology related to the feature space, while ignoring the research on the label output space. Therefore, most multi-label classifiers are deeply affected by the label quality.

[0004] In addition, noise caused by data corruption and missing is common in high-dimensional raw data sets, which greatly reduces the accuracy of classification results. Currently, most classification technologies only focus on feature space noise, while very few studies on label space noise are conducted. Therefore, these technologies are not effective in anti-noise function in multi-label classification. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the present invention proposes a multi-label image classification method for co-learning robust feature space, which adopts low-rank representation technology to jointly learn the robust low-rank feature structure in the label and feature space. Secondly, the high-dimensional space is projected into the low-dimensional space through dimensionality reduction technology, so as to further learn the robust low-rank label projection. In addition, this method also uses the nuclear norm method to make the classifier perform better in the original noisy data. To improve the effectiveness and accuracy of multi-label classification.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A multi-label image classification method for robust feature space joint learning includes the following steps:

[0008] Step 1: Prepare multi-label data to form a feature matrix X of multi-label data and a label matrix Y related to the feature matrix X;

[0009] Step 2: Introduce feature selection and low-rank representation methods of nuclear norm in feature space and label space to construct a robust multi-label classification model that jointly learns labels and feature spaces; the objective function of this model is expressed as:

[0010]

[0011] Wherein, P is a robust low-rank coefficient matrix, V is a robust low-rank projection matrix, λ1, λ2 represent the regularization parameters of the matrix P and the matrix V respectively; the robust low-rank projection matrix V is used to map the q-dimensional label space to the r-dimensional label space, r<<q, so that V represents the robust low-rank projection learned in the label space;

[0012] Step 3: Introduce the augmented Lagrange multiplier method to solve the matrix P and matrix V in the objective function of the multi-label classification model;

[0013] Step 4: According to the regularization parameters λ1, λ2 and the penalty parameter μ, the multi-label data is used as training data samples to train the proposed multi-label classification model to obtain the optimal robust low-rank coefficient matrix P and the robust low-rank projection matrix V; based on the optimal robust low-rank coefficient matrix P and the robust low-rank projection matrix V, the training of the multi-label classification model is completed, and the multi-label classification model is used to perform multi-label image classification.

[0014] Furthermore, the method for solving the matrix P and the matrix V in step 3 is as follows:

[0015] Two auxiliary matrices J and W are used to replace P and V in the nuclear norm symbols in the objective function, and the augmented Lagrange multiplier method is used to transform the objective function into the following unconstrained optimization problem:

[0016]

[0017] Among them, Y1 and Y2 are Lagrange multipliers, Y is the label space, and X is the feature space; λ1, λ2>0 represent the regularization parameters of P and V respectively, and μ>0 is a penalty parameter; the unconstrained optimization solution problem based on the above formula solves J, P, W, V, Y1, and Y2 respectively; further, the values ​​of matrices P, W, V, Y1, and Y2 are fixed, and the solution problem of the auxiliary matrix J is transformed into an effective sub-problem of the unconstrained optimization solution problem by using the method of nuclear norm minimization. Therefore, the solution formula of J is:

[0018]

[0019] Furthermore, the values ​​of matrices P, Q, J, V, Y1, and Y2 are fixed, and the solution problem of the auxiliary matrix J is transformed into an effective subproblem of the unconstrained optimization solution problem by using the method of nuclear norm minimization. Therefore, the solution formula of W is:

[0020]

[0021] Furthermore, the auxiliary matrix Q is introduced to replace the matrix P to solve the robust low-rank coefficient matrix P, which is expressed as:

[0022]

[0023] Among them, Y3 is the Lagrange multiplier;

[0024] Taking partial derivative of the matrix Q in the above formula, we get the following formula:

[0025] Then take the partial derivative of the matrix P and get the following formula:

[0026] Furthermore, the matrix V in the problem of unconstrained optimization solution is directly derived to obtain the following formula:

[0027]

[0028] Further, update the matrices Y1, Y2, Y3

[0029] Y1=Y1+μ(PJ)

[0030] Y3=Y3+μ(PQ)

[0031] Y2=Y2+μ(VW)

[0032] At the beginning of the training process, the initial values ​​of the three matrices Y1, Y2, and Y3 are all set to 0.

[0033] Furthermore, the iterative update process is as follows: set the initial matrix J=0, Q=0, P=0, W=0, V=0; solve the parameter J, and then update the matrices Q, P, W, V and Y1, Y2, Y3 in turn, and calculate the loss value based on the solved parameters; at the same time, the number of iterations is increased by 1; then the above cycle is continued; until the loss value is less than 1e-7 or the number of iterations is greater than 1e4, the loop is stopped to obtain the optimal values ​​of the matrices P and V.

[0034] Beneficial effects of the present invention:

[0035] The multi-label image classification method proposed in this application uses feature learning methods in both feature space and label space to solve related problems in high-dimensional data samples and labels. In addition, this method further effectively deals with the problem of the influence of noise on image classification accuracy by introducing low-rank representation methods in feature and label space. Through extensive experiments on real-world multi-label datasets, this method is compared with traditional multi-label image classification methods, and all evaluation indicators are better than other algorithms. Therefore, the multi-label image classification method proposed in this method can work better in noise and high-dimensional space. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the multi-label image classification method with robust feature space joint learning. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] Combined with Figure 1 The multi-label image classification method proposed in this application for robust feature space co-learning can be divided into two parts, wherein 1. Training is performed based on different types of multi-label data sets provided by relevant websites. The proposed model is continuously optimized to obtain a robust low-rank coefficient matrix and a robust low-rank projection matrix. 2. Based on the optimal relevant parameters of the obtained classification model, further evaluation and testing of image classification learning is performed.

[0039] 1. Train the optimal parameters of the proposed multi-label classification model

[0040] The multi-label image classification method provided by the present invention includes the following steps:

[0041] Step 1: According to the multi-label dataset provided on the Mulan website, a multi-label training sample is given x i is the i-th feature, y i is the i-th label, i = 1, 2, ..., N; for space R, let X = [x1, x2, ..., x n ]∈R pxn is a p-dimensional feature matrix, x n is the nth feature, n is the number of features; Y = [y1, y2, ..., y n ]∈R qxn It is represented as the label matrix associated with the feature matrix X in the q-dimensional output label space. Where n represents the number of labels in the training set.

[0042] Step 2: Construct a robust multi-label classification model that jointly learns labels and feature spaces; the details are as follows:

[0043] S2.1. Based on the provided data set, Y is the label space and X is the feature space. A projection matrix is ​​introduced into the label space to map a high-dimensional space to a low-dimensional space.

[0044] S2.2, and introduce the low-rank representation method of the nuclear norm in the feature space and label space, so as to construct a robust multi-label classification model that learns the label and feature space together. The model is defined as the following objective function:

[0045]

[0046] Among them, λ1, λ2>0, respectively represent the regularization parameters of the matrix P and the matrix V. During training, the values ​​of these two parameters can be respectively in {10 -2 , 10 0 , ..., 10 3} interval, ||·|| * represents the nuclear norm, represents the L2 norm. P∈R pxq Defined as a p-dimensional robust low-rank coefficient matrix. V∈R qxr It is a q-dimensional robust low-rank projection matrix. This robust low-rank projection matrix V is used to map the q-dimensional label space to the r-dimensional label space (r<<q), so that V represents the robust low-rank projection learned in the label space. Among them, the value of r can be arbitrarily selected within the range of the label dimension.

[0047] Step 3: Introduce the augmented Lagrange multiplier method to solve the objective function of the multi-label classification model, namely the matrix P and the matrix V in formula (1).

[0048] First, use two auxiliary matrices J∈R pxq and W∈R qxr Respectively replace P and V in the nuclear norm symbols of formula (1), and use the augmented Lagrange multiplier method to transform formula (1) into the following unconstrained optimization solution problem:

[0049]

[0050] Among them, Y1 and Y2 are Lagrange multipliers, tr(*) is the trace of the matrix, μ>0 is a penalty parameter, and the value of parameter μ can be in {10 -6 , 10 0 , ..., 10 6} to make a selection.

[0051] The matrix solutions for J, P, W, V, Y1, and Y2 in formula (2) are as follows:

[0052] A. Solving the auxiliary matrix J

[0053] The solution problem of the auxiliary matrix J is transformed into a valid subproblem of formula (2) by using the method of minimizing the nuclear norm. The values ​​of matrices P, W, V, Y1, and Y2 are fixed, so the solution formula of J is:

[0054]

[0055] B. Solving the robust low-rank coefficient matrix P

[0056] The auxiliary matrix Q is used to replace the matrix P in the formula, so the formula is as follows:

[0057]

[0058] Among them, Y3 is the Lagrange multiplier, and its initial value is 0.

[0059] Taking partial derivative of the matrix Q in formula (4), we get the following formula:

[0060]

[0061] Taking partial derivative of the matrix P in formula (4), we get the following formula:

[0062]

[0063] C. Solving the auxiliary matrix W

[0064] The method for solving J is the same as that for solving J. The solution problem of the auxiliary matrix W is transformed into a valid subproblem of formula (2) by using the method of minimizing the nuclear norm. The values ​​of matrices P, Q, J, V, Y1, and Y2 are fixed, so the solution formula for W is:

[0065]

[0066] D. Solving the robust low-rank projection matrix V

[0067] Directly calculate the partial derivative of the matrix V in formula (2):

[0068]

[0069] E. Update matrices Y1, Y2, Y3

[0070] Y1=Y1+μ(PJ) (9)

[0071] Y3=Y3+μ(PQ) (10)

[0072] Y2=Y2+μ(VW) (11)

[0073] At the beginning of the training process, the initial values ​​of the three matrices Y1, Y2, and Y3 are all set to 0.

[0074] Step 4: According to the above-mentioned parameters λ1, λ2, μ and the input training data set, the proposed multi-label classification model is trained to obtain the optimal robust low-rank coefficient matrix P and robust low-rank projection matrix V.

[0075] The iterative update process is as follows: First, initialize the matrix in formula (2). At the beginning, set the initial matrix J = 0, Q = 0, P = 0, W = 0, V = 0. First, fix the Q, P, W, V, Y1, Y2 matrices, solve the parameter J, then update the matrices Q, P, W, V and Y1, Y2, Y3 in turn, and calculate the loss value based on the solved parameters. At the same time, the number of iterations is increased by 1, and then the above cycle is repeated. The cycle stops when the loss value is less than 1e-7 or the number of iterations is greater than 1e4, and the optimal values ​​of matrices P and V are obtained.

[0076] 2. Evaluation and testing of multi-label classification models

[0077] The following multi-label classification tests and evaluations are performed based on the obtained optimal parameters. The size of the label space dimension is selected by feature selection to further reduce the high-dimensional space. The r setting can be arbitrarily selected from the interval ranging from 1 to the size of the label space dimension. Then, the optimal classification model is evaluated based on the relevant metric evaluation criteria for multi-label classification, such as Micro-F1, Example-F1 and Hamming Loss, and the evaluation results are compared with the results of other multi-label classification methods. Our method is compared with the most advanced multi-label classification methods such as BR, PLST, CSSP and CPLST on datasets such as Cal500, Rcv1s1, Corel16k 7 and Corel16k4. The experimental results show that the Example-F1 metric results of our method on Cal500, Rcv1s 5, Corel16k 7 and Corel16k4 are 1.4%, 1.3%, 1.3% and 1.8% higher than the results of all comparison methods on average. At the same time, we also added Gaussian noise to the Cal500 and Corel6k7 datasets and conducted experiments. We can observe that in the Cal500 dataset environment with a noise density of 10%, for the Micro-F1 evaluation index, the Micro-F1 evaluation result of our method is 0.3647, and the Micro-F1 evaluation result of the FaIE method with the best performance among other methods is 0.3588. Finally, the comparison results clearly verify that our method performs better than other algorithms in noisy data.

[0078] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A multi-label image classification method with robust feature space joint learning, characterized in that: It includes the following steps: Step 1: Prepare multi-label data to form a feature matrix X of the multi-label data and a label matrix Y related to the feature matrix X; Step 2: Introduce the low-rank representation of the nuclear norm in the feature space and the label space, and construct a multi-label classification model for jointly learning the robust label and feature space; the objective function of this model is expressed as: where P is a robust low-rank coefficient matrix, V is a robust low-rank projection matrix, and λ1 and λ2 respectively represent the regularization parameters of matrix P and matrix V; the q-dimensional label space is mapped to an r-dimensional label space by using the robust low-rank projection matrix V, where r << q, so V represents the robust low-rank projection learned in the label space; Step 3: Introduce the augmented Lagrangian multiplier method to solve matrices P and V in the objective function of the multi-label classification model; Step 4: According to the regularization parameters λ1, λ2 and the penalty parameter μ, use the multi-label data as training data samples to train the proposed multi-label classification model to obtain the optimal robust low-rank coefficient matrix P and robust low-rank projection matrix V; complete the training of the multi-label classification model based on the optimal robust low-rank coefficient matrix P and robust low-rank projection matrix V, and use this multi-label classification model for multi-label image classification; The method for solving matrices P and V in Step 3 is as follows: Use two auxiliary matrices J and W to replace P and V in the nuclear norm symbol in the objective function respectively, and adopt the augmented Lagrangian multiplier method to transform the objective function into the following problem of unconstrained optimization solution: where Y1 and Y2 are Lagrangian multipliers, Y is the label space, and X is the feature space; λ1, λ2 > 0 respectively represent the regularization parameters of P and V, and μ > 0 is a penalty parameter; solve J, P, W, V, Y1, and Y2 respectively based on the problem of unconstrained optimization solution of the above formula.

2. The multi-label image classification method of robust feature space co-learning according to claim 1, characterized in that: Fix the values of matrices P, W, V, Y1, and Y2, and use the method of minimizing the nuclear norm to transform the solution problem of the auxiliary matrix J into an effective sub-problem of the problem of unconstrained optimization solution. Therefore, the solution formula for J is:

3. The multi-label image classification method of robust feature space co-learning according to claim 1, characterized in that: Fix the values of matrices P, J, V, Y1, and Y2, and use the method of minimizing the nuclear norm to transform the solution problem of the auxiliary matrix W into an effective sub-problem of the problem of unconstrained optimization solution. Therefore, the solution formula for W is:

4. The method for multi-label image classification based on robust feature space co-learning according to claim 1, characterized in that: Introduce an auxiliary matrix Q to replace matrix P, and solve the robust low-rank coefficient matrix P, which is expressed as: where Y3 is a Lagrangian multiplier; Taking partial derivative of the matrix Q in the above formula, we get the following formula: Then take the partial derivative of matrix P to obtain the following Mode:

5. The method for multi-label image classification based on robust feature space co-learning according to claim 4, characterized in that: Take the direct partial derivative of matrix V in the problem of unconstrained optimization solution to obtain the following formula:

6. The multi-label image classification method of robust feature space co-learning according to claim 4, characterized in that: Update matrices Y1, Y2, and Y3 Y1 = Y1 + μ(P - J); Y3 = T3 + μ(P - Q); T2 = Y2 + μ(V - W).

7. The method for multi-label image classification based on robust feature space co-learning according to claim 4, characterized in that: The process of iterative update is as follows: Let the initial matrices J = 0, Q = 0, P = 0, W = 0, V = 0; solve the parameter J, and then update matrices Q, P, W, V, Y1, Y2, and Y3 in turn, and calculate the loss value according to the solved parameters; at the same time, increase the iteration count by 1; then continuously perform iterative update; stop the loop until the loss value is less than 1e-7 or the iteration count is greater than 1e4, and obtain the optimal values of matrices P and V.

Images