A semi-supervised image feature extraction method and device, and storage medium

Abstract

The present invention discloses a semi-supervised image feature extraction method and device, and a storage medium, including: constructing a similarity graph and a divergence matrix according to the original features of image data; constructing a semi-supervised image feature extraction model according to the similarity graph and the divergence matrix; performing equivalent transformation on the semi-supervised image feature extraction model; performing alternating iterative calculations on the semi-supervised image feature extraction model after equivalent transformation to obtain a mapping matrix for feature extraction; and obtaining an image feature extraction result according to the image data and the mapping matrix. The technical solution of the present invention can overcome the shortcomings of the existing semi-supervised image feature extraction method based on linear discriminant analysis, which is difficult to meet the imposed constraints and thus affects the quality of image feature extraction.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to a semi-supervised image feature extraction method and device based on separation manifold constraints for image recognition, and a storage medium. Background Art

[0002] As the scale of data continues to expand, authoritative data labels are becoming increasingly difficult to obtain. Semi-supervised methods, as a learning framework that uses data information to enhance the model's ability to extract information, have received widespread attention. The document "D. Cai, X. He, and J. Han, Semi-Supervised Discriminant Analysis, Proceedings of International Conference on Computer Vision, 2007" discloses a semi-supervised image feature extraction method based on linear discriminant analysis. This method introduces the local structural relationship of the image in the feature space into the goal of linear discriminant analysis to form a semi-supervised model that simultaneously utilizes label information and local manifold structure. In image recognition, each image is regarded as a sample point in the data space. This method first constructs a similarity graph based on the original features of the image data using a Gaussian kernel function to capture the local structural information of the image data distributed in the feature space. The basic idea is to use the assumption that similar image data should be close in the feature space and the corresponding similarity should be large to establish a smooth constraint term so that the extracted sample features also meet this constraint. Since the constraint target and the target direction of the smoothness term of the linear discriminant analysis for the same type of samples are both minimized, this method adds the two through the regularization coefficient to replace the corresponding terms in the original linear discriminant analysis, and finally uses the problem-solving method of the linear discriminant analysis to extract features. However, when establishing the target, this method combines the smoothness constraint with the constraint on the same type of image data and uses the same target to control it, so that each target may not meet the target assumption, and it is difficult to ensure that the final extracted features meet the constraints added at the beginning, so the quality of the extracted image data features cannot be guaranteed. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a semi-supervised image feature extraction method and device based on separation manifold constraints for image recognition, and a storage medium, so as to overcome the shortcomings of the existing semi-supervised image feature extraction method based on linear discriminant analysis that it is difficult to meet the imposed constraints, thereby affecting the quality of image feature extraction.

[0004] To achieve the above object, the present invention adopts the following technical solution:

[0005] A semi-supervised image feature extraction method comprises the following steps:

[0006] Step S1, constructing a similarity graph and a divergence matrix according to the original features of the image data;

[0007] Step S2: constructing a semi-supervised image feature extraction model according to the similarity graph and the divergence matrix;

[0008] Step S3, performing equivalent transformation on the semi-supervised image feature extraction model;

[0009] Step S4, the semi-supervised image feature extraction model after equivalent transformation is calculated by using an optimization method of alternating iterative updates of the mapping matrix and the auxiliary variables to obtain a mapping matrix for feature extraction;

[0010] Step S5: Obtain feature extraction results based on the image data and the mapping matrix.

[0011] Specifically, in step S1, the similarity graph is:

[0012]

[0013] Among them, s ij Represents image data x i With image data x j The similarity between ij is the element located in the i-th row and j-th column of the similarity matrix S, and γ is the regularization parameter to ensure that the building block of the similarity graph will not obtain a trivial solution, that is, each row in the obtained similarity matrix has only one non-zero value.

[0014] In step S1, the scatter matrix includes: intra-class scatter matrix S w and the inter-class scatter matrix S b , which is constructed as follows:

[0015]

[0016]

[0017] Among them, X i represents the set of image data of the i-th category, μ i represents the average value of the features of the i-th category image data, μ is the average value of all labeled image features, and n i is the number of image data of the i-th category, and C is the total number of categories of image data.

[0018] In step S2, the semi-supervised image feature extraction model is:

[0019]

[0020] Where Tr(·) represents the trace operation of the matrix, α is the regularization coefficient, W is the mapping matrix used for feature extraction, and L is the graph Laplacian matrix corresponding to the similarity matrix S.

[0021] In step S3, the semi-supervised image feature extraction model is rewritten into the following form after equivalent transformation:

[0022]

[0023] Among them, s is an auxiliary variable and can be determined by the extreme point of the quadratic function. The specific calculation method is:

[0024] The present invention also provides a semi-supervised image feature extraction device, comprising:

[0025] The first construction module is used to construct a similarity graph and a divergence matrix according to the original features of the image data;

[0026] A second construction module is used to construct a semi-supervised image feature extraction model according to the similarity graph and the divergence matrix;

[0027] A conversion module, used for performing equivalent conversion on the semi-supervised image feature extraction model;

[0028] A calculation module, used for calculating the semi-supervised image feature extraction model after equivalent transformation by using an optimization method of alternately iteratively updating the mapping matrix and the auxiliary variables to obtain a mapping matrix for feature extraction;

[0029] The extraction module is used to obtain the extraction result of the image features according to the image data and the mapping matrix.

[0030] Preferably, the building blocks of the similarity graph are:

[0031]

[0032] Among them, sij represents the image data x i The similarity between sij and the image data xj, sij is the element located in the i-th row and j-th column in the similarity matrix S, and γ is the regularization parameter, which can be adjusted according to the obtained data.

[0033] Preferably, the scatter matrix includes: the intra-class scatter matrix S w and the inter-class scatter matrix S b ,Right now:

[0034]

[0035]

[0036] Among them, Xi represents the set of image data of the i-th category, μ i represents the average value of the features of the i-th category image data, μ is the average value of all labeled image features, and n i is the number of image data of the i-th category, and C is the total number of categories of image data.

[0037] As a preferred option, the semi-supervised image feature extraction model is equivalently converted to:

[0038]

[0039] Among them, s is an auxiliary variable, and its calculation method is Tr(·) represents the trace operation of the matrix, and α is the regularization coefficient.

[0040] The present invention also provides a storage medium, which stores machine executable instructions. When the machine executable instructions are called and executed by a processor, the machine executable instructions prompt the processor to implement a semi-supervised image feature extraction method.

[0041] The present invention further clarifies the role of supervised information and unsupervised information by extracting the smoothness term constraint from the linear discriminant objective as an independent regularization term constraint. At the same time, the design of the separation structure makes the meaning of each constraint more singular, and there will be no problem of constraints being entangled with each other. Based on this, the present invention uses a more general semi-supervised model design principle to establish a semi-supervised image feature extraction method with clearer constraint objectives, which can better meet the target constraints, thereby improving the quality of the extracted image features and the accuracy of tasks such as image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a semi-supervised image feature extraction method based on separation manifold constraints provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of the structure of a semi-supervised image feature extraction device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0045] Embodiment 1:

[0046] like Figure 1As shown, the present invention provides a semi-supervised image feature extraction method, comprising the following steps:

[0047] Step 1: Image data preprocessing

[0048] In order to reduce the noise in the image data, this embodiment first normalizes the image data and uses the principal component analysis method PCA to eliminate the influence of the null space on the image data feature extraction process, and then obtains the original feature X = {x1, x2, ..., xn} of the image data and uses it as input, where x i , i=1,2,...,n, represents the i-th data in the image data.

[0049] Step 2: Construct similarity graph and divergence matrix

[0050] In order to better utilize the local structural information of the image data in the original feature space, this embodiment constructs a similarity graph, and its specific implementation rules are as follows:

[0051]

[0052] Among them, sij represents the image data x i The similarity between sij and the image data xj. sij is the element in the similarity matrix S located in the i-th row and j-th column. γ is a regularization parameter, which can be calculated through the image data features in a subsequent calculation process in a specific implementation to avoid manual setting of this parameter. Here, this embodiment normalizes each row of the similarity graph and limits the similarity between image data to be greater than or equal to 0. In order for the similarity graph to be able to show the differences between image data, this embodiment limits each row in S to have only k non-zero values, that is, for each image data, only the similarity with its k most similar image data is retained. In order to obtain a solution to the problem formula (1), first calculate the matrix A composed of the squares of the Euclidean distances between the two image data, and sort the elements in each row of A from small to large, and then the elements in the similarity matrix S can be obtained as:

[0053]

[0054] in, Represents image data x i The square of the Euclidean distance from the data xj, and a ij is the element in the i-th row and j-th column of matrix A,

[0055] The scatter matrix includes the intra-class scatter matrix S w and the inter-class scatter matrix S b , and its calculation process is shown in the following two formulas:

[0056]

[0057]

[0058] Among them, X i represents the set of image data of the i-th category, μ i represents the average value of the features of the i-th category image data, μ is the average value of all labeled image features, and n i is the number of image data of the i-th category, and C is the total number of categories of image data.

[0059] Step 3: Establish a semi-supervised image feature extraction model based on separation structure. According to the idea of ​​the proposed separation structure, a semi-supervised image feature extraction model is established:

[0060]

[0061] Where Tr(·) represents the trace operation of the matrix and α is the regularization coefficient. b , S w are the inter-class and intra-class divergence matrices respectively. The second term is the feature smoothing term, which aims to maintain the original local structural relationship of the image data in the target feature space. W is the mapping matrix used for feature extraction. L is the graph Laplacian matrix corresponding to S, and its calculation method is L=DS. D is the degree matrix corresponding to the similarity graph S. D is a diagonal matrix, and its diagonal element in the i-th row and i-th column is the sum of the elements in the i-th row of the similarity matrix.

[0062] Step 4: Perform equivalent transformation on the semi-supervised image feature extraction model

[0063] The semi-supervised image feature extraction model is converted to an equivalent form. The purpose of the conversion is to convert the complex form of the original target into an equivalent target that is easy to solve. After introducing the auxiliary variable s, the semi-supervised image feature extraction model is converted to the following form:

[0064]

[0065] Step 5: Solve for the auxiliary variable s

[0066] When only focusing on s, the target is a quadratic function about s, so the auxiliary variable s can be obtained by calculating the extreme point of the quadratic function, that is:

[0067]

[0068] Step 6: Calculate the mapping matrix W

[0069] The mapping matrix W can first be equivalently transformed by Algorithm 2 in the document "F. Nie, J. Yuan, and H. Huang, Optimal Mean Robust Principal component analysis, Proceedings of International Conference on Machine Learning, 2014, 1062-1070", and then calculated using the GPI algorithm in the document "F. Nie, R. Zhang, X. Li, A Generalized Power Iteration Method for Solving Quadratic Problem on the Stiefel Manifold, Science China Information Sciences, vol. 50, no. 11, pp. 1-10, 2017". The mapping matrix can be regarded as a set of bases in the target feature space, and the extracted features can be expressed as a combination of this set of bases.

[0070] Furthermore, the calculation of the mapping matrix W in step 6 as described above includes the following steps:

[0071] Step 6-1. Calculate the intermediate variable M

[0072] Since the first and third terms in the target equation (6) can be combined, let A = s 2 S w +αXLX T , then the intermediate variable M=2A λ W+2B, where A λ =λI-A, λ is the guarantee of A λ is the parameter of the positive definite matrix, where λ can be directly set as the main eigenvalue of matrix A. It is an intermediate variable generated by equivalent transformation using Algorithm 2 in the document “F. Nie, J. Yuan, and H. Huang, Optimal Mean Robust Principal component analysis, Proceedings of International Conference on Machine Learning, 2014, 1062-1070”.

[0073] Step 6-2: Perform compact singular value decomposition on the intermediate variable M

[0074] The matrix M can be obtained by compact singular value decomposition as follows

[0075] M=U∑VT (8)

[0076] Among them, U and V are matrices composed of left singular vectors and right singular vectors respectively, and ∑ is a diagonal matrix composed of non-zero singular values ​​of M as diagonal elements.

[0077] Step 6-3, calculate the mapping matrix W

[0078] The update rule for W is

[0079] W=UV T (9)

[0080] Step 7: Repeat the calculation process of the auxiliary variable s and the mapping matrix W in steps 5 and 6, and iterate and update them alternately until the objective function in formula (6) converges, that is, the change in the objective function value is less than a certain threshold, such as 10 -6 .

[0081] Step 8: Output the mapping matrix W after the objective function converges as shown in formula (6). Correspondingly, for the image data x i , the target feature extracted by the semi-supervised image feature extraction model is

[0082] y=W T x i (10)

[0083] The present invention can be further illustrated by the following simulation results.

[0084] The present invention is that the central processing unit is The simulation is performed using MATLAB software on an i7-9700K 3.5GHz CPU, 16G memory, and WINDOWS 10 operating system.

[0085] The datasets used in the simulation come from 1632 images in the CMU_Pie database and 2600 images in the AR database. In each database, 20% of each type of image has a category label. First, the relevant methods are used to extract features from all images used in each database, and then the 1NN classifier is used to obtain the classification results of the corresponding images. In order to verify the superiority of the proposed method in image recognition, face images are selected as specific tasks, so the simulation can be regarded as a comparison of face recognition performance.

[0086] In order to prove the effectiveness of the present invention, this embodiment selects SDA, TR-FSDA, SPGO, SELF, and ALLDA as comparison methods on two databases. Among them, SDA uses the method proposed in the document "D. Cai, X. He, and J. Han, Semi-Supervised Discriminant Analysis, Proceedings of International Conference on Computer Vision, 2007" to extract features; TR-FSDA uses the method proposed in the document "Y. Huang, D. Xu, and F. Nie, Semi-supervised Dimension Reduction using Trace Ratio Criterion, IEEE Transactions on Neural Networks and Learning Systems, vol. 23, no. 3, pp. 519–526, 2012" to extract data features; SPGO uses the method proposed in the document "F. Nie, X. Dong, and X. Li, Unsupervised and Semi-supervised Projection with Graph Optimization, IEEE Transactions on Neural Networks and Learning Systems, vol. 23, no. 3, pp. 519–526, 2012" to extract data features. Systems, vol.32, no.4, pp.1547–1559, 2020”; SALE uses the method proposed in the literature “F. Nie, Z. Wang, R. Wang, and X. Li, Adaptive Local Embedding Learning for Semi-supervised Dimensionality Reduction, IEEE Transactions on Knowledge and Data Engineering, 2021” to extract data features. SDELP is the feature extracted by this method. F is the evaluation index of face recognition performance. The performance comparison results of the corresponding image feature extraction models on image classification (face recognition) tasks are shown in Tables 1 and 2, which respectively represent the results on the CMU_Pie and AR databases.

[0087] Table 1

[0088] method SDA TR-FSDA SPGO SALE SDELP F 53.636 42.495 28.523 46.918 64.655

[0089] Table 2

[0090]

[0091]

[0092] It can be seen from Table 1 and Table 2 that the image classification performance of the method of this embodiment on both databases is significantly improved, especially on the AR database, where the performance improvement reaches 20.54%.

[0093] The separation structure design adopted by the method of this embodiment, that is, on the basis of the linear discriminant analysis objective, through the design criteria of general semi-supervised objectives, the smoothness constraint is added to the objective of the linear discriminant analysis through the regularization coefficient. The role of supervised and unsupervised information in the feature extraction model is further clarified, and the deficiency of the existing semi-supervised image feature extraction method based on linear discriminant analysis that it is difficult to meet the imposed constraints is overcome, thereby improving the quality of the extracted features.

[0094] By separating the manifold constraint from the objective of linear discriminant analysis, the manifold information can be better captured; the constraint of supervised information is separated from the constraint of unsupervised information, so that this embodiment can better balance the information provided by the two, and then extract more effective image data features. The simulation experiment results show that compared with the existing method, the facial image data features extracted by the method of this embodiment can greatly improve the recognition accuracy on the basis of the same image recognition technology, with the maximum improvement exceeding 20%.

[0095] Embodiment 2:

[0096] like Figure 2 As shown, the present invention provides a semi-supervised image feature extraction device, comprising:

[0097] The preprocessing module is used to preprocess the original image features and eliminate the influence of the zero space in the image feature space.

[0098] A first construction module is used to construct the similarity graph and the corresponding divergence matrix according to the image data obtained by preprocessing;

[0099] A second construction module is used to construct a semi-supervised image feature extraction model according to the similarity graph and the divergence matrix;

[0100] A conversion module, used for performing equivalent conversion on the semi-supervised image feature extraction model;

[0101] A calculation module, used for calculating the semi-supervised image feature extraction model after equivalent transformation by using an optimization method of alternately iteratively updating the mapping matrix and the auxiliary variables to obtain a mapping matrix for feature extraction;

[0102] The extraction module is used to obtain the extraction result of the image features according to the image data and the mapping matrix.

[0103] Embodiment 3:

[0104] The present invention also provides a storage medium, which stores machine executable instructions. When the machine executable instructions are called and executed by a processor, the machine executable instructions prompt the processor to implement the semi-supervised image feature extraction method.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semi-supervised image feature extraction method, characterized in that: The following steps are involved: Step S1, constructing a similarity graph and a divergence matrix according to the original features of the image data; Step S2: constructing a semi-supervised image feature extraction model according to the similarity graph and the divergence matrix; Step S3, performing equivalent transformation on the semi-supervised image feature extraction model; Step S4, the semi-supervised image feature extraction model after equivalent transformation is calculated by using an optimization method of alternating iterative updates of the mapping matrix and the auxiliary variables to obtain a mapping matrix for feature extraction; Step S5, obtaining an image feature extraction result according to the image data and the mapping matrix; In step S1, the similarity graph is established as follows: Among them, sij represents the image data x i The similarity between sij and image data xj, sij is the element in the i-th row and j-th column in the similarity matrix S, and γ is the regularization parameter; In step S1, the scatter matrix includes: intra-class scatter matrix S w and the inter-class scatter matrix S b ,Right now: Among them, X i represents the set of image data of the i-th category, μ i represents the average value of the image data features of the i-th category, μ is the average value of all labeled image data features, and n i is the number of image data of the i-th category, and C is the total number of categories of image data; In step S2, the semi-supervised image feature extraction model is: Where W is the mapping matrix used for feature extraction, L is the graph Laplacian matrix corresponding to the similarity matrix S, Tr(·) represents the trace operation of the matrix, and α is the regularization coefficient; In step S3, the semi-supervised image feature extraction model is equivalently converted to: Among them, s is an auxiliary variable and can be determined by the extreme point of the quadratic function. The specific calculation method is:

2. A semi-supervised image feature extraction device, characterized in that: include: The first construction module is used to construct a similarity graph and a divergence matrix according to the original features of the image data; A second construction module is used to construct a semi-supervised image feature extraction model according to the similarity graph and the divergence matrix; A conversion module, used for performing equivalent conversion on the semi-supervised image feature extraction model; A calculation module, used for calculating the semi-supervised image feature extraction model after equivalent transformation by using an optimization method of alternately iteratively updating the mapping matrix and the auxiliary variables to obtain a mapping matrix for feature extraction; An extraction module, used for obtaining an extraction result of image features according to the image data and the mapping matrix; The building blocks of the similarity graph are: Among them, sij represents the image data x i The similarity between sij and image data xj, sij is the element in the i-th row and j-th column in the similarity matrix S, and γ is the regularization parameter; The scatter matrix includes: intra-class scatter matrix S w and the inter-class scatter matrix S b ,Right now: Among them, X i represents the set of image data of the i-th category, μ i represents the average value of the features of the image data of the i-th category, μ is the average value of the features of all labeled images, and n i is the number of image data of the i-th category, and C is the total number of categories of image data; The semi-supervised image feature extraction model is equivalently converted to: Among them, s is an auxiliary variable, which is calculated as follows: Tr(·) represents the trace operation of the matrix, and α is the regularization coefficient.

3. A storage medium, characterized in that: The storage medium stores machine executable instructions, and when the machine executable instructions are called and executed by the processor, the machine executable instructions prompt the processor to implement the semi-supervised image feature extraction method as described in claim 1.

Images