Probability Collaborative Representation Image Set Classification Method Based on Prior Knowledge

Abstract

The present invention discloses a probability collaborative representation image set classification method based on prior knowledge to solve the problem of the failure of probability representation caused by insufficient learning of inter-class differences in collaborative representation in image set classification. In the algorithm model, the image set is regarded as a point in the manifold space. Through a representation method based on the distance between samples, a probability collaborative representation method of the image set model in the space of symmetric positive definite matrices is derived. And by learning the representation samples, the prior knowledge of the samples is extracted, and the probability representation of the model is re-weighted. Finally, the corresponding closed-form solution is derived. The algorithm is compared on different data sets. Experiments show that the model has very excellent classification performance on unbalanced data sets, pays more attention to sample features, and solves the problem of difficult classification caused by inter-class ambiguity in the collaborative representation method.

Description

Technical Field

[0001] The present invention relates to a method for classifying an image set by probability collaborative representation, in particular to a method for classifying an image set by probability collaborative representation based on prior knowledge. Background Art

[0002] With the rapid development of current data transmission technology and storage technology, images and videos have become very common content presentation forms, and at the same time, it has brought about the application of image processing in the field of pattern recognition. In image processing tasks and machine learning, the classification method based on image sets has also attracted wide attention in the industrial and academic circles due to its multi-view expression ability. Compared with the recognition of single images, image sets can better highlight the relationships between samples, thus achieving better classification. However, the intra-class diversity and inter-class ambiguity of image set samples are still very important research directions.

[0003] In classification tasks, sparse representation is a very effective method. It can quickly and accurately extract features from samples and can accurately classify them, and this is also the case in the field of image classification. The Sparse Representation Classification (SRC) algorithm is a very effective classification method, which is widely used in common classification scenarios and is constantly improved. SRC determines sample classification with the minimum coding loss by performing sparse coding on test samples based on dataset samples. And the l1 regularization norm is used to normalize the sparse coding matrix. However, later some scholars proposed and proved that the effectiveness of SRC is largely attributed to the property of collaborative representation of a test sample by training samples of all classes. Subsequently, they proposed the Collaborative Representation Classifier (CRC) by using the l2 norm. The SRC / CRC classifier can be regarded as a distance-based classifier because the classification principle is based on the distance between the query sample and the representation samples of each class. Many improved iterative versions of SRC / CRC have been applied to face recognition and other visual recognition tasks. Inspired by CRC, the Image Set-based Collaborative Representation Classification (ISCRC) algorithm was proposed. It extends CRC to the task of image set classification and has achieved very good results. However, ISCRC is based on the Euclidean space, which will reduce its algorithm performance and bring a very large computational overhead, and ISCRC is only applicable to face recognition. In the past decade or so, linear subspace statistical modeling has been widely used in video-based image set classification problems. In these methods, image frames in the video are extracted and organized into an image set, and then the image set is reduced-dimensional represented by principal component analysis to obtain a linear subspace, so as to be able to model the image change patterns in the video. Such a processing method can stably handle some complex within-class variations (such as facial expressions, clothing, etc.) in unconstrained video-based image set classification. Although these methods have greatly promoted the application of subspaces in the field of image set classification, due to the unique non-Euclidean geometric structure of linear subspaces, how to design recognition algorithms has always been a thorny problem. Recently, a large amount of work has shown that linear subspaces with the same dimension can be regarded as a sample point on a specific manifold, and such a manifold is called the Grassmann Manifold. However, the characteristics of the non-Euclidean space make it impossible to apply many excellent pattern recognition techniques in the Euclidean space well on it. Therefore, some scholars proposed to represent the basic elements on the Grassmann Manifold as corresponding projection matrices by using the Projection Mapping framework. Since the projection distance derived from the projection matrix can approximately represent the true geodesic distance of the Grassmann Manifold, and such a projection distance also satisfies the three important distance metric elements: non-negativity, symmetry, and triangle inequality. Therefore, the Riemannian geometry of the Grassmann Manifold can be estimated by the projection distance, and thus traditional algorithms in the Euclidean space can be extended to the Grassmann Manifold.For example, among the methods based on sparse coding representation, the Grassmann sparse coding representation method (GSC) studies sparse coding and dictionary learning problems by embedding Grassmann manifolds in the symmetric matrix space. Due to its high complexity, some scholars were inspired by the probabilistic collaborative representation (ProCRC) and proposed the probabilistic collaborative representation based image set classifier (PGCRC), which gave a closed form explanation for the image set classification task from a probabilistic perspective. However, the collaborative representation based method still does not make full use of the representation differences between classes and it is difficult to mine the diversity information within the sample class. Summary of the invention

[0004] The purpose of the present invention is to provide a probability collaborative representation image set classification method based on prior knowledge, and combine it with a linear discrimination method to improve sample divergence, make the applied prior knowledge more effective, and complete high-accuracy classification of image sets.

[0005] The technical solution to achieve the purpose of the present invention is: a probability collaborative representation image set classification method based on prior knowledge, comprising the following steps:

[0006] Step 1: Obtain the subspace [X1, X2, ..., X N ];

[0007] Step 2: By assuming that the sample points of the image set are located on the Grassmann manifold space, the projection metric method is applied to transform the sample data of the image set [X1, X2, ..., X N ] is transformed into a point [g(X1), g(X2), ..., g(X N )], where g(X) = XX T , N is the number of classes;

[0008] Step 3: For [g(X1), g(X2), ..., g(X N )] Apply linear discriminant representation to obtain the representative samples of the separation distribution, and then use the nearest neighbor algorithm to calculate the distance between each class sample and the centroid of the overall sample, that is, the prior knowledge of each class: β = {β1, ..., β k};

[0009] Step 4: construct a probabilistic collaborative representation algorithm model on the Grassmann manifold, and incorporate the prior knowledge β acquired in step 3; analyze and solve the probabilistic collaborative representation algorithm model to obtain the final representation sparsity coefficient α based on the prior knowledge of the image set;

[0010] Step 5: Obtain the classification result for the test image set through the classification algorithm.

[0011] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the above-mentioned classification method for image sets based on prior knowledge using probabilistic collaborative representation is implemented.

[0012] A computer-readable storage medium stores a computer program, which when executed by a processor implements the above-mentioned classification method for image sets based on prior knowledge using probabilistic collaborative representation.

[0013] Compared with the prior art, the significant advantages of the present invention are as follows: By learning the prior knowledge of the samples, the relationship between each class of samples and the overall class of samples can be fully explored, and the representation ability of probabilistic collaborative representation can be fully utilized. Among them, the training data set is analyzed in detail, and the prior knowledge data most suitable for the data set is learned through the nearest neighbor algorithm. And the prior knowledge is incorporated into the probabilistic collaborative representation framework, which very well solves the problem of differences between samples.

[0014] The present invention will be further described in detail below with reference to the accompanying drawings. Description of the Drawings

[0015] Figure 1 It is a schematic diagram of the operation process of the classification method for image sets based on prior knowledge using probabilistic collaborative representation in the present invention.

[0016] Figure 2 It is a schematic diagram of the classification method for image sets based on prior knowledge using probabilistic collaborative representation in the present invention.

[0017] Figure 3 It is a comparison of the parameter sensitivity performance between the method of the present invention and the Probabilistic Grassmannian Collaborative Representation (PGCRC). The upper and lower parts respectively show the performance of the Probabilistic Grassmannian Collaborative Representation (PGCRC) and the method of the present invention under different parameter settings. The horizontal axis is the parameter λ1 (the value range is 0.5 to 3), and the vertical axis is the parameter λ2 (the value range is 0.1 to 1). Detailed Embodiment

[0018] Combined with Figure 1 , the classification method for image sets based on prior knowledge using probabilistic collaborative representation of the present invention includes the following steps:

[0019] Step 1: Learn the subspace representation of the corresponding image set through a subspace learning algorithm. Training set: [X1, X2,..., X N , test sample: Y.

[0020] Step 2: Assume that the sample points of the image set are located in the Grassmann Manifold space, and apply the projection metric method to project the image set sample data [X1, X2,..., X N, Y is converted into points in the symmetric matrix space [g(X1), g(X2),..., g(X N )], where g(X) = XX T , g(Y) = YY T , and N is the number of categories;

[0021] Step 3: Process the samples in the symmetric space through the linear discriminant representation method (LDA), and calculate the prior knowledge β through the following formula:

[0022]

[0023] where N is the total number of training samples, and n k is the number of samples in each class.

[0024] Step 4: Construct a probabilistic collaborative representation algorithm model on the Grassmann manifold, and incorporate the prior knowledge β learned in Step 3 into it; analyze and solve the probabilistic collaborative representation algorithm model to obtain the final representation sparse coefficient α based on the prior knowledge of the image set;

[0025] 1) Construct a probabilistic collaborative representation model based on prior knowledge to determine the class label probability corresponding to the query sample Y;

[0026]

[0027] where l (Y) is the label value of the sample Y, α is the coefficient of the sparse representation, γ is a preset parameter, and β k refers to the prior knowledge of the k-th class;

[0028] 2) Calculate the collaborative representation coefficient through the following calculation process.

[0029]

[0030] where I is the identity matrix, K is the total number of classes, λ1, are two Lagrange coefficients, and are preset parameters. Custom variables A, Θ, are used for the final formula optimization and solution, and the specific acquisition process will be given in detail below.

[0031] Define the kernel matrix as the kernel matrices between Y and Y, Y and X, and X and X respectively:

[0032]

[0033] Subsequently, based on the singular value decomposition method (SVD), perform the following transformation:

[0034]

[0035] where ∑ and U are the diagonal matrix and symmetric matrix obtained by SVD decomposition respectively, A is a self-defined variable symbol, and the following can be obtained from the above formula For Make the following transformation:

[0036]

[0037] Here, Θ is a self-defined variable parameter, that is

[0038] Define the parameter Z:

[0039]

[0040] where i refers to the index number of the sample in the training set represents the set of all index numbers of the k-th class, and the kernel matrix is obtained:

[0041] Based on the above parameters, the collaborative representation coefficients are finally obtained through the following closed-form solution:

[0042]

[0043] where I is the identity matrix, K is the total number of categories, λ1, are two Lagrange coefficients, which are preset parameters.

[0044] Step 5, finally calculate the probabilities that the test sample belongs to each class according to the following method, and confirm the final classification of the sample:

[0045]

[0046] The following further describes the present invention in detail with an example:

[0047] The present invention mainly takes image set data as input, implements a probabilistic collaborative representation algorithm on the Grassmann manifold, and finally completes the classification of the image set through a classifier.

[0048] The general process of the implementation example of the present invention is as Figure 2 shown.

[0049] (1) The example is the test implemented by the present invention on the dataset ETH-80. This dataset is a classic benchmark dataset in image set classification and can be easily obtained from the network. The dataset contains 8 different categories, and each category contains 10 objects. Each object is regarded as an image set, and each image set contains 41 images captured from different perspectives. For each class, five groups are randomly selected as the training set, and the remaining five groups are used as the test set.

[0050] (2) For each training and test image cluster in ETH-80, a subspace is learned and mapped into the symmetric matrix space. The corresponding parameters A, Θ, are obtained according to the corresponding formula in the specific implementation requirements.

[0051] (3) Take the data samples in the symmetric matrix space and map the data into a space with a separable data distribution through linear discriminant analysis. On this basis, the internal relationship of each class of data in ETH-80 relative to the overall class of data, that is, the prior knowledge β of the data set, is calculated by implementing the calculation method in step 3.

[0052] (4) Implement step 4 to obtain the representation coefficients of the test image set ETH-80 as the training set dictionary, and calculate the classification probability of the test image set through the classification method in step 5, and finally obtain the classification result l(Y).

[0053] (5) Experimental results

[0054] The invention compares multiple algorithms, mainly including the following categories:

[0055] 1) Based on linear subspaces:

[0056] Mutual Subspace Method (MSM)

[0057] Constrained Mutual Subspace Method (CMSM)

[0058] Discriminant Canonical Correlation Method (DCC)

[0059] 2) Based on non-linear manifolds:

[0060] Manifold Discriminant Analysis (MDA)

[0061] Grassmann Discriminant Analysis (GDA)

[0062] Grassmann Graph Embedding Discriminant Analysis (GEDA)

[0063] Projection Metric Learning (PML)

[0064] 3) Based on affine subspaces:

[0065] Affine Hull-based Image Set Distance (AHISD)

[0066] Convex Hull-based Image Set Distance (CHISD)

[0067] 4) Based on sparse coding:

[0068] Regularized Nearest Point (RNP)

[0069] Sparse Approximate Nearest Point (SANP)

[0070] Image Set Classification Based on Image Sets (ISCRC)

[0071] 5) Sparse coding based on Riemannian manifold.

[0072] Grassmann Sparse Coding (GSC)

[0073] Grassmann Probabilistic Collaborative Representation (PGCRC)

[0074] Method Accuracy rate (%) MSM 86.00 CMSM 88.25 DCC 88.25 MDA 89.50 GDA 91.00 GEDA 92.50 PML 93.75 AHISD 70.00 CHISD 73.50 RNP 72.50 SANP 69.75 ISCRC 70.00 GSC 95.00 PGCRC 97.00 GMPKPCRC (This method) 98.10

[0075] The comparison results are shown in the above table. It can be seen that the method of this patent has achieved the best performance, and is 1.1 percentage points higher than the best method. Among them, the Euclidean space sparse coding methods (such as RNP, SANP, ISCRC) are not as effective as the linear subspace-based methods (such as MSM, CMSM) in the scenario of image set recognition. Among the non-linear manifold-based methods (GDA, GEDA, PML), they show very superior performance compared with the other methods, which can prove that the Grassmann manifold is beneficial to image set classification. Observation Figure 3 It can be seen that under different values of the collaborative representation parameters λ1 and λ2, this method has better stability and classification performance.

[0076] Since this patent proposes to extract prior knowledge from the original training samples, without affecting the classification performance of collaborative representation, this method has achieved the highest accuracy, optimized the algorithm of collaborative representation classification, and thus achieved a good image set classification effect, which has important promotion significance for the field of image set classification and video analysis.

Claims

1. A probability collaborative representation image set classification method based on prior knowledge, characterized by the following steps: Step 1. Obtain the subspace of the image set collection [X1, X2, …, X N by the subspace method; Step 2: By assuming that the sample points of the image set lie on the Grassmann manifold space and applying the projection metric method, the image set sample data [X1, X2, …, X N is converted into points [g(X1), g(X2), …, g(X N )] on the symmetric matrix space, where g(X) = XX T , and N is the number of classes; Step 3. Apply linear discriminant representation to [g(X1), g(X2), …, g(X N )] to obtain representation samples with separated distributions, and then use the nearest neighbor algorithm to calculate the distance between each class sample and the centroid of the overall samples, that is, the prior knowledge of each class: β = {β1, ……, β k}; Step 4: Construct a probability collaborative representation algorithm model on the Grassmann manifold, incorporating the prior knowledge β learned in Step 3; analyze and solve the probability collaborative representation algorithm model to obtain the sparse representation coefficient α of the image set prior knowledge finally; the specific content of Step 4 is as follows: 1) For the image set samples in the symmetric space generated in Step 2, construct a probability collaborative representation model based on prior knowledge to determine the class label probability of the query sample Y. where l (Y) is the label value of sample Y, α is the sparse representation coefficient, γ is a preset parameter, and β k refers to the prior knowledge of the k-th class; 2) Define the kernel matrix as the kernel matrices between Y and Y, Y and X, and X and X respectively: Subsequently, based on the singular value decomposition method, perform the following transformation: where ∑ and U are the diagonal matrix and the symmetric matrix obtained from the SVD decomposition respectively, Λ is a custom variable symbol; obtain the parameter For K(X, Y), perform the following transformation: Obtain parameters Define the parameter Z: Here, i refers to the index number of the sample in the training set Denote the set of all index numbers of the k-th class, and obtain 3) Based on the above-obtained parameters Θ, Λ, β k , the sparse representation coefficient α is solved through the following closed-form solution formula: where I is the identity matrix, K is the total number of categories, λ1 and λ2 are two Lagrange coefficients, and is a preset parameter; Step 5: Obtain the classification result for the test image set through a classification algorithm.

2. The probability collaborative representation image set classification method based on prior knowledge according to claim 1, characterized in that: The specific content of Step 3 is as follows: Map the training data to the image set, process the image set through linear discriminant analysis, separately calculate the centroid of the image set samples of each category and the centroid of all samples, and obtain k prior knowledge coefficients by calculating the distance between the centroid of the overall samples and the centroid of the samples of each category; the calculation method is as follows: where n k is the number of samples in each class.

3. The probability collaborative representation image set classification method based on prior knowledge according to claim 1, characterized in that: In Step 5, the minimum reconstruction error of sparse representation is used to determine the category of the query sample. The specific method is as follows: By multiplying the samples on the symmetric space of the k-th class by the sparse representation coefficient α solved in step 4 k a reconstructed sample is obtained. The collaborative representation error of the query sample corresponding to the k-th class sample is obtained by subtracting the reconstructed sample from the query sample; the class label with the minimum error is selected as the final classification label of the query sample.

4. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the probability collaborative representation image set classification method based on prior knowledge as described in any one of claims 1-3.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the probability collaborative representation image set classification method based on prior knowledge as described in any one of claims 1-3.

Images