Non-negative representation classification method and device based on affine locality sensitivity
By introducing regular terms of local constraint matrix on the objective function of the non-negative representation classifier, the problem of great noise influence in the prior art is solved, and the classification accuracy and robustness are improved.
Patent Information
- Application Number
- CN202510163163.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-14
- Publication Date
- 2025-04-11
AI Technical Summary
In the presence of noise, the classification results are easily affected and it is difficult to effectively suppress noise interference.
By applying regular terms with local constraint matrix on the objective function, local constraints are used instead of sparse constraints, thereby suppressing the influence of noise on classification results.
Improves the classification accuracy and robustness of the non-negative representation classifier, and enhances its performance in noisy environments.
Smart Images
Figure CN119807932A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a field, and in particular to a non-negative representation classification method and device based on affine local sensitivity. Background Art
[0002] In the past decade, a large number of pattern classification algorithms have been designed and applied to tasks such as face recognition and object classification, and have shown their excellent performance in various applications.
[0003] In pattern classification, sparse representation based classifiers (SRC) and collaborative representation based classifiers (CRC) are the most popular for their efficiency and simplicity. SRC solves the l1 norm problem to find a sparse encoding vector to represent the sample to be tested. Although it has achieved great success in face representation, it also raises the question of whether it is the l1 norm sparsity or other reasons that make it work. Therefore, the collaborative representation based classifier (CRC) was proposed, revealing that it is the l2 norm collaboration that really improves the accuracy of face recognition, and in multiple experiments, CRC achieved better classification results than SRC.
[0004] In SRC and CRC, the test samples are approximated using a linear combination of training samples. Although they are successful in pattern classification, SRC / CRC cannot avoid negative encoding vectors because they do not set positive constraints on the encoding vectors of the test samples. Therefore, based on the non-negative matrix factorization (NMF), some scholars have proposed a non-negative representation classifier (NRC), which uses the least squares model of non-negative constraints to restrict the encoding vector to be non-negative. For test samples, the approximate residual of each category is used for classification. Experimental results show that the NRC classifier is very effective and achieves higher accuracy than the SRC / CRC classifier.
[0005] However, NMF also has a disadvantage - it requires both the factor matrix and the weight matrix to have non-negative elements. Therefore, in the presence of noise, even a small amount of noise may have a significant impact on the classification results of the NRC classifier. Summary of the invention
[0006] In order to overcome the above-mentioned defects of the prior art, the present invention provides a non-negative representation classification method and device based on affine local sensitivity, which can improve the classification efficiency of the non-negative representation classifier.
[0007] An embodiment of the present invention provides a non-negative representation classification method based on affine local sensitivity, comprising the following steps:
[0008] Obtaining a control sample set, and preprocessing the control sample set to obtain a control set; wherein the control samples in the control set belong to a number of different sample categories;
[0009] By solving a preset objective function, a coding vector of the sample to be classified on the control set is obtained; wherein the preset objective function includes a local constraint matrix;
[0010] According to the encoding vector, a residual between the sample to be classified and each of the sample categories is calculated, and according to the residual, the sample category corresponding to the sample to be classified is determined.
[0011] Furthermore, the control sample set specifically includes:
[0012] Samples belonging to several different sample categories are collected as the control samples to form the control sample set; wherein the mathematical expression of the control sample set is specifically:
[0013] X=[X1,X2,...X C ]=[x1,x2,...x n ]∈R m×n
[0014]
[0015] Wherein, X is the control sample set, X i is the control sample in the i-th sample category, C is the total number of the sample category, n is the total number of the control samples in each sample category, x is the feature vector of the control sample, and m is the feature dimension of the control sample.
[0016] Furthermore, the preprocessing of the control sample set specifically includes:
[0017] All the control samples are respectively subjected to l2 norm normalization processing, so that the numerical ranges of the control samples after preprocessing remain consistent in each dimension.
[0018] Furthermore, obtaining the encoding vector of the sample to be classified on the control set by solving the preset objective function specifically includes:
[0019] The preset objective function is split and transformed to obtain an equivalent objective function; wherein the preset objective function is specifically:
[0020]
[0021] Wherein, y is the sample to be classified, α is the encoding vector, λ is the preset regularization coefficient, and W is the local constraint matrix;
[0022] The encoding vector is obtained by solving the equivalent objective function.
[0023] Preferably, the splitting and transforming of the preset objective function to obtain an equivalent objective function specifically includes:
[0024] The preset auxiliary variables are introduced, and the preset objective function is split and transformed by the variable splitting method to obtain the equivalent objective function; wherein the equivalent objective function is specifically:
[0025]
[0026] Wherein, z is the preset auxiliary variable.
[0027] Furthermore, obtaining the encoding vector by solving the equivalent objective function specifically includes:
[0028] The augmented Lagrangian function corresponding to the equivalent objective function is calculated, and the augmented Lagrangian function is solved by an alternating direction multiplier method to obtain the encoding vector; wherein the augmented Lagrangian function is specifically:
[0029]
[0030] Among them, l is the preset augmented Lagrange multiplier, μ is the preset penalty parameter,<l,z-α> =trace(l T (z-α)).
[0031] Preferably, the augmented Lagrangian function is solved by an alternating direction multiplier method to obtain the encoding vector, specifically:
[0032] Assume (α t ,z t ) and l t are the optimization variables and Lagrange multipliers obtained at the t-th iteration, and α0=z0=l0=0, then:
[0033] When z,l are fixed, the augmented Lagrangian function can be simplified to:
[0034]
[0035] Then solving the above formula we can get: Where I is the identity matrix;
[0036] When α,l is fixed, the augmented Lagrangian function can be simplified to:
[0037]
[0038] When α,z is fixed, the augmented Lagrangian function can be updated as:
[0039] l t+1 = l t +μ(z t+1 -α t+1 )
[0040] Repeat the iteration until the preset convergence condition is met or the number of iterations exceeds the preset iteration threshold, and then output the α obtained in the last iteration. t is the encoding vector; wherein the preset convergence condition is specifically:
[0041] ‖α t -z t ‖2≤η,‖α t+1 -α t ‖2≤η,‖z t+1 -z t ‖2≤η
[0042] Wherein, η is the preset convergence threshold.
[0043] Furthermore, the calculating the residual between the sample to be classified and each of the sample categories according to the encoding vector specifically includes:
[0044] The calculation formula of the residual is specifically:
[0045] r k =‖yX k α k ‖2
[0046] Among them, r k is the residual between the sample to be classified and the kth sample category, X k is the control sample of the kth category, α k is the encoding sub-vector associated with the k-th class control sample.
[0047] Furthermore, determining the sample category corresponding to the sample to be classified according to the residual specifically includes:
[0048] The samples to be classified are classified according to the residuals, and the sample category corresponding to the minimum residual is determined as the sample category corresponding to the sample to be classified; wherein the specific formula for the classification is:
[0049]
[0050] Another embodiment of the present invention provides a non-negative representation classification device based on affine local sensitivity, comprising: a preprocessing module, a solution module and a classification module;
[0051] The preprocessing module is used to obtain a control sample set and preprocess the control sample set to obtain a control set; wherein the control samples in the control set belong to a number of different sample categories;
[0052] The solution module is used to obtain the encoding vector of the sample to be classified on the control set by solving a preset objective function; wherein the preset objective function includes a local constraint matrix;
[0053] The classification module is used to calculate the residual between the sample to be classified and each of the sample categories according to the encoding vector, and determine the sample category corresponding to the sample to be classified according to the residual.
[0054] Compared with the prior art, the beneficial effects of the present invention are:
[0055] By applying a regularization term with a local constraint matrix to the objective function and using local constraints instead of sparse constraints, the non-negative representation is made more discriminative and sparse, resulting in higher classification accuracy. At the same time, the local constraint term can suppress the impact of noise on the classification results, improving the robustness and classification efficiency of the non-negative representation classifier. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 A flowchart of a non-negative representation classification method based on affine local sensitivity is provided for one embodiment of the present invention.
[0057] Figure 2 A schematic structural diagram of a non-negative representation classification device based on affine local sensitivity is provided in accordance with another embodiment of the present invention. DETAILED DESCRIPTION
[0058] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;
[0059] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0060] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0061] Reference Figure 1, is a flow chart of a non-negative representation classification method based on affine local sensitivity provided by an embodiment of the present invention, comprising the following steps:
[0062] S1: obtaining a control sample set, and preprocessing the control sample set to obtain a control set; wherein the control samples in the control set belong to a number of different sample categories;
[0063] S2: Obtaining the encoding vector of the sample to be classified on the control set by solving a preset objective function; wherein the preset objective function includes a local constraint matrix;
[0064] S3: Calculate the residual between the sample to be classified and each of the sample categories according to the encoding vector, and determine the sample category corresponding to the sample to be classified according to the residual.
[0065] For step S1, specifically, the control sample set is:
[0066] Samples belonging to several different sample categories are collected as the control samples to form the control sample set; wherein the mathematical expression of the control sample set is specifically:
[0067] X=[X1,X2,...X C ]=[x1,x2,...x n ]∈R m×n
[0068]
[0069] Wherein, X is the control sample set, X i is the control sample in the i-th sample category, C is the total number of the sample category, n is the total number of the control samples in each sample category, x is the feature vector of the control sample, and m is the feature dimension of the control sample.
[0070] Furthermore, the preprocessing of the control sample set specifically includes:
[0071] All the control samples are respectively subjected to l2 norm normalization processing, so that the numerical ranges of the control samples after preprocessing remain consistent in each dimension.
[0072] In a preferred embodiment, before classifying the sample to be classified, it is necessary to obtain a control sample set including C sample categories and n control samples as a control reference. After obtaining the control sample set, it is also necessary to perform normalization processing on all the control samples in the control sample set, dividing each control sample by its l2 norm, so that the numerical range of the control sample in each dimension remains consistent.
[0073] For step S2, specifically, obtaining the encoding vector of the sample to be classified on the control set by solving the preset objective function specifically includes:
[0074] The preset objective function is split and transformed to obtain an equivalent objective function; wherein the preset objective function is specifically:
[0075]
[0076] Wherein, y is the sample to be classified, α is the encoding vector, λ is the preset regularization coefficient, and W is the local constraint matrix;
[0077] The encoding vector is obtained by solving the equivalent objective function.
[0078] Preferably, the splitting and transforming of the preset objective function to obtain an equivalent objective function specifically includes:
[0079] The preset auxiliary variables are introduced, and the preset objective function is split and transformed by the variable splitting method to obtain the equivalent objective function; wherein the equivalent objective function is specifically:
[0080]
[0081] Wherein, z is the preset auxiliary variable.
[0082] In a preferred embodiment, the preset objective function is actually the objective function corresponding to the local affine locally sensitive non-negative representation classifier (ALNRC) provided in the embodiment of the present invention. It can be seen that when λ=0, ALNRC degenerates into NRC classifier, so NRC is a special case of ALNRC.
[0083] For representation-based pattern classification tasks, the key issue is how to obtain discriminant coding coefficients on training samples, which can be used to approximate test samples from isomorphic training samples. Given a test sample y and a training sample matrix X, the discriminant coding coefficient of y on X should be positive for training samples from the same class as y, and should be zero for training samples from different classes. However, the classifier based on non-negative representation (NRC) is easily interfered by noise due to the NMF problem. Therefore, the present invention applies a regularization term with a local constraint matrix to its objective function and uses local constraints instead of sparse constraints, which can effectively suppress noise interference and significantly enhance the classification performance of the algorithm.
[0084] Since the preset objective function belongs to a non-negative least squares problem and has no analytical solution, the variable splitting method is used to solve the objective function of this method. By introducing an auxiliary variable z, the preset objective function can be converted into the equivalent objective function containing two variables (α, z), which is convenient for subsequent solution.
[0085] Furthermore, obtaining the encoding vector by solving the equivalent objective function specifically includes:
[0086] The augmented Lagrangian function corresponding to the equivalent objective function is calculated, and the augmented Lagrangian function is solved by an alternating direction multiplier method to obtain the encoding vector; wherein the augmented Lagrangian function is specifically:
[0087]
[0088] Among them, l is the preset augmented Lagrange multiplier, μ is the preset penalty parameter,<l,z-α> =trace(l T (z-α)).
[0089] Preferably, the augmented Lagrangian function is solved by an alternating direction multiplier method to obtain the encoding vector, specifically:
[0090] Assume (α t ,z t ) and l t are the optimization variables and Lagrange multipliers obtained at the t-th iteration, and α0=z0=l0=0, then:
[0091] When z,l are fixed, the augmented Lagrangian function can be simplified to:
[0092]
[0093] Then solving the above formula we can get: Where I is the identity matrix;
[0094] When α,l is fixed, the augmented Lagrangian function can be simplified to:
[0095]
[0096] When α,z is fixed, the augmented Lagrangian function can be updated as:
[0097] l t+1 = l t +μ(z t+1 -α t+1 )
[0098] Repeat the iteration until the preset convergence condition is met or the number of iterations exceeds the preset iteration threshold, and then output the α obtained in the last iteration.t is the encoding vector; wherein the preset convergence condition is specifically:
[0099] ‖α t -z t ‖2≤η,‖α t+1 -α t ‖2≤η,‖z t+1 -z t ‖2≤η
[0100] Wherein, η is the preset convergence threshold.
[0101] In a preferred embodiment, the equivalent objective function can be solved by using the Alternating Direction Method of Multipliers (ADMM). At this time, the augmented Lagrangian function corresponding to the equivalent objective function can be expressed as:
[0102]
[0103] Among them, l is the preset augmented Lagrange multiplier, μ>0 is the preset penalty parameter,<l,z-α> =trace(l T (z-α)).
[0104] After obtaining the augmented Lagrangian function, it can be solved by iterating α and z. First, the relevant variables are initialized: α0 = z0 = l0 = 0, μ> 0 and selected appropriately. t ,z t ) and l t represents the optimization variable and Lagrange multiplier when the number of iterations is t (t = 0, 1, 2, ... T), and T is the preset iteration threshold.
[0105] Then, each parameter is solved by fixing the other two parameters. When z, l are fixed, the augmented Lagrangian function can be converted into a standard least squares problem, and then the analytical solution of α can be obtained. When α, l are fixed, the simplified augmented Lagrangian function has no specific analytical solution. The specific solution can be referred to the public paper "A New Simplex Sparse Learning Model to Measure Data Similarity for Clustering" by Jin Huang et al.
[0106] Repeat the iteration α t ,z t and l t , until the preset convergence condition is met or the number of iterations exceeds T, the α obtained in the last iteration can be determinedt is the encoding vector corresponding to the sample to be classified.
[0107] Since the least squares objective function, linear equality, non-negative constraints and local constraints are all strict convex optimization problems, solving the equivalent objective function through ADMM can ensure convergence to the global optimal solution.
[0108] For step S3, specifically, calculating the residual between the sample to be classified and each sample category according to the encoding vector specifically includes:
[0109] The calculation formula of the residual is specifically:
[0110] r k =‖yX k α k ‖2
[0111] Among them, r k is the residual between the sample to be classified and the kth sample category, X k is the control sample of the kth category, α k is the encoding sub-vector associated with the k-th class control sample.
[0112] Furthermore, determining the sample category corresponding to the sample to be classified according to the residual specifically includes:
[0113] The samples to be classified are classified according to the residuals, and the sample category corresponding to the minimum residual is determined as the sample category corresponding to the sample to be classified; wherein the specific formula for the classification is:
[0114]
[0115] In a preferred embodiment, in order to better illustrate the performance of the ALNRC classifier, the preferred embodiment further performs classification on the AR dataset and the Aircraft dataset. The AR dataset has more than 4,000 color images corresponding to 126 faces (70 males and 56 females), all of which have different facial expressions, lighting conditions, and occlusions.
[0116] In this preferred embodiment, a data subset consisting of 50 men and 50 women is selected, and 14 images with lighting changes and expression changes are selected for each person, 7 of which are used for training and the remaining 7 are used for testing. All images are cropped to a size of 60×43 and normalized to have an l2 norm standard, and then projected into a subspace of dimensions 54, 120, and 300.
[0117] In both 54 and 300 dimensions, ALNRC achieved the best classification results, 86.1% and 94.0% respectively, which were 0.9% and 0.7% higher than NRC's 85.2% and 93.3% respectively. In 120 dimensions, ALNRC and NRC had the same classification results, both 91.3%.
[0118] The Aircraft dataset contains images of 100 different aircraft model variants, with 100 images for each aircraft model, totaling 10,000 images. The aircraft appear in different scales, design structures, and postures, making this dataset very difficult to visually classify.
[0119] In this preferred embodiment, 6667 images are used as training samples and the remaining 3333 images are used as test samples. Finally, ALNRC achieves an accuracy of 87.6% on this dataset, which is 0.3% higher than NRC's 87.3%. In addition, compared with traditional classifiers such as SRC and CRC, the classification accuracy of ALNRC is significantly improved on these two datasets.
[0120] It can be seen that, in most cases, the affine local sensitive non-negative representation classifier (ALNRC) adopted in the preferred embodiment has higher classification accuracy than the traditional NRC classifier.
[0121] In summary, the non-negative representation classification method based on affine local sensitivity proposed in an embodiment of the present invention applies a regularization term with a local constraint matrix on the objective function of NRC, uses local constraints instead of sparse constraints, makes the non-negative representation more discriminative and sparse, suppresses the interference of noise on the classification results, and effectively improves the classification accuracy and efficiency of the non-negative representation classifier.
[0122] Reference Figure 2 , is a schematic diagram of the structure of a non-negative representation classification device based on affine local sensitivity provided by another embodiment of the present invention, comprising: a preprocessing module 101, a solving module 102 and a classification module 103;
[0123] The preprocessing module 101 is used to obtain a control sample set and preprocess the control sample set to obtain a control set; wherein the control samples in the control set belong to a number of different sample categories;
[0124] The solving module 102 is used to obtain the encoding vector of the sample to be classified on the control set by solving a preset objective function; wherein the preset objective function includes a local constraint matrix;
[0125] The classification module 103 is used to calculate the residual between the sample to be classified and each of the sample categories according to the encoding vector, and determine the sample category corresponding to the sample to be classified according to the residual.
[0126] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A non-negative representation classification method based on affine local sensitivity, characterized in that: The steps include: Obtaining a control sample set, and preprocessing the control sample set to obtain a control set; wherein the control samples in the control set belong to a number of different sample categories; By solving a preset objective function, a coding vector of the sample to be classified on the control set is obtained; wherein the preset objective function includes a local constraint matrix; According to the encoding vector, a residual between the sample to be classified and each of the sample categories is calculated, and according to the residual, the sample category corresponding to the sample to be classified is determined.
2. The non-negative representation classification method based on affine local sensitivity as claimed in claim 1, characterized in that: The control sample set is specifically: Samples belonging to several different sample categories are collected as the control samples to form the control sample set; wherein the mathematical expression of the control sample set is specifically: X=[X1,X2,...X C ]=[x1,x2,...x n ]∈R m×n Wherein, X is the control sample set, X i is the control sample in the i-th sample category, C is the total number of the sample category, n is the total number of the control samples in each sample category, x is the feature vector of the control sample, and m is the feature dimension of the control sample.
3. The non-negative representation classification method based on affine local sensitivity as claimed in claim 1, characterized in that: The preprocessing of the control sample set specifically includes: All the control samples are respectively subjected to l2 norm normalization processing, so that the numerical ranges of the control samples after preprocessing remain consistent in each dimension.
4. The non-negative representation classification method based on affine local sensitivity as claimed in claim 1, characterized in that: The step of obtaining the encoding vector of the sample to be classified on the control set by solving the preset objective function specifically includes: The preset objective function is split and transformed to obtain an equivalent objective function; wherein the preset objective function is specifically: Wherein, y is the sample to be classified, α is the encoding vector, λ is the preset regularization coefficient, and W is the local constraint matrix; The encoding vector is obtained by solving the equivalent objective function.
5. The non-negative representation classification method based on affine local sensitivity as claimed in claim 4, characterized in that: The splitting and transforming of the preset objective function to obtain an equivalent objective function specifically includes: The preset auxiliary variables are introduced, and the preset objective function is split and transformed by the variable splitting method to obtain the equivalent objective function; wherein the equivalent objective function is specifically: Wherein, z is the preset auxiliary variable.
6. The non-negative representation classification method based on affine local sensitivity as claimed in claim 5, characterized in that: The obtaining of the encoding vector by solving the equivalent objective function specifically includes: The augmented Lagrangian function corresponding to the equivalent objective function is calculated, and the augmented Lagrangian function is solved by an alternating direction multiplier method to obtain the encoding vector; wherein the augmented Lagrangian function is specifically: Among them, l is the preset augmented Lagrange multiplier, μ is the preset penalty parameter,<l,z-α> =trace(l T (z-α)).
7. The non-negative representation classification method based on affine local sensitivity as claimed in claim 6, characterized in that: The augmented Lagrangian function is solved by the alternating direction multiplier method to obtain the encoding vector, which is specifically: Assume (α t ,z t ) and l t are the optimization variables and Lagrange multipliers obtained at the t-th iteration, and α0=z0=l0=0, then: When z,l are fixed, the augmented Lagrangian function can be simplified to: Then solving the above formula we can get: Where I is the identity matrix; When α,l is fixed, the augmented Lagrangian function can be simplified to: When α,z is fixed, the augmented Lagrangian function can be updated as: l t+1 =l t +μ(z t+1 -a t+1 ) Repeat the iteration until the preset convergence condition is met or the number of iterations exceeds the preset iteration threshold, and then output the α obtained in the last iteration. t is the encoding vector; wherein the preset convergence condition is specifically: ‖α t -With t ‖2≤η,‖α t+1 -α t ‖2≤η,‖z t+1 -With t ‖2≤η Wherein, η is the preset convergence threshold.
8. The non-negative representation classification method based on affine local sensitivity as claimed in claim 1, characterized in that: The calculating, according to the encoding vector, the residual between the sample to be classified and each of the sample categories specifically includes: The calculation formula of the residual is specifically: r k =‖yX k a k ‖2 Among them, r k is the residual between the sample to be classified and the kth sample category, X k is the control sample of the kth category, α k is the encoding sub-vector associated with the k-th class control sample.
9. The non-negative representation classification method based on affine local sensitivity as claimed in claim 8, characterized in that: Determining the sample category corresponding to the sample to be classified according to the residual specifically includes: The samples to be classified are classified according to the residuals, and the sample category corresponding to the minimum residual is determined as the sample category corresponding to the sample to be classified; wherein the specific formula for the classification is:
10. A non-negative representation classification device based on affine local sensitivity, characterized in that: include: Preprocessing module, solution module and classification module; The preprocessing module is used to obtain a control sample set and preprocess the control sample set to obtain a control set; wherein the control samples in the control set belong to a number of different sample categories; The solution module is used to obtain the encoding vector of the sample to be classified on the control set by solving a preset objective function; wherein the preset objective function includes a local constraint matrix; The classification module is used to calculate the residual between the sample to be classified and each of the sample categories according to the encoding vector, and determine the sample category corresponding to the sample to be classified according to the residual.