Pattern classification method and system based on local sensitive competition matrix regression
By introducing a locally sensitive competition matrix regression method in pattern classification, using competition representation terms and local constraint regular terms, the problems of low classification accuracy and unstable algorithm in the existing technology are solved, and higher classification accuracy and stability are achieved , especially on Extended Yale B and AR datasets outperform traditional methods.
Patent Information
- Application Number
- CN202411570612.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-06
- Publication Date
- 2024-12-06
AI Technical Summary
The existing pattern classification methods have problems such as low classification accuracy and unstable algorithms, and have failed to effectively utilize the category information of the training samples and the local structural information between samples.
Using a method based on locally sensitive competition matrix regression, by solving the objective function of LSCMR, competition representation terms and local constraint regular terms are introduced, the representation coefficients and reconstruction errors of the test samples on the training sample set are calculated, and classification is performed. Specific steps include unitization processing, variable splitting method, alternating direction multiplication method and iterative solution of augmented Lagrangian function.
The accuracy of pattern classification and the robustness of the algorithm are improved, and the resulting representation coefficients are more discriminant and sparse, suppress noise interference, and significantly improve classification accuracy.
Smart Images

Figure CN119089324A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of pattern classification, and in particular to a pattern classification method and system based on local sensitive competition matrix regression. Background Art
[0002] In recent years, regression-based methods have attracted widespread attention due to their simplicity, efficiency, and strong interpretability, and have been successfully applied in fields such as face recognition, image processing, and visual tracking.
[0003] In pattern classification, regression-based methods assume that the test sample can be approximately represented by a linear combination of all training samples and classify the test sample based on the minimum reconstruction error. Sparse representation based classification (SRC) aims to use the sparse linear combination of training samples to represent the test sample. Sparsity is achieved by The collaborative representation based classification (CRC) believes that the real reason for improving recognition performance lies in the collaborative mechanism between samples of different categories, rather than The sparsity of the norm. Therefore, CRC will The norm regularization term is replaced by The norm regularization term significantly reduces the computational complexity while achieving recognition performance comparable to SRC.
[0004] It should be pointed out that SRC / CRC belongs to a one-dimensional regression model, that is, the representation error is measured in vector form. Their common underlying assumption is that pixel-by-pixel errors are independent. However, the true distribution of representation errors in actual situations is more complicated. Experiments have found that the errors generated by continuous noise have a low-rank or approximately low-rank structure. In order to maintain the two-dimensional structural information of the error image, nuclear norm based matrix regression (NMR) imposes nuclear norm constraints on the error image, which has good robustness to structured noise and has achieved excellent performance in the field of face recognition. However, NMR has two problems: one is that it does not use the category information of the training samples, and the other is that it ignores the local structural information between samples. These two types of information play an important role in pattern classification, promoting the obtained representation coefficients to be more sparse and discriminative, which is beneficial to pattern classification. Summary of the invention
[0005] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.
[0006] In view of the above-mentioned problems, the present invention is proposed.
[0007] Therefore, the technical problem solved by the present invention is that the existing classification methods have the problems of low classification accuracy, unstable algorithms, and limited patterns that can be used for classification.
[0008] In order to solve the above technical problems, the present invention provides the following technical solutions: a pattern classification method based on local sensitive competitive matrix regression, comprising: obtaining training samples for a test object and obtaining test sample data; performing unitization processing on the training and test sample data; calculating the representation coefficient of the test sample on the training sample set, and calculating the reconstruction error of the test sample on each category of training samples; classifying according to the reconstruction error, and the test sample is identified as the category corresponding to the minimum residual;
[0009] The representation coefficient includes the representation coefficient of the test sample on the training sample set obtained by solving the objective function of LSCMR; the objective function of LSCMR is: (1)
[0010] Where y is the test sample; x is the representation coefficient of the test sample y on the training sample D; is an operator that transforms a vector into a matrix; and are two regularization parameters; ; It is a test sample and training samples The Euclidean distance between is the Hadamard product; T represents the transpose.
[0011] The variable splitting method is used to solve the objective function, and the auxiliary variable e is introduced to rewrite the objective function into the following optimization problem: (2)
[0012] The alternating direction multiplier method is used to solve the problem, and the augmented Lagrangian function is expressed as: (3)
[0013] where z is the augmented Lagrange multiplier, >0 is the penalty parameter.
[0014] As a preferred solution of the pattern classification method based on local sensitive competition matrix regression described in the present invention, wherein: the training samples include, and the training sample set is expressed as: (4)
[0015] in, is the number of sample categories; For the i-th category training sample data; i=1, 2,…, c; R is a set of real numbers; is the dimension of each sample; is the total number of training samples, ;d j is the feature vector data of the jth sample j=1, 2,…, n.
[0016] As a preferred solution of the pattern classification method based on local sensitive competitive matrix regression described in the present invention, the unitization processing includes dividing the data vector of each sample by its own length to obtain a vector with a length of 1.
[0017] As a preferred solution of the pattern classification method based on local sensitive competitive matrix regression described in the present invention, wherein: solving by alternately iterating e and x, initializing and setting the relevant variables: For variables The initial value at the beginning of the algorithm iteration, For variables The initial value of And choose appropriately; respectively and Iteration is Optimization variables and Lagrange multipliers when ;
[0018] Update e: Fix x and z, and the objective function simplified by the augmented Lagrangian function is expressed as: (5)
[0019] The simplified objective function is equivalent to the optimization problem: (6)
[0020] in is the Frobenius norm of the matrix, and the closed-form solution of formula (6) is: (7)
[0021] in, is the singular value contraction operator;
[0022] The optimal solution of the simplified objective function is:
[0023] in, ; is an operator that transforms a matrix into a vector. It's Mat The inverse operation of
[0024] Update x: Fix e and z, and the objective function simplified to formula (3) is: (8)
[0025] Optimize x and construct a diagonal matrix , H i The elements corresponding to the i-th category are all 1, and the rest are 0; , formula (8) can be expressed as: (9)
[0026] Taking the partial derivative of formula (9) with respect to x and setting it to 0, the closed-form solution is: (10)
[0027] in, ;
[0028] Update z: Update the augmented Lagrange multiplier z, (11)
[0029] By repeating the iterative process until the convergence condition is met;
[0030] The convergence condition of the iterative process is and At the same time, t and t denote the primal residual and the dual residual respectively, (12)
[0031] in, and is the preset stopping criterion parameter.
[0032] As a preferred solution of the pattern classification method based on local sensitive competitive matrix regression described in the present invention, the reconstruction error of the test sample on each category training sample is calculated as:
[0033] (13)
[0034] Among them, D i is the i-th type of training sample, x i is the coefficient sub-vector corresponding to the i-th type of training samples.
[0035] As a preferred solution of the pattern classification method based on local sensitive competitive matrix regression described in the present invention, wherein: classification is performed according to the reconstruction error, and the test sample is identified as the category corresponding to the minimum residual: .
[0036] A pattern classification system based on local sensitive competitive matrix regression is used to execute the above method, and the pattern classification system includes: a sample acquisition module, which obtains training samples for test objects, obtains test sample data, and transmits the obtained samples to an analysis module; a recognition module, which performs unit processing on training and test sample data; calculates the representation coefficient of the test sample on the training sample set, and calculates the reconstruction error of the test sample on each category of training samples; classifies according to the reconstruction error, the test sample is identified as the category corresponding to the minimum residual, and the recognition result is transmitted to an output module; an output module, which outputs the recognition result uploaded by the recognition module.
[0037] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of any one of the methods of the present invention when executing the computer program.
[0038] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any one of the methods of the present invention.
[0039] Beneficial effects of the present invention: In the pattern classification method based on local sensitive competitive matrix regression provided by the present invention, the category information of the training samples and the local structural information between samples play an important role in pattern classification. By introducing the competitive representation term and the local constraint regularization term in the objective function of NMR, the representation coefficient obtained by LSCMR is more discriminative and sparse, which brings higher classification accuracy. The local constraint term of the present invention can suppress the influence of noise on the classification results and improve the robustness of the algorithm. The present invention has a good classification effect on the test samples, and achieves a high classification accuracy on the Extended Yale B dataset and the AR dataset, which is better than many current classifiers and can be widely used in pattern classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:
[0041] Figure 1 An overall flow chart of a pattern classification method based on local sensitive competition matrix regression provided by the first embodiment of the present invention;
[0042] Figure 2A schematic diagram of using the alternating direction method of multipliers (ADMM) to solve the LSCMR objective function in a pattern classification method based on local sensitive competitive matrix regression provided by the first embodiment of the present invention. DETAILED DESCRIPTION
[0043] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.
[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0045] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0046] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.
[0047] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0048] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0049] Example 1, reference Figure 1-Figure 2 , as an embodiment of the present invention, provides a pattern classification method based on local sensitive competition matrix regression, comprising:
[0050] S1: Obtain training sample and test sample data.
[0051] Furthermore, the training sample set is expressed as: (4)
[0052] in, is the number of sample categories; For the i-th category training sample data; i=1, 2,…, c; R is a set of real numbers; is the dimension of each sample; is the total number of training samples, ;d j is the feature vector data of the jth sample j=1, 2,…, n.
[0053] It is important to know that when selecting samples, the category of the test sample to be identified (classified) must appear in the training sample. The test sample can be a collected image (face image, fingerprint image, iris image, etc.), or it can be data formed after feature extraction from the image. And the sample does not need to meet specific characteristics, only the feature dimension of the training and test samples needs to be the same.
[0054] S2: Unitize the training and test sample data.
[0055] Furthermore, the unitization process includes dividing the data vector of each sample by its length (also called modulus) to obtain a vector of length 1. The purpose of unitization is to constrain the size of the data to the range of [0,1] to avoid data that is too large or too small, making the optimization process of the algorithm more stable.
[0056] S3: Solve the representation coefficient of the test sample on the training sample set; calculate the reconstruction error of the test sample on each category of training samples.
[0057] Furthermore, the representation coefficient of the test sample on the training sample set is obtained by solving the objective function of LSCMR; Figure 2Schematic diagram of using alternating direction method of multipliers (ADMM) to solve the LSCMR objective function. The objective function of LSCMR is: (1)
[0058] Where y is the test sample; x is the representation coefficient of the test sample y on the training sample D; is an operator that transforms a vector into a matrix; and are two regularization parameters; ; It is a test sample and training samples The Euclidean distance between is the Hadamard product; T represents the transpose.
[0059] Furthermore, since the objective function of the LSCMR method has no closed-form solution, the variable splitting method is used to solve the objective function of the method. In order to optimize formula (2), the auxiliary variable e is first introduced, and formula (2) is rewritten as the following optimization problem: (2)
[0060] Furthermore, formula (3) can be solved by the alternating direction multiplier method, and the augmented Lagrangian function can be expressed as follows: (3)
[0061] where z is the augmented Lagrange multiplier, >0 is the penalty parameter.
[0062] Furthermore, formula (4) can be solved by alternately iterating e and x to initialize the relevant variables: For variables The initial value at the beginning of the algorithm iteration, For variables The initial value of And choose appropriately; respectively and Iteration is The optimization variables and Lagrange multipliers when ; the detailed solution process is as follows:
[0063] Update e: Fix x and z, and the objective function simplified by the augmented Lagrangian function is expressed as: (5)
[0064] Formula (5) is equivalent to the following optimization problem: (6)
[0065] in, is the Frobenius norm of the matrix, and the closed-form solution of formula (6) is: (7)
[0066] in, is the singular value contraction operator; therefore, the optimal solution of formula (5) is:
[0067] in, ; is an operator that transforms a matrix into a vector. It's Mat The reverse operation of .
[0068] Update x: Fix e and z, and the objective function simplified from formula (4) is as follows: (8)
[0069] To optimize x, construct a diagonal matrix , H i The elements corresponding to the i-th category are all 1, and the rest are 0; , formula (8) can be expressed as: (9)
[0070] Taking the partial derivative of formula (9) with respect to x and setting it to 0, the closed-form solution is: (10)
[0071] in, ;
[0072] Update z: Update the augmented Lagrange multiplier z: (11)
[0073] Repeat the above iterative steps until the convergence condition is met. The convergence condition of the above iterative algorithm is and At the same time, t and t They represent the original residual and the dual residual respectively, and their definitions are as follows: (12)
[0074] in, and is the preset stopping criterion parameter.
[0075] Furthermore, the reconstruction error of the test sample on each category of training samples is calculated as: (13)
[0076] Among them, D i is the i-th type of training sample, x i is the coefficient sub-vector corresponding to the i-th type of training samples.
[0077] S4: Classification is performed based on the reconstruction error, and the test sample is identified as the category corresponding to the minimum residual error: .
[0078] A pattern classification system based on local sensitive competitive matrix regression includes: a sample acquisition module, which obtains training samples for a test object, obtains test sample data, and transmits the obtained samples to an analysis module; a recognition module, which performs unit processing on the training and test sample data; calculates the representation coefficient of the test sample on the training sample set, and calculates the reconstruction error of the test sample on each category of training samples; classifies according to the reconstruction error, the test sample is identified as the category corresponding to the minimum residual, and the recognition result is transmitted to an output module; and an output module outputs the recognition result uploaded by the recognition module.
[0079] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory, tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory, magnetic memory, ferroelectric memory, phase change memory, graphene memory, etc. Volatile memory may include random access memory or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory or dynamic random access memory, etc. The database involved in the embodiments provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto.
[0080] The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but is not limited thereto.
[0081] Example 2: The following is an embodiment of the present invention, which provides a pattern classification method based on local sensitive competition matrix regression. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.
[0082] In order to better illustrate the performance of the LSCMR classification method, this embodiment further performs classification on the Extended Yale B dataset and the AR dataset. The Extended Yale B dataset contains 2414 face images of 38 people, each person has 59-64 images, and these images have different lighting changes. According to the size of the lighting change, the entire dataset is divided into 5 subsets, namely subset1 to subset5, and all images are scaled to 48×42 pixels. Use subset1 as the training set, and subset4 and subset5 as test sets, respectively. On subset4 and subset5, LSCMR achieved the best classification results, which were 97.37% and 62.18% respectively, which were 9.21% and 20.02% higher than NMR's 88.16% and 42.16%, respectively. The AR face database contains more than 4,000 face images of 126 people (70 men and 56 women). The 26 images of each experimenter were taken in two different stages (two weeks apart). These images have lighting, expression changes, sunglasses and scarf occlusion. In each stage, each person has 8 unobstructed images, 3 sunglasses occlusion images and 3 scarf occlusion images. All images are cropped and scaled to 50×40 pixels. The first 4 images of each person in the two stages are selected as the training set, and the sunglasses occlusion (Sunglasses) and scarf occlusion (Scarf) images in the two stages are used as the test set. In the end, LSCMR achieved 97.08% and 71.81% correct recognition rates in the Sunglasses and Scarf cases, respectively, which are 1.94% and 7.37% higher than NMR's 95.14% and 64.44%, respectively. Compared with traditional classifiers such as SRC and CRC, the classification accuracy of LSCMR has been greatly improved on these two datasets. It can be seen that in most cases, the local sensitive competitive matrix regression method (LSCMR) of this embodiment has higher classification accuracy.
[0083] Furthermore, the calculation of the correct recognition rate is, for example, now we have collected photos of 3 people, 2 photos of each person, recorded as 1 1 2 2 3 3; if these 6 photos are input into the algorithm, the prediction result given is 1 2 1 2 2 3, that is, there are a total of 6 test samples, and 3 are recognized correctly, then the correct recognition rate is 3 / 6*100%=50%.
[0084] In summary, the present invention proposes a classification method based on local sensitive competitive matrix regression (LSCMR), specifically, introducing competitive representation terms and local constraint regularization terms into the objective function of NMR, so that the representation coefficients obtained by LSCMR are more discriminative and sparse, suppressing the interference of noise on the classification results, and effectively improving the classification accuracy of the algorithm.
[0085] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A pattern classification method based on local sensitive competitive matrix regression, characterized in that: include: Obtain training samples for the test object and obtain test sample data; Unitize the training and test sample data; Calculate the representation coefficient of the test sample on the training sample set, and calculate the reconstruction error of the test sample on each category of training samples; Classification is performed based on the reconstruction error, and the test sample is identified as the category corresponding to the minimum residual; The representation coefficient includes the representation coefficient of the test sample on the training sample set obtained by solving the objective function of LSCMR; the objective function of LSCMR is: (1) Where y is the test sample; x is the representation coefficient of the test sample y on the training sample D; is an operator that transforms a vector into a matrix; and are two regularization parameters; ; It is a test sample and training samples The Euclidean distance between is the Hadamard product; T represents transpose; The variable splitting method is used to solve the objective function, and the auxiliary variable e is introduced to rewrite the objective function into the following optimization problem: (2) The alternating direction multiplier method is used to solve the problem, and the augmented Lagrangian function is expressed as: (3) where z is the augmented Lagrange multiplier, >0 is the penalty parameter.
2. The pattern classification method based on local sensitive competitive matrix regression as claimed in claim 1, characterized in that: The training samples include: (4) in, is the number of sample categories; For the i-th category training sample data; i=1, 2,…, c; R is a set of real numbers; is the dimension of each sample; is the total number of training samples, ;d j is the feature vector data of the jth sample j=1, 2,…, n.
3. The pattern classification method based on local sensitive competitive matrix regression as claimed in claim 2, characterized in that: The unitization process includes dividing the data vector of each sample by its own length to obtain a vector with a length of 1.
4. The pattern classification method based on local sensitive competitive matrix regression as claimed in claim 3, characterized in that: Solve by alternating e and x and initialize the relevant variables: For variables The initial value at the beginning of the algorithm iteration, For variables The initial value of And choose appropriately; respectively and Iteration is Optimization variables and Lagrange multipliers when ; Update e: Fix x and z, and the objective function simplified by the augmented Lagrangian function is expressed as: (5) The simplified objective function is equivalent to the optimization problem: (6) in is the Frobenius norm of the matrix, and the closed-form solution of formula (6) is: (7) in, is the singular value contraction operator; The optimal solution of the simplified objective function is: in, ; is an operator that transforms a matrix into a vector. It's Mat The inverse operation of Update x: Fix e and z, and the objective function simplified to formula (4) is: (8) Optimize x and construct a diagonal matrix , H i The elements corresponding to the i-th category are all 1, and the rest are 0; , formula (8) can be expressed as: (9) Taking the partial derivative of formula (9) with respect to x and setting it to 0, the closed-form solution is: (10) in, ; Update z: Update the augmented Lagrange multiplier z, (11) By repeating the iterative process until the convergence condition is met; The convergence condition of the iterative process is and At the same time, t and t denote the primal residual and the dual residual respectively, (12) in, and is the preset stopping criterion parameter.
5. The pattern classification method based on local sensitive competitive matrix regression as claimed in claim 4, characterized in that: The reconstruction error of the test sample on each category of training samples is calculated as: (13) Among them, D i is the i-th type of training sample, x i is the coefficient sub-vector corresponding to the i-th type of training samples.
6. The pattern classification method based on local sensitive competitive matrix regression as claimed in claim 5, characterized in that: According to the classification of the reconstruction error, the test sample is identified as the category corresponding to the minimum residual: .
7. A pattern classification system based on local sensitive competitive matrix regression, characterized in that: For executing the method according to any one of claims 1 to 6, the pattern classification system comprises: The sample acquisition module acquires training samples for the test object, acquires test sample data, and transmits the acquired samples to the analysis module; The recognition module performs unit processing on the training and test sample data; calculates the representation coefficient of the test sample on the training sample set, and calculates the reconstruction error of the test sample on each category of training samples; classifies according to the reconstruction error, the test sample is identified as the category corresponding to the minimum residual, and the recognition result is transmitted to the output module; The output module outputs the recognition results uploaded by the recognition module.
8. A computer device comprising: Memory and processor; The memory stores a computer program, wherein the processor implements the steps of any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
