Improved Incremental Linear Discriminant Analysis Feature Extraction Method

Abstract

The present invention proposes an improved incremental linear discriminant analysis feature extraction method. First, calculate the initial OWA global mean, between-class scatter matrix, and within-class scatter matrix of the image training samples to generate an initial model. In subsequent steps, for each newly added sample, dynamically update the global mean variable, and adjust the between-class scatter matrix and within-class scatter matrix based on the updated global mean. Subsequently, use the Fisher discriminant criterion to calculate a new projection matrix, and project the test image onto this new projection matrix to achieve data dimensionality reduction and feature extraction. The core innovation of the present invention lies in the adoption of the OWA operator, which generates a new global sample mean according to the different contributions of features between samples. In addition, an improved between-class scatter matrix is proposed, which significantly enhances the difference between different classes. By improving the class discrimination, this method can more comprehensively retain data information and provide a higher-quality feature representation for subsequent classification tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of data dimensionality reduction, and particularly to an improved incremental linear discriminant analysis feature extraction method. Background Art

[0002] Linear Discriminant Analysis (LDA) is an important supervised dimensionality reduction tool, which is widely used in fields such as clinical medicine, education, aerospace, etc. LDA processes data by finding the optimal projection matrix to gather data of the same class together and separate data of different classes as much as possible. To improve the performance of the LDA algorithm, researchers have continuously optimized the LDA algorithm to make up for the limitations of feature extraction and projection methods. Since the Fisher method requires the constructed divergence matrix to satisfy non-singularity, when the divergence matrix is non-singular and there is an undersampling problem where the data dimension is much larger than the sample size, the LDA algorithm cannot be directly applied to the undersampling problem where the data dimension is much larger than the sample size. Therefore, researchers have continuously explored the problem of LDA dimension being larger than the sample size.

[0003] When the classical LDA algorithm calculates the class spacing using the arithmetic mean, the same weight is assigned to the distances between different classes, thus unable to meet the scenario where there are differences in the distances between different classes. Therefore, researchers have proposed LDA algorithms for different application scenarios. For example, Zhu et al. proposed the Neighborhood LDA (NLDA) algorithm to solve the problem that when a class contains subclasses, the internal structure of the class cannot be correctly described. The NLDA algorithm regards the neighborhood as the smallest subclass, enabling the optimal projection direction to be found without any clustering algorithm. Zheng et al. developed the Harmonic Mean LDA (HMLDA) algorithm to calculate the class spacing, improving the performance of the LDA algorithm. Li et al. defined a new kernel function constraint (IIKC) to assist LDA in extracting more features and further improving the difference between nearby data points. For feature extraction in different application scenarios, researchers have proposed a feature extraction method combining PCA and LAD. For example, Tong applied the PCA-LDA algorithm to analyze SERS spectra to judge liver cancer, Rao et al. used the PCA-LDA algorithm to solve the small sample problem (SSS), and Dilip et al. used PCA-LDA for feature extraction of diabetes datasets to improve the diagnostic accuracy.

[0004] When the amount of data grows dynamically and in real time, the LDA algorithm cannot meet the requirements of real-time feature extraction. For this reason, researchers have proposed the incremental LDA algorithm and continuously improved it. Among them, Pang et al. designed the Incremental LDA (ILDA) algorithm based on the incremental idea, solved the situation where the complete training sample set was not given in advance, and applied it to the scenario of real-time feature extraction. It has become an important tool in education, image processing, etc. However, ILDA does not consider the case where there are differences in class spacing. To handle the difficulty of the inverse of the within-class scatter matrix in the ILDA algorithm, Zhao et al. developed the GSVD-ILDA algorithm from the perspective of generalized singular value decomposition to find the projection matrix in the entire space. Chu et al. developed the ILDA / QR algorithm based on QR of matrix decomposition to improve the classification accuracy and computational complexity efficiency of ILDA. Through analysis, it is found that the existing LDA and ILDA algorithms assign the same weight to different class spacings during feature extraction. However, in real life, larger class spacings will affect the accuracy of feature extraction.

[0005] In summary, the existing LDA and ILDA algorithms do not consider the differences in class spacing during the feature extraction process. However, in the data collected in real life, there are often differences in class spacing, and larger class spacings will affect the accuracy of feature extraction. Summary of the Invention

[0006] In view of the above situation, the main purpose of the present invention is to propose an improved incremental linear discriminant analysis feature extraction method to solve the above technical problems.

[0007] The present invention proposes an improved incremental linear discriminant analysis feature extraction method, and the method includes the following steps:

[0008] Step 1: Obtain image training samples of different categories and convert the image training samples into a data matrix;

[0009] Step 2: Calculate the global sample mean of the data matrix based on the OWA operator, calculate the class means of the image training samples of different categories, and calculate the between-class scatter matrix of the data matrix according to the global sample mean;

[0010] Calculate the within-class scatter matrix according to the class means;

[0011] Step 3: When a new image training sample is input, update the global sample mean to obtain a new global sample mean;

[0012] Update the between-class scatter matrix and the within-class scatter matrix respectively according to the new global sample mean and the class mean of the new image training sample for incremental learning to obtain the between-class scatter matrix after incremental learning and the within-class scatter matrix after incremental learning;

[0013] Step 4: According to the within-class scatter matrix after incremental learning and the between-class scatter matrix after incremental learning, or the within-class scatter matrix and the between-class scatter matrix, use the Fisher discriminant criterion to obtain a new projection matrix;

[0014] Step 5: Obtain test image samples, project the test image samples onto the new projection matrix, and thus achieve data dimensionality reduction and feature extraction.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] Classic linear discriminant analysis minimizes the within-class distance and maximizes the between-class distance. In the process of calculating the between-class scatter matrix, the mean value is solved by arithmetic mean. However, the between-class distance obtained by arithmetic mean has great limitations. This method assumes that each feature makes the same contribution, so that all between-class distances are given the same weight, but the contributions of different features are often not the same. According to different feature contributions, a new global sample mean is obtained based on the OWA operator, and a new between-class scatter matrix is proposed. The new between-class scatter matrix makes the differences between different categories larger, and more comprehensively retains data information by improving the category discrimination.

[0017] The additional aspects and advantages of the present invention will be partially given in the following description, partially become obvious from the following description, or be understood through the embodiments of the present invention. Description of the Drawings

[0018] Figure 1 It is a change diagram of non-decreasing proportional fuzzy linguistic quantifiers;

[0019] Figure 2 It is a comparison diagram of the algorithm recognition rates of the present invention and the prior art on the AR dataset;

[0020] Figure 3 It is a comparison diagram of the algorithm recognition rates of the present invention and the prior art on the YALE dataset;

[0021] Figure 4 It is an average recognition rate diagram of the present invention and the prior art on the AR dataset;

[0022] Figure 5 It is an average recognition rate diagram of the present invention and the prior art on the YALE dataset;

[0023] Figure 6 It is an average recognition rate diagram of the present invention and the prior art on the ORL dataset;

[0024] Figure 7 It is an average recognition rate diagram of the present invention and the prior art on the FERET dataset;

[0025] Figure 8This is the average recognition rate graph of the present invention and the prior art on the MNIST dataset;

[0026] Figure 9 This is the result graph of visualizing the PIE dataset by the ILDA algorithm in a two-dimensional subspace;

[0027] Figure 10 This is the result graph of visualizing the PIE dataset by Example 2 in a two-dimensional subspace;

[0028] Figure 11 This is the result graph of visualizing the IRIS dataset by the ILDA algorithm in a two-dimensional subspace;

[0029] Figure 12 This is the result graph of visualizing the IRIS dataset by Example 2 in a two-dimensional subspace. Detailed implementation manners

[0030] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present invention and should not be construed as limiting the present invention.

[0031] Referring to the following description and drawings, these and other aspects of the embodiments of the present invention will be clear. In these descriptions and drawings, some specific implementation manners in the embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the embodiments of the present invention. However, it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0032] Example 1

[0033] This embodiment provides an improved incremental linear discriminant analysis feature extraction method, and the method includes the following steps:

[0034] Step 1, obtain image training samples of different categories and convert the image training samples into a data matrix;

[0035] Step 2, calculate the global sample mean of the data matrix based on the OWA operator, and calculate the class means of the image training samples of different categories, and calculate the between-class scatter matrix of the data matrix according to the global sample mean;

[0036] Calculate the within-class scatter matrix according to the class means;

[0037] In the said Step 2, the following relational expression exists in the process of calculating the global sample mean of the data matrix based on the OWA operator:

[0038] ;

[0039] Among them, represents the global sample mean, represents the N th training sample, represents the total number of training samples, represents the j th weight of the training sample, represents the j th largest value in the training sample;

[0040] ;

[0041] Among them, represents a non-decreasing fuzzy quantifier function, used to represent the distribution of non-decreasing proportional fuzzy linguistic quantifiers in fuzzy logic; represents the number of parameter vectors.

[0042] The following relationship exists in the process corresponding to the non-decreasing fuzzy quantifier function:

[0043] ;

[0044] Among them, represents the input variable, represents the lower bound of the increasing interval, represents the upper bound of the increasing interval.

[0045] In the said step 2, the following relationship exists in the process of calculating the between-class scatter matrix of the data matrix based on the global sample mean:

[0046] ;

[0047] Among them, represents the between-class scatter matrix, represents the number of categories, represents the j th number of samples in the th class, j represents the mean of the th class samples, represents the transpose operation, represents the set of real numbers.

[0048] In the said step 2, the following relationship exists in the process of calculating the within-class scatter matrix based on the class means:

[0049] ;

[0050] Among them, represents the within-class scatter matrix, represents the sample vector belonging to the j th class, Represents the newly input training sample.

[0051] Step 3: According to the within-class scatter matrix and the between-class scatter matrix, use the Fisher discriminant criterion to obtain a new projection matrix;

[0052] In the said Step 3, the method of using the Fisher discriminant criterion to obtain a new projection matrix according to the between-class scatter matrix and the within-class scatter matrix specifically includes the following steps:

[0053] Calculate the generalized eigenvalue matrix by using the between-class scatter matrix and the within-class scatter matrix;

[0054] Perform eigenvalue decomposition on the generalized eigenvalue matrix to obtain a diagonal matrix containing several eigenvalues and an eigenvector matrix composed of eigenvectors corresponding to the eigenvalues;

[0055] Sort the several eigenvalues in descending order, and select the eigenvectors corresponding to the first several eigenvalues to form a new projection matrix.

[0056] Step 4: Obtain the test image samples, project the test image samples on the new projection matrix, and thus realize the dimensionality reduction and feature extraction of the data.

[0057] Embodiment 2

[0058] This embodiment provides an improved incremental linear discriminant analysis feature extraction method, and the method includes the following steps:

[0059] Step 1: Obtain the image training samples of different categories, and convert the image training samples into a data matrix;

[0060] Step 2: Calculate the global sample mean of the data matrix based on the OWA operator, and calculate the class means of the image training samples of different categories. Calculate the between-class scatter matrix of the data matrix according to the global sample mean;

[0061] Calculate the within-class scatter matrix according to the class means;

[0062] In the said Step 2, the following relationship exists in the process of calculating the global sample mean of the data matrix based on the OWA operator:

[0063] ;

[0064] Wherein, Represents the global sample mean, Represents the N th training sample, Represents the total number of training samples, Represents the j th weight of the training sample, Represents thej the maximum value;

[0065] ;

[0066] where represents a non - decreasing fuzzy quantifier function, used to represent the distribution of non - decreasing proportional fuzzy linguistic quantifiers in fuzzy logic; represents the number of parameter vectors.

[0067] The following relationship exists in the process corresponding to the non - decreasing fuzzy quantifier function:

[0068] ;

[0069] where represents the input variable, represents the lower bound of the increasing interval, represents the upper bound of the increasing interval.

[0070] In step 2, the following relationship exists in the process of calculating the between - class scatter matrix of the data matrix based on the global sample mean:

[0071] ;

[0072] where represents the between - class scatter matrix, represents the number of classes, represents the j number of samples in the th class, j represents the mean of the samples in the th class, represents the transpose operation,

[0073] In step 2, the following relationship exists in the process of calculating the within - class scatter matrix based on the class mean:

[0074] ;

[0075] where represents the within - class scatter matrix, represents the sample vector belonging to the j th class, represents the newly input training sample.

[0076] Step 3: When a new image training sample is passed in, update the global sample mean to obtain a new global sample mean;

[0077] Update the between-class scatter matrix and the within-class scatter matrix respectively according to the new global sample mean and the class means of the new image training samples, and obtain the trained between-class scatter matrix and the trained within-class scatter matrix;

[0078] In step 3, when updating the global sample mean to obtain the new global sample mean, the following relationship exists:

[0079] ;

[0080] where, represents the new global sample mean.

[0081] In step 3, when updating the between-class scatter matrix according to the new global sample mean and the class means of the new image training samples, the following relationship exists:

[0082] ;

[0083] where, represents the updated between-class scatter matrix, represents the class sample mean of the updated i th class.

[0084] For the calculation process of the class sample mean of the updated i th class, the following relationship exists:

[0085] ;

[0086] where, represents the number of samples in the i th class, represents the mean of the samples in the i th class.

[0087] In step 3, when updating the within-class scatter matrix according to the new global sample mean and the class means of the new image training samples, the following relationship exists:

[0088] ;

[0089] where, represents the updated within-class scatter matrix.

[0090] Step 4, According to the within-class scatter matrix after incremental learning and the between-class scatter matrix after incremental learning, use the Fisher discriminant criterion to obtain a new projection matrix;

[0091] In step 4, according to the between-class scatter matrix after incremental learning and the within-class scatter matrix after incremental learning, the method for obtaining a new projection matrix using the Fisher discriminant criterion specifically includes the following steps:

[0092] Calculate the generalized eigenvalue matrix using the between-class scatter matrix after incremental learning and the within-class scatter matrix after incremental learning;

[0093] Perform eigenvalue decomposition on the generalized eigenvalue matrix to obtain a diagonal matrix containing several eigenvalues and an eigenvector matrix composed of eigenvectors corresponding to the eigenvalues;

[0094] Sort the several eigenvalues in descending order, and select the eigenvectors corresponding to the first several eigenvalues to form a new projection matrix.

[0095] Step 5: Obtain test image samples, project the test image samples onto the new projection matrix, and thus achieve data dimensionality reduction and feature extraction.

[0096] In summary, the present invention proposes an improved incremental linear discriminant analysis feature extraction method. First, calculate the initial OWA global mean, between-class scatter matrix, and within-class scatter matrix of the image training samples to generate an initial model. In subsequent steps, for each newly added sample, dynamically update the global mean variable, and adjust the between-class scatter matrix and within-class scatter matrix based on the updated global mean. Subsequently, calculate a new projection matrix using the Fisher discriminant criterion, and project the test image onto the new projection matrix, thereby achieving data dimensionality reduction and feature extraction. The core innovation of the present invention lies in using the OWA operator to generate a new global sample mean according to the different contributions of the features between samples. In addition, an improved between-class scatter matrix is proposed, which significantly enhances the difference between different classes. By improving the class discrimination, this method can more comprehensively retain data information and provide a higher-quality feature representation for subsequent classification tasks.

[0097] To more detailedly elaborate the specific process and achieved effects of the present invention, assume is a data matrix of l training samples with N classes, is the number of samples in j class, is the new global sample mean, is the mean of the samples in the j th class. Then the between-class scatter matrix based on OWA is shown in formula (7).

[0098] (7)

[0099] The new global sample mean is for the sample set The OWA global sample mean of the sample set X can be obtained according to formula (8). , in formula (8), is the j th largest value in

[0100] (8)

[0101] In formula (8), how to obtain effective weights to calculate the global mean of the sample set is a key issue. Since the OWA operator is often used to implement the concept of the majority of the model, especially in the case of aggregation guided by quantifiers, fuzzy quantifiers are used to calculate the weights of each element. In order to reasonably and accurately display the characteristics of different categories of samples in the sample set, the present invention adjusts the weights of the OWA operator through relatively decreasing quantifiers, so as to achieve the flexibility of weighted average. The representation of relatively decreasing quantifiers and the calculation of weights are shown in formula (9).

[0102] (9)

[0103] In formula (9), n represents the number of parameter vectors. Among them, the calculation of the non-decreasing proportional fuzzy linguistic quantifier Q is shown in formula (10).

[0104] (10)

[0105] In formula (10), . In this embodiment, a = 0.3 and b = 0.7. The change of the non-decreasing proportional fuzzy linguistic quantifier Q is as Figure 1 shown

[0106] Figure 1 shows how the quantifier decreases as the input parameter decreases. Starting from 1, it linearly decreases to 0. The obtained weight matrix shows superior discriminant ability for the following three main reasons.

[0107] First of all, the decreasing quantifier allows the decision maker to emphasize the key factors with higher weights and assign lower weights to unimportant factors. In this way, both the main factors can be fully considered and the secondary factors will not be overemphasized, so that the decision result can better reflect the true preferences and goals of the decision maker. Secondly, the linear weight decrease provides a consistent and predictable weight distribution, which helps to establish clear priorities among factors. Finally, although the weight only contains two elements, the method still has flexibility. By reducing the weight to 0 and a fixed constant value, the decision is simplified and the adaptability to different scenarios is maintained. Among them, the fixed constant value can be adjusted to adapt to different preferences and needs.

[0108] Generalized eigenvalue matrix SIt is obtained by multiplying the inverse of the within-class scatter matrix and the between-class scatter matrix, as shown in Equation (11). It contains the within-class and between-class information of the data. Through the generalized eigenvalue matrix S to find the optimal projection direction, so that samples of different classes can be better separated after projection, realizing data dimensionality reduction and classification.

[0109] (11)

[0110] Among them, represents the generalized eigenvalue matrix, represents the inverse of the within-class scatter matrix;

[0111] The within-class scatter matrix is used to measure the difference between samples within the same class. The within-class scatter matrix can be calculated by the following formula.

[0112] (12)

[0113] Among them, represents the sample vector belonging to the j th class.

[0114] In the solution proposed in Embodiment 1 of the present invention, hereinafter denoted as the OWA-LDA algorithm, the calculated is substituted into Equation (11) to obtain a new generalized eigenvalue matrix . Perform eigen-decomposition on : . Among them P is the eigenvector matrix composed of eigenvectors , is the diagonal matrix composed of eigenvalues . After sorting the eigenvalues in descending order, the first r largest eigenvalues are selected to form a new diagonal matrix , and the corresponding eigenvector matrix is also the new projection matrix . Finally, project the data set onto the new projection matrix to realize data dimensionality reduction and feature extraction.

[0115] ILDA is an incremental learning algorithm based on LDA. It can gradually accept new data and update the discriminant space, so it has higher computational efficiency in large-scale datasets and can adapt to new data. Generally speaking, ILDA has the ability of incremental learning compared with LDA, can gradually accept new data and update the model, has higher computational efficiency and lower memory occupancy. This makes ILDA more advantageous in scenarios of processing large-scale datasets and real-time updating of models. However, due to the characteristics of incremental learning, the ILDA algorithm cannot guarantee the discrimination between different data categories. To solve this problem and improve the stability and feature extraction efficiency of ILDA, based on the OWA-LDA algorithm, i.e., Embodiment 1, the present invention proposes Embodiment 2 with the ability of incremental learning, hereinafter referred to as the OWA-ILDA algorithm.

[0116] Based on the OWA-LDA algorithm, first calculate the initial OWA global mean , between-class scatter matrix and within-class scatter matrix . Then, during the incremental learning process, for each newly incoming sample x , use the formula to update the global mean variable, then update the between-class scatter matrix and within-class scatter matrix, and finally use the Fisher discriminant criterion to obtain the new projection matrix. The specific process of the OWA-ILDA algorithm proposed in Embodiment 2 of the present invention is as follows.

[0117] Algorithm 1: OWA-ILDA Algorithm Process

[0118] Input: Initial training sample set ;

[0119] Initialization:

[0120] 1. Calculate the weight W using formula (9);

[0121] 2. Calculate the OWA global sample mean using formula (8);

[0122] 3. Calculate the between-class scatter matrix , within-class scatter matrix using formulas (7) and (12);

[0123] When a new training sample x is input:

[0124] 4. Update the global sample mean :

[0125] ;

[0126] 5. Update the between-class scatter matrix ;

[0127] If x is a training sample of a new category:

[0128] x The affiliated label y is not found in the category set X corresponding to the initial training sample set Y ;

[0129] ;

[0130] Otherwise:

[0131] ;

[0132] is the mean of the category samples of the updated i th class, ;

[0133] 6. Update the within-class scatter matrix :

[0134] ;

[0135] End the loop.

[0136] Use the Fisher discriminant criterion to compress the high-dimensional face image discriminant matrix into a low-dimensional feature space matrix:

[0137] 7. Calculate the new generalized feature matrix using formula (11) ;

[0138] 8. Perform eigen-decomposition on to obtain:

[0139] ;

[0140] ;

[0141] and are the eigenvalues and corresponding eigenvectors obtained by decomposing the generalized eigenvalue matrix respectively;

[0142] 9. After sorting the eigenvalues from largest to smallest, select the first r largest eigenvectors and eigenvalues, to obtain:

[0143] ;

[0144] ;

[0145] is the optimal projection direction, is the eigenvalue corresponding to the projection direction;

[0146] 10. Multiply the new sample dataset by to project the data into a low-dimensional space;

[0147] Output: the projection vector of the new sample set x To verify the effectiveness of the present invention, 7 datasets were used for experiments. Among them, the AR, YALE, ORL, FERET, and MNIST datasets were used for recognition experiments, and the IRIS and PIE datasets were used to study the influence of the OWA operator on class discrimination and intra-class aggregation in two-dimensional subspaces.

[0148] The AR dataset contains more than 4000 color facial images of 126 people (70 males and 56 females). For the images taken of the same person, there are different occlusions, lighting conditions, and facial expressions. The original images are 576*768 pixels. For the convenience of experiments, in this embodiment, only the complete facial images of 100 people (50 males and 50 females) are selected, with a total of 1400 pictures, and each person has 14 different images. And all facial images are uniformly cropped to 50*40 pixels.

[0149] The YALE dataset contains 165 grayscale face images of 15 people. Each person has 11 photos taken under different conditions, including different facial expressions and different lighting conditions. The image size is 100*100 pixels. All are cropped to 100*80 pixels in the experiment.

[0150] The ORL dataset contains 400 face images of 40 people, and these images contain some changes in poses, expressions, and facial organs. Each participant provided 10 normalized grayscale images, and the image size is 92*112, with a black background.

[0151] The FERET dataset was created by the FERET project Error! Reference source not found. It contains 14051 face images with multiple poses and different lighting conditions. Photos of the same person in the dataset have different expressions, poses, lighting, and age changes, and most of the people are Westerners. For the convenience of experiments, the images of 200 people are selected, and all images are uniformly cropped to 80*80 pixels.

[0152]

[0153] IRIS dataset: The IRIS dataset is a widely used classification dataset, collected and organized by Fisher. It contains 150 data samples, divided into 3 categories, with 50 samples in each category. Each sample consists of four attributes.​

[0154] PIE dataset: The PIE dataset is a face database from the Robotics Institute of Carnegie Mellon University. It includes images of 68 individuals, with each person having 10 images with different poses, lighting conditions, and expressions.

[0155] MNIST dataset: The MNIST dataset is derived from the original NIST database. This dataset includes 60,000 training images and 10,000 test images sampled from the same distribution. Each grayscale digit is normalized and centered within a 28 × 28 pixel image, and the centroid of the intensity is located at the center of the image.

[0156] These datasets provide comprehensive facial image samples, capturing different positions, expressions, and lighting conditions. They provide a solid foundation for evaluating and validating the performance of the proposed algorithms and are crucial for research in areas such as attention detection, face recognition, and expression analysis. To make the results persuasive, each database is preprocessed first, and then one-third of the samples within each category are randomly selected as the training set, and two-thirds of the samples are used as the test set. All algorithms are run ten times (changing the random samples each time) to calculate the average accuracy. All experiments are run on a PC equipped with an Intel Core I5-12400F CPU and a GTX4060Ti GPU.

[0157] To compare and analyze the performance of the proposed method, the following steps are taken: First, the proposed algorithm and the original algorithm are used to extract the feature vectors from the face images. Then, a centroid classifier is used to judge the labels of these feature vectors, and the Manhattan distance is used as the distance metric for the centroid in the judgment process. This method can effectively measure the similarity between different samples, thus enabling more accurate classification. Finally, to comprehensively evaluate the feature extraction efficiency and performance of different algorithms in face recognition, the recognition rate is used as the core evaluation index. The recognition rate, that is, the proportion of correctly classified samples, is shown in formula (13). It can intuitively reflect the accuracy and reliability of the algorithm in the face recognition task. By comparing the recognition rates of different algorithms, the advantages and disadvantages of various algorithms in feature extraction and classification can be clearly seen, providing a strong basis for subsequent research and improvement.

[0158] (13)

[0159] In formula (13), TP represents True Positive, that is, the number of samples correctly classified as positive examples; TN represents True Negative, that is, the number of samples correctly classified as negative examples; FP represents False Positive, that is, the number of samples misclassified as positive examples; FN represents False Negative, that is, the number of samples misclassified as negative examples.

[0160] In the experiment, the AR, YALE, ORL, FERET, and MNIST image datasets were used to verify the improvement of the LDA and ILDA algorithms. To prove the superiority of OWA-ILDA, the centroid classifier was used to classify the images, and the superiority of the algorithms was determined by comparing the recognition accuracy results of eight algorithms: LDA, PCA, NLDA, HMLDA, ILDA, OWA-NLDA, OWA-LDA, and OWA-ILDA.

[0161] In this experiment, the training and test samples need to be set for the AR, YALE, ORL, FERET, and MNIST datasets, as shown in Table 1.

[0162] Table 1 Division of the datasets

[0163]

[0164] After 10 rounds of iterative calculations using PCA, LDA, ILDA, NLDA, HMLDA, OWA-LDA, OWA-NLDA, and OWA-ILDA, the average recognition rates of each algorithm are compared in Table 2.

[0165] Table 2 Average recognition rates of the algorithms on the datasets (%)

[0166]

[0167] As shown in Table 2, the OWA-ILDA algorithm has the highest average recognition rate, and the recognition rates on the AR, YALE, ORL, FERET, and MNIST datasets are 90.22%, 87.62%, 88.94%, 84.76%, and 83.87% respectively. In addition, compared with the original algorithms, the algorithms using the OWA operator have obvious improvements.

[0168] To statistically verify the performance advantages of OWA-ILDA, we conducted a Friedman test analysis on the results shown in Table 2. Table 3 gives the Friedman test statistic, its corresponding p-value, and the critical value of OWA-ILDA. In Table 3, Greater than the critical value, the p-value of the algorithm accuracy is less than the significance level α = 0.05. This means that there is sufficient evidence in the observed data to reject the null hypothesis, indicating that there is a significant difference between OWA-ILDA and other algorithms.

[0169] Table 3 Friedman test results F F ( = 8, = 5).

[0170]

[0171] Note: "k" represents the number of algorithms being compared; "N" represents the number of data sets

[0172] To more intuitively illustrate the superiority of the algorithms incorporating the OWA operator, each algorithm was compared with its corresponding algorithm after adding the OWA operator. The 10-run results of the average recognition rate of each algorithm on the AR and YALE data sets are as Figure 2 and Figure 3 shown.

[0173] It can be seen from the experimental comparison graph that after making method decisions by adding the OWA operator, the average recognition rate of the algorithms has significantly improved, and this result is universal. This verifies that the OWA operator can make the divergence matrix have better performance, and through the divergence matrix, the Fisher criterion has better results.

[0174] To further verify the advantage of the OWA-ILDA algorithm in terms of recognition rate, the experiment adopted the method of gradually increasing the number of samples to compare and analyze the recognition rates of each algorithm. Among them Figure 4 is the average recognition rate of 10 operations of each algorithm on the AR data set when the number of test samples ranges from 1 to 500. Figure 5 is the average recognition rate of 10 operations of each algorithm on the YALE data set when the number of test samples ranges from 1 to 105. Figure 6 is the average recognition rate of 10 operations of each algorithm on the ORL data set when the number of test samples ranges from 1 to 240. Figure 7 is the average recognition rate of 10 operations of each algorithm on the FERET data set when the number of test samples ranges from 1 to 800. Figure 8 shows the average recognition rate of 10 runs of each algorithm on the MNIST data set when the number of test samples ranges from 1 to 5000.

[0175] As Figures 4 to 8As shown, the recognition rate of OWA-ILDA is the highest on all datasets: 92.36% (AR > 100 samples), 88.64% (YALE > 20 samples), 90.28% (ORL > 50 samples), 86.76% (FERET > 60 samples), and 83.87% (MNIST > 500 samples). These results are better than those of PCA, LDA, NLDA, HMLDA, ILDA, OWA-LDA, and OWA-NLDA. Compared with traditional LDA, the recognition rates of OWA-LDA are 63.56%, 80.86%, 82.42%, 75.88%, and 75.83% respectively.

[0176] In summary, the recognition rate of OWA-ILDA is not only higher than that of ILDA, but overall, the recognition rate of OWA-ILDA is the highest among all algorithms. In addition, the average recognition rates of OWA-NLDA and OWA-LDA are higher than those of NLDA and LDA respectively. The results show that the between-class scatter matrix based on the OWA operator improves the recognition rates of LDA, NLDA, and ILDA. As the sample size increases, the recognition rate of OWA-ILDA is always higher than that of HMLDA, indicating that the generalization ability of this algorithm is stronger.

[0177] To verify the feature extraction ability of the OWA-LDA algorithm, this embodiment will compare and analyze the OWA-LDA algorithm from the perspective of class discrimination. Figure 9 and Figure 10 are the visualization results of the PIE dataset in a two-dimensional subspace. Four categories in this dataset are selected, and 10 samples are selected from each category. After the decision of the OWA operator, it is found that the discrimination between categories is significantly improved, and different categories do not cluster together. Figure 11 and Figure 12 are the projection results of the IRIS dataset in a two-dimensional subspace. There are three categories in total, with 40 samples in each category. After the decision of the OWA operator, the samples of the same category are more closely clustered.

[0178] The experimental results show that the OWA-LDA algorithm proposed in Embodiment 1 of the present invention can enhance the discrimination between categories, and at the same time better extract the main information of the data between the same categories, thereby improving the ability to process high-dimensional data. However, at the same time, it also has a certain impact on the distance between different categories. Although the discrimination between categories is improved, the distance between categories is reduced. In large-scale datasets, it may have more or less negative impacts on data categories, resulting in unsatisfactory results in data dimensionality reduction and classification.

[0179] Based on the OWA operator and the LDA algorithm, the present invention combines them to obtain the OWA-LDA and OWA-ILDA algorithms. By calculating the global sample mean using the OWA operator, a new between-class scatter matrix is obtained, which increases the discrimination between different classes and improves the recognition rate of image feature extraction.

[0180] Experimental results on AR, YALE, ORL, FERET, and MNIST datasets show that: compared with the classical LDA algorithm, the recognition rate of OWA-LDA has been improved, and the recognition rate of the OWA-ILDA algorithm is better than that of the ILDA algorithm, and it has the highest recognition rate among all the comparison algorithms. This fully proves that under the action of the OWA operator, the successful recognition rates of the LDA and ILDA algorithms become more stable and effective, and it also represents the superiority and feasibility of the OWA-ILDA algorithm. The scatter plots of two-dimensional subspace sample distributions on the Iris dataset and the PIE dataset show that: the between-class scatter matrix based on the OWA operator can improve the discrimination between different data classes, thereby more effectively improving the fisher discriminant criterion and making the feature information more complete after data dimensionality reduction. This confirms the effectiveness of OWA-LDA and OWA-ILDA in data class discrimination. OWA-ILDA proposed in Embodiment 2 of the present invention not only shows higher accuracy and reliability in face recognition tasks, but also can effectively improve the discrimination of data classes.

[0181] It should be understood that although the steps in the flowcharts of the embodiments of the present invention are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages does not necessarily have to be sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0182] It should be understood that each part of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following technologies well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0183] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0184] The above-described embodiments merely represent several implementation manners of the present invention. The description is relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims

1. An improved incremental linear discriminant analysis feature extraction method, characterized in that: The method comprises the following steps: Step 1, obtain image training samples of different categories and convert the image training samples into data matrices; Step 2: Calculate the global sample mean of the data matrix based on the OWA operator, calculate the category means of image training samples of different categories, and calculate the inter-class scatter matrix of the data matrix according to the global sample mean; Calculate the intra-class scatter matrix based on the class means; Step 3: When a new image training sample is input, the global sample mean is updated to obtain a new global sample mean; According to the new global sample mean and the category mean of the new image training sample, the inter-class scatter matrix and the intra-class scatter matrix are updated respectively to perform incremental learning, and the inter-class scatter matrix and the intra-class scatter matrix after incremental learning are obtained; Step 4: According to the intra-class scatter matrix after incremental learning and the inter-class scatter matrix after incremental learning, the new projection matrix is ​​obtained using the Fisher discriminant criterion; Step 5: Obtain test image samples and project the test image samples onto the new projection matrix to achieve data dimensionality reduction and feature extraction.

2. The improved incremental linear discriminant analysis feature extraction method according to claim 1, characterized in that: In step 2, the process corresponding to the global sample mean of the data matrix calculated based on the OWA operator has the following relationship: ； in, represents the global sample mean, Indicates N training samples, represents the total number of training samples, Indicates j The weight of the training samples, Indicates the training sample j The largest value; ； in, It represents the non-decreasing fuzzy quantifier function, which is used to represent the distribution of non-decreasing ratio fuzzy language quantifiers in fuzzy logic; Represents the number of parameter vectors.

3. The improved incremental linear discriminant analysis feature extraction method according to claim 2, characterized in that: The process corresponding to the non-decreasing fuzzy quantifier function has the following relationship: ； in, represents the input variable, represents the lower bound of the increasing interval, Indicates the upper bound of the increment interval.

4. The improved incremental linear discriminant analysis feature extraction method according to claim 3, characterized in that: In step 2, the process corresponding to the inter-class scatter matrix of the data matrix calculated according to the global sample mean value has the following relationship: ； in, represents the inter-class scatter matrix, represents the number of categories, Indicates j The number of class samples, Indicates j The mean of the class samples, represents the transpose operation, represents the vector dimension, Represents the set of real numbers.

5. The improved incremental linear discriminant analysis feature extraction method according to claim 4, characterized in that: In step 2, the process of calculating the intra-class scatter matrix according to the class mean value has the following relationship: ； in, represents the intra-class scatter matrix, Indicates that it belongs to j The sample vector of the class, Represents the new input training sample.

6. The improved incremental linear discriminant analysis feature extraction method according to claim 5, characterized in that: In step 3, the global sample mean is updated, and the process corresponding to obtaining the new global sample mean has the following relationship: ； in, represents the new global sample mean.

7. The improved incremental linear discriminant analysis feature extraction method according to claim 6, characterized in that: In step 3, the process of updating the inter-class scatter matrix according to the new global sample mean and the class mean of the new image training sample has the following relationship: ； in, represents the updated inter-class scatter matrix, Indicates the updated The class sample mean of the class.

8. The improved incremental linear discriminant analysis feature extraction method according to claim 7, characterized in that: Updated i The calculation process of the class sample mean of a class has the following relationship: ； in, Indicates i The number of class samples, Indicates i The mean of the class samples.

9. The improved incremental linear discriminant analysis feature extraction method according to claim 8, characterized in that: In step 3, the process of updating the intra-class scatter matrix according to the new global sample mean and the class mean of the new image training sample has the following relationship: ； in, represents the updated intra-class divergence matrix.

10. The improved incremental linear discriminant analysis feature extraction method according to claim 9, characterized in that: In step 4, the method of obtaining a new projection matrix using the Fisher discriminant criterion according to the inter-class scatter matrix after incremental learning and the intra-class scatter matrix after incremental learning specifically includes the following steps: The generalized eigenvalue matrix is ​​calculated using the inter-class scatter matrix after incremental learning and the intra-class scatter matrix after incremental learning; Perform eigendecomposition on the generalized eigenvalue matrix to obtain a diagonal matrix containing several eigenvalues ​​and an eigenvector matrix consisting of eigenvectors corresponding to the eigenvalues; Sort several eigenvalues ​​in descending order, and select the eigenvectors corresponding to the first several eigenvalues ​​to form a new projection matrix.

Images