Improved incremental linear discriminant analysis feature extraction method

Abstract

The invention provides an improved incremental linear discriminant analysis feature extraction method. The method comprises the following steps: firstly, calculating an initial OWA global mean value, an inter-class divergence matrix and an intra-class divergence matrix of an image training sample to generate an initial model; in the subsequent step, for each newly added sample, the global mean value variable is dynamically updated, and the inter-class divergence matrix and the intra-class divergence matrix are adjusted based on the updated global mean value. And then, a Fisher discriminant criterion is used to calculate a new projection matrix, and a test image is projected to the new projection matrix, so that dimension reduction and feature extraction of data are realized. The core innovation of the method is that an OWA operator is adopted, and a brand new global sample mean value is generated according to different contributions of features among samples. In addition, an improved inter-class divergence matrix is provided, so that the difference between different classes is remarkably enhanced. By improving the classification distinction degree, the method can more comprehensively reserve data information, and provides feature representation with higher quality for subsequent classification tasks.

Description

Technical Field

[0001] The invention relates to the technical field of data dimension reduction, and in particular to an improved incremental linear discriminant analysis feature extraction method. Background Art

[0002] Linear discriminant analysis (LDA) is an important supervised dimensionality reduction tool that is widely used in clinical medicine, education, aerospace and other fields. LDA processes data by finding the optimal projection matrix so that data of the same category are clustered together and data of different categories are separated as much as possible. In order to improve the performance of the LDA algorithm, researchers continue to optimize the LDA algorithm to make up for the limitations of feature extraction and projection methods. Because the Fisher method requires that the constructed divergence matrix satisfies non-singularity, when the divergence matrix is ​​non-singular and the data dimension is much larger than the sample size, the LDA algorithm cannot be directly applied to the undersampling problem where the data dimension is much larger than the sample size. For this reason, researchers have continuously explored the problem of LDA dimension being larger than the sample size.

[0003] The classic LDA algorithm gives the same weight to different inter-class distances when using the arithmetic mean to calculate the inter-class distances, which cannot meet the scenario where there are differences in the inter-class distances. For this reason, researchers have proposed LDA algorithms for different application scenarios. For example, Zhu et al. proposed the Neighborhood LDA (NLDA) algorithm to solve the problem that the intra-class structure cannot be correctly described when the class contains subclasses. The NLDA algorithm regards the neighborhood as the smallest subclass, so that the optimal projection direction can be found without any clustering algorithm. Zheng et al. developed the Harmonic Mean LDA (HMLDA) algorithm to calculate the inter-class distances, which improved the performance of the LDA algorithm. Li et al. defined a new kernel function constraint (IIKC) to assist LDA in extracting more features and further improve the differences between nearby data points. For feature extraction in different application scenarios, researchers proposed a feature extraction method that combines PCA and LAD. For example, Tong applied the PCA-LDA algorithm to analyze SERS spectra to determine the condition of liver cancer. Rao et al. used the PCA-LDA algorithm to solve the small sample problem (SSS). Dilip et al. used PCA-LDA for feature extraction of diabetes data sets to improve the accuracy of diagnosis.

[0004] When the amount of data grows dynamically in real time, the LDA algorithm cannot meet the needs of real-time feature extraction. For this reason, researchers have proposed an incremental LDA algorithm and continuously improved it. Among them, Pang et al. designed the Incremental LDA (ILDA) algorithm based on the incremental idea, which solved the situation where a complete training sample set was not given in advance and was applied to the scene of real-time feature extraction. It has become an important tool for education, image processing, etc. However, ILDA does not consider the situation where there are differences in class distances. In order to deal with the difficulty of inverse of the intra-class divergence matrix of the ILDA algorithm, Zhao et al. developed the GSVD-ILDA algorithm based on the perspective of generalized singular value decomposition, which is used to calculate the projection matrix in the entire space. In order to improve the classification accuracy and computational complexity efficiency of ILDA, Chu et al. developed the ILDA / QR algorithm based on QR of matrix decomposition. Through analysis, it is found that the existing LDA and ILDA algorithms give the same weight to different class distances during feature extraction, but in real life, larger class distances will affect the accuracy of feature extraction.

[0005] In summary, the existing LDA and ILDA algorithms do not consider the differences in class distances during feature extraction. However, in real life, the data collected often have differences in class distances, and a larger class distance will affect the accuracy of feature extraction. Summary of the invention

[0006] In view of the above situation, the main purpose of the present invention is to propose an improved incremental linear discriminant analysis feature extraction method to solve the above technical problems.

[0007] The present invention proposes an improved incremental linear discriminant analysis feature extraction method, which comprises the following steps: Step 1, obtain image training samples of different categories and convert the image training samples into data matrices; Step 2: Calculate the global sample mean of the data matrix based on the OWA operator, calculate the category means of image training samples of different categories, and calculate the inter-class scatter matrix of the data matrix according to the global sample mean; Calculate the intra-class scatter matrix based on the class means; Step 3: When a new image training sample is input, the global sample mean is updated to obtain a new global sample mean; According to the new global sample mean and the category mean of the new image training sample, the inter-class scatter matrix and the intra-class scatter matrix are updated respectively to perform incremental learning, and the inter-class scatter matrix and the intra-class scatter matrix after incremental learning are obtained; Step 4: According to the intra-class scatter matrix after incremental learning and the inter-class scatter matrix after incremental learning or the intra-class scatter matrix and the inter-class scatter matrix, a new projection matrix is ​​obtained using the Fisher discriminant criterion; Step 5: Obtain test image samples and project the test image samples onto the new projection matrix to achieve data dimensionality reduction and feature extraction.

[0008] Compared with the prior art, the present invention has the following beneficial effects: The classic linear discriminant analysis is to minimize the intra-class distance and maximize the inter-class distance. In the process of calculating the inter-class scatter matrix, the mean is solved by arithmetic average. However, the inter-class distance obtained by arithmetic average has great limitations. This method assumes that the contribution of each feature is the same, so that all inter-class distances are given the same weight, but the contributions of different features are often different. According to different feature contributions, a new global sample mean is obtained based on the OWA operator, and a new inter-class scatter matrix is ​​proposed. The new inter-class scatter matrix makes the differences between different categories larger, and retains data information more comprehensively by improving the class distinction.

[0009] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is the change diagram of non-decreasing ratio fuzzy language quantifier; Figure 2 This is a comparison chart of the algorithm recognition rates of the present invention and the prior art on an AR data set; Figure 3 This is a comparison chart of the algorithm recognition rates of the present invention and the prior art on the YALE data set; Figure 4 This is a graph showing the average recognition rate of the present invention and the prior art in an AR data set; Figure 5 This is a graph showing the average recognition rates of the present invention and the prior art in the YALE dataset; Figure 6 This is a graph showing the average recognition rate of the present invention and the prior art in the ORL data set; Figure 7 This is a graph showing the average recognition rate of the present invention and the prior art in the FERET data set; Figure 8 This is a graph showing the average recognition rates of the present invention and the prior art in the MNIST data set; Fig. 9 This is the result of the ILDA algorithm visualizing the PIE dataset in a two-dimensional subspace; Fig.10 This is a result diagram of visualizing the PIE data set in a two-dimensional subspace in Example 2; Fig.11 This is the result of the ILDA algorithm visualizing the IRIS dataset in a two-dimensional subspace; Fig.12 This is a graph showing the result of visualizing the IRIS dataset in a two-dimensional subspace in Example 2. DETAILED DESCRIPTION

[0011] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0012] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to represent some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0013] Example 1 This embodiment provides an improved incremental linear discriminant analysis feature extraction method, the method comprising the following steps: Step 1, obtain image training samples of different categories and convert the image training samples into data matrices; Step 2: Calculate the global sample mean of the data matrix based on the OWA operator, calculate the category means of image training samples of different categories, and calculate the inter-class scatter matrix of the data matrix according to the global sample mean; Calculate the intra-class scatter matrix based on the class means; In step 2, the process corresponding to the global sample mean of the data matrix calculated based on the OWA operator has the following relationship: ; in, represents the global sample mean, Indicates N training samples, represents the total number of training samples, Indicates j The weight of the training samples, Indicates the training sample j The largest value; ; in, It represents the non-decreasing fuzzy quantifier function, which is used to represent the distribution of non-decreasing ratio fuzzy language quantifiers in fuzzy logic; Represents the number of parameter vectors.

[0014] The process corresponding to the non-decreasing fuzzy quantifier function has the following relationship: ; in, represents the input variable, represents the lower bound of the increasing interval, Indicates the upper bound of the increment interval.

[0015] In step 2, the process corresponding to the inter-class scatter matrix of the data matrix calculated according to the global sample mean value has the following relationship: ; in, represents the inter-class scatter matrix, represents the number of categories, Indicates j The number of class samples, Indicates j The mean of the class samples, represents the transpose operation, represents the vector dimension, Represents the set of real numbers.

[0016] In step 2, the process of calculating the intra-class scatter matrix according to the class mean value has the following relationship: ; in, represents the intra-class scatter matrix, Indicates that it belongs to j The sample vector of the class, Represents the new input training sample.

[0017] Step 3: According to the intra-class scatter matrix and the inter-class scatter matrix, the new projection matrix is ​​obtained using the Fisher discriminant criterion; In step 3, the method of obtaining a new projection matrix using the Fisher discriminant criterion according to the inter-class scatter matrix and the intra-class scatter matrix specifically includes the following steps: The generalized eigenvalue matrix is ​​calculated using the between-class scatter matrix and the within-class scatter matrix; Perform eigendecomposition on the generalized eigenvalue matrix to obtain a diagonal matrix containing several eigenvalues ​​and an eigenvector matrix consisting of eigenvectors corresponding to the eigenvalues; Sort several eigenvalues ​​in descending order, and select the eigenvectors corresponding to the first several eigenvalues ​​to form a new projection matrix.

[0018] Step 4: Obtain test image samples and project the test image samples onto the new projection matrix to achieve data dimensionality reduction and feature extraction.

[0019] Example 2 This embodiment provides an improved incremental linear discriminant analysis feature extraction method, the method comprising the following steps: Step 1, obtain image training samples of different categories and convert the image training samples into data matrices; Step 2: Calculate the global sample mean of the data matrix based on the OWA operator, calculate the category means of image training samples of different categories, and calculate the inter-class scatter matrix of the data matrix according to the global sample mean; Calculate the intra-class scatter matrix based on the class means; In step 2, the process corresponding to the global sample mean of the data matrix calculated based on the OWA operator has the following relationship: ; in, represents the global sample mean, Indicates N training samples, represents the total number of training samples, Indicates j The weight of the training samples, Indicates the training sample j The largest value; ; in, It represents the non-decreasing fuzzy quantifier function, which is used to represent the distribution of non-decreasing ratio fuzzy language quantifiers in fuzzy logic; Represents the number of parameter vectors.

[0020] The process corresponding to the non-decreasing fuzzy quantifier function has the following relationship: ; in, represents the input variable, represents the lower bound of the increasing interval, Indicates the upper bound of the increment interval.

[0021] In step 2, the process corresponding to the inter-class scatter matrix of the data matrix calculated according to the global sample mean value has the following relationship: ; in, represents the inter-class scatter matrix, represents the number of categories, Indicates j The number of class samples, Indicates j The mean of the class samples, represents the transpose operation, Represents the vector dimension.

[0022] In step 2, the process of calculating the intra-class scatter matrix according to the class mean value has the following relationship: ; in, represents the intra-class scatter matrix, Indicates that it belongs to j The sample vector of the class, Represents the new input training sample.

[0023] Step 3: When a new image training sample is input, the global sample mean is updated to obtain a new global sample mean; The inter-class scatter matrix and the intra-class scatter matrix are updated according to the new global sample mean and the category mean of the new image training sample, respectively, to obtain the inter-class scatter matrix and the intra-class scatter matrix after training; In step 3, the global sample mean is updated, and the process corresponding to obtaining the new global sample mean has the following relationship: ; in, represents the new global sample mean.

[0024] In step 3, the process of updating the inter-class scatter matrix according to the new global sample mean and the class mean of the new image training sample has the following relationship: ; in, represents the updated inter-class scatter matrix, Indicates the updated i The class sample mean of the class.

[0025] Updated i The calculation process of the class sample mean of a class has the following relationship: ; in, Indicates i The number of class samples, Indicates i The mean of the class samples.

[0026] In step 3, the process of updating the intra-class scatter matrix according to the new global sample mean and the class mean of the new image training sample has the following relationship: ; in, represents the updated intra-class divergence matrix.

[0027] Step 4: According to the intra-class scatter matrix after incremental learning and the inter-class scatter matrix after incremental learning, the new projection matrix is ​​obtained using the Fisher discriminant criterion; In step 4, the method of obtaining a new projection matrix using the Fisher discriminant criterion according to the inter-class scatter matrix after incremental learning and the intra-class scatter matrix after incremental learning specifically includes the following steps: The generalized eigenvalue matrix is ​​calculated using the inter-class scatter matrix after incremental learning and the intra-class scatter matrix after incremental learning; Perform eigendecomposition on the generalized eigenvalue matrix to obtain a diagonal matrix containing several eigenvalues ​​and an eigenvector matrix consisting of eigenvectors corresponding to the eigenvalues; Sort several eigenvalues ​​in descending order, and select the eigenvectors corresponding to the first several eigenvalues ​​to form a new projection matrix.

[0028] Step 5: Obtain test image samples and project the test image samples onto the new projection matrix to achieve data dimensionality reduction and feature extraction.

[0029] In summary, the present invention proposes an improved incremental linear discriminant analysis feature extraction method. First, the initial OWA global mean, inter-class scatter matrix and intra-class scatter matrix of the image training sample are calculated to generate an initial model. In the subsequent steps, for each newly added sample, the global mean variable is dynamically updated, and the inter-class scatter matrix and the intra-class scatter matrix are adjusted based on the updated global mean. Subsequently, the new projection matrix is ​​calculated using the Fisher discriminant criterion, and the test image is projected onto the new projection matrix, thereby achieving data dimensionality reduction and feature extraction. The core innovation of the present invention is to use the OWA operator to generate a new global sample mean based on the different contributions of features between samples. In addition, an improved inter-class scatter matrix is ​​proposed, which significantly enhances the differences between different categories. By improving the class distinction, this method can more comprehensively retain data information and provide higher quality feature representation for subsequent classification tasks.

[0030] In order to explain the specific process and effect of the present invention in more detail, it is assumed that is a group of l Category N The data matrix of training samples, yes j The number of class samples, is the new global sample mean, It is j The mean of the class samples. Then the inter-class scatter matrix based on OWA is shown in formula (7).

[0031] (7); The new global sample mean is the sum of the sample set According to formula (8), the global sample mean of OWA of sample set X can be obtained: , in formula (8), yes Middle j The largest value.

[0032] (8); In formula (8), how to obtain effective weights to calculate the global mean of the sample set is a key issue. Since the OWA operator is often used to implement the concept of the majority of models, especially in the case of quantifier-guided aggregation, fuzzy quantifiers are used to calculate the weights of each element. In order to reasonably and accurately display the characteristics of samples of different categories in the sample set, the present invention adjusts the weights of the OWA operator through relatively decreasing quantifiers, thereby achieving the flexibility of weighted averaging. The relatively decreasing quantifier representation and weight calculation are shown in formula (9).

[0033] (9); In formula (9), n represents the number of parameter vectors, where the non-decreasing proportional fuzzy linguistic quantifier Q is calculated as shown in formula (10).

[0034] (10); In formula (10), In this embodiment, a=0.3, b=0.7. The change of the non-decreasing ratio fuzzy language quantifier Q is as follows: Figure 1 shown.

[0035] Figure 1 It shows how the quantifier decreases as the input parameter decreases. It starts from 1 and decreases linearly to 0. The resulting weight matrix shows excellent discriminative ability for the following three reasons.

[0036] First, decreasing quantifiers allow decision makers to emphasize key factors with higher weights and assign lower weights to unimportant factors. This allows the main factors to be fully taken into account without overemphasizing the secondary factors, so that the decision results better reflect the decision makers' true preferences and goals. Second, linear weight decline provides a consistent and predictable weight distribution, which helps to establish clear priorities between factors. Finally, although the weight contains only two elements, the method is still flexible. By reducing the weight to 0 and a fixed constant value, the decision is simplified while maintaining adaptability to different scenarios. Among them, the fixed constant value can be adjusted to adapt to different preferences and needs.

[0037] Generalized Eigenvalue Matrix SIt is obtained by multiplying the inverse of the intra-class scatter matrix and the inter-class scatter matrix, as shown in formula (11). It contains the intra-class and inter-class information of the data. S To find the best projection direction, so that samples of different categories can be better separated after projection, and realize data dimensionality reduction and classification.

[0038] (11); in, represents the generalized eigenvalue matrix, represents the inverse of the within-class scatter matrix; The intra-class scatter matrix is ​​used to measure the differences between samples in the same category. The intra-class scatter matrix can be calculated using the following formula.

[0039] (12); in, Indicates that it belongs to j A sample vector of classes.

[0040] In the solution proposed in Example 1 of the present invention, the following is denoted as the OWA-LDA algorithm, and the calculated Substituting into formula (11) we get the new generalized eigenvalue matrix .right Eigendecomposition: .in P The eigenvector The eigenvector matrix composed of The eigenvalue After sorting the eigenvalues ​​in descending order, select the first r largest eigenvalues ​​to form a new diagonal matrix , and its corresponding eigenvector matrix is ​​also a new projection matrix Finally, project the dataset onto a new projection matrix In this way, data dimension reduction and feature extraction can be achieved.

[0041] ILDA is an incremental learning algorithm based on LDA. It can gradually accept new data and update the discriminant space. Therefore, it has higher computational efficiency in large-scale data sets and can adapt to new data. In general, ILDA has the ability of incremental learning relative to LDA, can gradually accept new data and update the model, has higher computational efficiency and lower memory usage. This makes ILDA more advantageous in scenarios where large-scale data sets are processed and models are updated in real time. However, due to the characteristics of incremental learning, the ILDA algorithm cannot guarantee the distinction between different data categories. In order to solve this problem and improve the stability and feature extraction efficiency of ILDA, the present invention proposes a solution embodiment 2 with incremental learning capability based on the OWA-LDA algorithm, namely embodiment 1, hereinafter referred to as the OWA-ILDA algorithm.

[0042] Based on the OWA-LDA algorithm, the initial OWA global mean is first calculated , the inter-class scatter matrix and the intra-class scatter matrix Then, in the incremental learning process, for each new incoming sample x , using the formula Update the global mean variable, then update the inter-class scatter matrix and the intra-class scatter matrix, and finally use the Fisher discriminant criterion to obtain a new projection matrix. The specific process of the OWA-ILDA algorithm proposed in Example 2 of the present invention is as follows.

[0043] Algorithm 1: OWA-ILDA algorithm flow Input: Initial training sample set ; initialization: 1. Use formula (9) to calculate the weight W; 2. Use formula (8) to calculate the OWA global sample mean ; 3. Use formulas (7) and (12) to calculate the inter-class scatter matrix , the intra-class scatter matrix ; When a new training sample is input x hour: 4. Update the global sample mean : ; 5. Update the inter-class scatter matrix ; if x Is a training sample of a new category: x Tags y In the initial training sample set XThe corresponding category set Y Not found in: ; otherwise: ; It is the updated i The class sample mean of the class, ; 6. Update the intra-class scatter matrix : ; End the loop.

[0044] The Fisher discriminant criterion is used to compress the high-dimensional face image discriminant matrix into a low-dimensional feature space matrix: 7. Use formula (11) to calculate the new generalized characteristic matrix ; 8. Yes The characteristic decomposition is: ; ; and They are the generalized eigenvalue matrices Decomposed eigenvalues ​​and corresponding eigenvectors; 9. After sorting the eigenvalues ​​from large to small, select the first r largest eigenvectors and eigenvalues, and we have: ; ; is the best projection direction, is the eigenvalue corresponding to the projection direction; 10. Multiply the new sample data set by , project the data into a low-dimensional space; Output: New sample set x The projection vector of .

[0045] In order to verify the effectiveness of the present invention, seven data sets were used for experiments. Among them, AR, YALE, ORL, FERET and MNIST data sets were used for recognition experiments, and IRIS and PIE data sets were used to study the influence of OWA operator on category distinction and intra-category aggregation in two-dimensional subspace.

[0046] The AR dataset contains more than 4,000 color facial images of 126 people (70 men and 56 women). The images taken of the same person have different occlusions, lighting conditions, and facial expressions. The original image is 576*768 pixels. For the convenience of the experiment, this example only selects complete facial images of 100 people (50 men and 50 women), with a total of 1,400 pictures. Each person has 14 different images, and all facial images are uniformly cropped to 50*40 pixels.

[0047] The YALE dataset contains 165 grayscale face images of 15 people. Each person has 11 photos taken under different conditions, including different facial expressions and different lighting conditions. The image size is 100*100 pixels. In the experiment, all images were cropped to 100*80 pixels.

[0048] The ORL dataset contains 400 face images of 40 people, which contain some changes in posture, expression and facial organs. Each participant provided 10 normalized grayscale images with image size of 92*112 and black background.

[0049] The FERET dataset was created by the FERET project. It contains 14,051 facial images with different poses and illuminations. The photos of the same person in the dataset have different expressions, poses, illuminations, and ages. Most of them are Westerners. For the convenience of the experiment, we selected images of 200 people and cropped all the images to 80*80 pixels.

[0050] IRIS dataset: The IRIS dataset is a widely used classification dataset collected and organized by Fisher. It contains 150 data samples divided into 3 categories with 50 samples in each category. Each sample consists of four attributes.

[0051] PIE dataset: The PIE dataset is a face database from the Carnegie Mellon Robotics Institute. It includes images of 68 people, each with 10 images of different poses, lighting conditions, and expressions.

[0052] MNIST dataset: The MNIST dataset is derived from the original NIST database, which includes 60,000 training images and 10,000 test images sampled from the same distribution. Each grayscale digit is normalized and centered within a 28 × 28 pixel image, with the center of gravity of the intensity at the center of the image.

[0053] These datasets provide comprehensive facial image samples that capture different positions, expressions, and lighting conditions. They provide a solid foundation for evaluating and validating the performance of the proposed algorithms, which is essential for research in areas such as attention detection, facial recognition, and expression analysis. In order to make the results convincing, each database is preprocessed, and then one-third of the samples in each category are randomly selected as training sets, and two-thirds of the samples are used as test sets. All algorithms are run ten times (each time the random samples are changed) to calculate the average accuracy. All experiments are run on a PC equipped with an Intel Core I5-12400F CPU and a GTX4060Ti GPU.

[0054] In order to compare and analyze the performance of the proposed method, the following steps are adopted: First, the proposed algorithm and the original algorithm are used to extract the feature vectors from the face image. Then, the label of these feature vectors is judged by the centroid classifier, and the Manhattan distance is used as the distance measurement method of the centroid. This method can effectively measure the similarity between different samples, so as to classify more accurately. Finally, in order to comprehensively evaluate the feature extraction efficiency and performance of different algorithms in face recognition, the recognition rate is used as the core evaluation index. The recognition rate, that is, the proportion of correctly classified samples, is shown in formula (13). It can intuitively reflect the accuracy and reliability of the algorithm in the face recognition task. By comparing the recognition rates of different algorithms, the advantages and disadvantages of various algorithms in feature extraction and classification can be clearly seen, thus providing a strong basis for subsequent research and improvement.

[0055] (13) In formula (13), TP represents true positive, that is, the number of samples correctly classified as positive examples; TN represents true negative, that is, the number of samples correctly classified as negative examples; FP represents false positive, that is, the number of samples incorrectly classified as positive examples; FN represents false negative, that is, the number of samples incorrectly classified as negative examples.

[0056] In the experiment, AR, YALE, ORL, FERET and MNIST image datasets were used to verify the improvement of LDA and ILDA algorithms. In order to prove the superiority of OWA-ILDA, the experiment used the centroid classifier to classify the images, and compared the results of recognition accuracy of eight algorithms including LDA, PCA, NLDA, HMLDA, ILDA, OWA-NLDA, OWA-LDA and OWA-ILDA to determine the superiority of the algorithm.

[0057] This experiment requires setting up training and testing samples for the AR, YALE, ORL, FERET, and MNIST datasets, as shown in Table 1.

[0058] Table 1. Division of the dataset

[0059] After 10 rounds of iterative calculations using PCA, LDA, ILDA, NLDA, HMLDA, OWA-LDA, OWA-NLDA, and OWA-ILDA, the average recognition rates of each algorithm are compared as shown in Table 2.

[0060] Table 2 Average recognition rate of algorithms on the dataset (%)

[0061] As shown in Table 2, the OWA-ILDA algorithm has the highest average recognition rate, which is 90.22%, 87.62%, 88.94%, 84.76% and 83.87% on the AR, YALE, ORL, FERET and MNIST datasets, respectively. In addition, compared with the original algorithm, the algorithm using the OWA operator has a significant improvement.

[0062] In order to statistically verify the performance advantage of OWA-ILDA, we performed a Friedman test analysis on the results shown in Table 2. Table 3 gives the Friedman test statistics and their corresponding p-values ​​and the critical values ​​of OWA-ILDA. In Table 3, The p-value of algorithm accuracy is less than the significance level α = 0.05, which means that there is enough evidence in the observed data to reject the null hypothesis, indicating that OWA-ILDA is significantly different from other algorithms.

[0063] Table 3 Friedman test results F F ( = 8, = 5).

[0064] Note: “k” represents the number of algorithms compared; “N” represents the number of datasets In order to more intuitively illustrate the superiority of the algorithm containing the OWA operator, each algorithm is compared with the algorithm after adding the OWA operator. The 10-run results of the average recognition rate of each algorithm on the AR and YALE datasets are shown in Figure 2 and Figure 3 shown.

[0065] The experimental comparison chart shows that after adding the OWA operator for method decision-making, the average recognition rate of the algorithm has been significantly improved, and this result is universal. This verifies that the OWA operator can make the divergence matrix have better performance and make the Fisher criterion have better results through the divergence matrix.

[0066] In order to further verify the advantage of the OWA-ILDA algorithm in recognition rate, the experiment adopts the method of gradually increasing the number of samples to compare and analyze the recognition rates of various algorithms. Figure 4 It is the average recognition rate of 10 operations of each algorithm on the AR dataset when the number of test samples ranges from 1 to 500. Figure 5 It is the average recognition rate of 10 operations of each algorithm on the YALE dataset when the number of test samples ranges from 1 to 105. Figure 6 It is the average recognition rate of 10 operations of each algorithm on the ORL dataset when the number of test samples ranges from 1 to 240. Figure 7 It is the average recognition rate of 10 operations of each algorithm on the FERET dataset when the number of test samples ranges from 1 to 800. Figure 8 The average recognition rate of each algorithm over 10 runs is shown on the MNIST dataset when the number of test samples ranges from 1 to 5000.

[0067] like Figures 4 to 8 As shown in the figure, OWA-ILDA has the highest recognition rate on all datasets: 92.36% (AR>100 samples), 88.64% (YALE>20 samples), 90.28% (ORL>50 samples), 86.76% (FERET>60 samples) and 83.87% (MNIST>500 samples). These results are better than PCA, LDA, NLDA, HMLDA, ILDA, OWA-LDA and OWA-NLDA. Compared with traditional LDA, the recognition rates of OWA-LDA are 63.56%, 80.86%, 82.42%, 75.88% and 75.83% respectively.

[0068] In summary, the recognition rate of OWA-ILDA is not only higher than that of ILDA, but also the highest among all algorithms in general. In addition, the average recognition rates of OWA-NLDA and OWA-LDA are higher than those of NLDA and LDA, respectively. The results show that the inter-class divergence matrix based on the OWA operator improves the recognition rates of LDA, NLDA, and ILDA. As the sample size increases, the recognition rate of OWA-ILDA is always higher than that of HMLDA, indicating that the algorithm has stronger generalization ability.

[0069] In order to verify the feature extraction capability of the OWA-LDA algorithm, this embodiment will compare and analyze the OWA-LDA algorithm from the perspective of category discrimination. Fig. 9 and Fig.10 The visualization result of the PIE dataset in the two-dimensional subspace is shown in Figure 2. Four categories in the dataset are selected, and 10 samples are selected from each category. After the OWA operator decision, it is found that the distinction between categories is significantly improved, and different categories are not clustered together. Fig.11 and Fig.12 This is the result of projecting the IRIS dataset in a two-dimensional subspace. There are three categories in total, with 40 samples in each category. After the OWA operator decision, the samples of the same category are more closely clustered.

[0070] The experimental results show that the OWA-LDA algorithm proposed in Example 1 of the present invention can enhance the discrimination between categories and improve the extraction of the main information of the data between the same categories, thereby improving the ability to process high-dimensional data. But at the same time, the distance between different categories also has a certain impact. Although the discrimination between categories is improved, the category distance is reduced. In a large-volume data set, it may have a more or less negative impact on the data category, resulting in unsatisfactory results of data dimensionality reduction and classification.

[0071] The present invention integrates the OWA operator and the LDA algorithm to obtain the OWA-LDA and OWA-ILDA algorithms. The global sample mean is calculated by the OWA operator to obtain a new inter-class scatter matrix, which increases the distinction between different categories and improves the recognition rate of image feature extraction.

[0072] The experimental results on AR, YALE, ORL, FERET and MNIST data sets show that: OWA-LDA has an improved recognition rate compared with the classic LDA algorithm, while the recognition rate of the OWA-ILDA algorithm is better than that of the ILDA algorithm, and the recognition rate is the highest among all the comparison algorithms. This fully proves that under the action of the OWA operator, the successful recognition rate of the LDA and ILDA algorithms becomes more stable and effective, and also represents the superiority and feasibility of the OWA-ILDA algorithm. The two-dimensional subspace sample distribution scatter plot on the Yingweihua data set and the PIE data set shows that the inter-class divergence matrix based on the OWA operator can improve the discrimination between different data categories, thereby more effectively improving the Fisher discriminant criterion, so that the feature information is more completely preserved after the data dimension reduction. This confirms the effectiveness of OWA-LDA and OWA-ILDA in data category discrimination. The OWA-ILDA proposed in Example 2 of the present invention not only shows higher accuracy and reliability in face recognition tasks, but also can effectively improve the discrimination of data categories.

[0073] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0074] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0075] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0076] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An improved incremental linear discriminant analysis feature extraction method, characterized in that: The method comprises the following steps: Step 1, obtain image training samples of different categories and convert the image training samples into data matrices; Step 2: Calculate the global sample mean of the data matrix based on the OWA operator, calculate the category means of image training samples of different categories, and calculate the inter-class scatter matrix of the data matrix according to the global sample mean; Calculate the intra-class scatter matrix based on the class means; Step 3: When a new image training sample is input, the global sample mean is updated to obtain a new global sample mean; According to the new global sample mean and the category mean of the new image training sample, the inter-class scatter matrix and the intra-class scatter matrix are updated respectively to perform incremental learning, and the inter-class scatter matrix and the intra-class scatter matrix after incremental learning are obtained; Step 4: According to the intra-class scatter matrix after incremental learning and the inter-class scatter matrix after incremental learning or the intra-class scatter matrix and the inter-class scatter matrix, a new projection matrix is ​​obtained using the Fisher discriminant criterion; Step 5: Obtain test image samples and project the test image samples onto the new projection matrix to achieve data dimensionality reduction and feature extraction.

2. The improved incremental linear discriminant analysis feature extraction method according to claim 1, characterized in that: In step 2, the process corresponding to the global sample mean of the data matrix calculated based on the OWA operator has the following relationship: ； in, represents the global sample mean, Indicates N training samples, represents the total number of training samples, Indicates j The weight of the training samples, Indicates the training sample j The largest value; ； in, It represents the non-decreasing fuzzy quantifier function, which is used to represent the distribution of non-decreasing ratio fuzzy language quantifiers in fuzzy logic; Represents the number of parameter vectors.

3. The improved incremental linear discriminant analysis feature extraction method according to claim 2, characterized in that: The process corresponding to the non-decreasing fuzzy quantifier function has the following relationship: ； in, represents the input variable, represents the lower bound of the increasing interval, Indicates the upper bound of the increment interval.

4. The improved incremental linear discriminant analysis feature extraction method according to claim 3, characterized in that: In step 2, the process corresponding to the inter-class scatter matrix of the data matrix calculated according to the global sample mean value has the following relationship: ； in, represents the inter-class scatter matrix, represents the number of categories, Indicates j The number of class samples, Indicates j The mean of the class samples, represents the transpose operation, represents the vector dimension, Represents the set of real numbers.

5. The improved incremental linear discriminant analysis feature extraction method according to claim 4, characterized in that: In step 2, the process of calculating the intra-class scatter matrix according to the class mean value has the following relationship: ； in, represents the intra-class scatter matrix, Indicates that it belongs to j The sample vector of the class, Represents the new input training sample.

6. The improved incremental linear discriminant analysis feature extraction method according to claim 5, characterized in that: In step 3, the global sample mean is updated, and the process corresponding to obtaining the new global sample mean has the following relationship: ； in, represents the new global sample mean.

7. The improved incremental linear discriminant analysis feature extraction method according to claim 6, characterized in that: In step 3, the process of updating the inter-class scatter matrix according to the new global sample mean and the class mean of the new image training sample has the following relationship: ； in, represents the updated inter-class scatter matrix, Indicates the updated The class sample mean of the class.

8. The improved incremental linear discriminant analysis feature extraction method according to claim 7, characterized in that: Updated i The calculation process of the class sample mean of a class has the following relationship: ； in, Indicates i The number of class samples, Indicates i The mean of the class samples.

9. The improved incremental linear discriminant analysis feature extraction method according to claim 8, characterized in that: In step 3, the process of updating the intra-class scatter matrix according to the new global sample mean and the class mean of the new image training sample has the following relationship: ； in, represents the updated intra-class divergence matrix.

10. The improved incremental linear discriminant analysis feature extraction method according to claim 9, characterized in that: In step 4, the method of obtaining a new projection matrix using the Fisher discriminant criterion according to the inter-class scatter matrix and the intra-class scatter matrix specifically includes the following steps: The generalized eigenvalue matrix is ​​calculated using the between-class scatter matrix and the within-class scatter matrix; Perform eigendecomposition on the generalized eigenvalue matrix to obtain a diagonal matrix containing several eigenvalues ​​and an eigenvector matrix consisting of eigenvectors corresponding to the eigenvalues; Sort several eigenvalues ​​in descending order, and select the eigenvectors corresponding to the first several eigenvalues ​​to form a new projection matrix.

Images