A kernelized inverse nearest neighbor discriminant analysis method
By mapping the data to a high-dimensional space and performing kernelized inverse nearest neighbor discriminant analysis, the problem of decreased recognition performance of inverse nearest neighbor discriminant analysis on nonlinear data is solved, achieving a high recognition rate.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU UNIVERSITY
- Filing Date
- 2023-03-10
- Publication Date
- 2026-05-26
AI Technical Summary
Existing inverse nearest neighbor discriminant analysis methods suffer from reduced recognition performance when dealing with nonlinear data, particularly in handwritten digit, face, and multi-view target recognition problems.
The original data is mapped to a high-dimensional Hilbert space using a Gaussian kernel mapping function. Inverse nearest neighbor analysis is then performed in the high-dimensional space, and a high-dimensional inverse nearest neighbor scatter matrix is constructed by deriving the kernel trick to eliminate outliers and achieve effective processing of nonlinear data.
It improves the recognition performance of inverse nearest neighbor discriminant analysis on nonlinear data, especially achieving high recognition rates of 97.50% and 99.30% on the USPS and MNIST handwritten datasets, respectively.
Smart Images
Figure CN116433960B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of inverse nearest neighbor discriminant analysis, and particularly relates to a kernelized inverse nearest neighbor discriminant analysis method. Background Technology
[0002] Linear Discriminant Analysis (LDA) is a classic supervised linear learning method derived from Fisher's discriminant formula. It can be used not only for feature dimensionality reduction but also effectively to solve supervised classification problems. Its principle is to find a projection matrix in the given training samples that maximizes the between-class scatter matrix and minimizes the within-class scatter matrix. This projection matrix is then used to reduce the dimensionality of the test samples, which can then be classified using nearest neighbor or other classifiers. As a supervised feature extraction method, LDA has proven its feasibility and effectiveness in pattern recognition and classification tasks and has been widely applied in data dimensionality reduction, signal analysis, image classification, and other fields. Significant research results have been achieved, particularly in visual classification tasks such as handwritten digit recognition, face recognition, and object classification.
[0003] In the LDA algorithm, it is necessary to assume that the samples follow an independent and identically distributed (i.i.d.) pattern to obtain good classification results. However, when samples of the same class come from several different clusters with different variances, the above assumption does not hold, and LDA struggles to achieve good classification results. Mixture discriminant analysis (MDA) is an important method to solve this problem, using a Gaussian model to model different clusters. Another solution is to use manifold or graph theory methods to model the local structure. Yet another approach uses nonparametric methods such as k-nearest neighbors to model the local neighborhood. Among these, linear discriminant analysis based on inverse nearest neighbors (nLDA) is a recently proposed effective method for solving multimodal problems. nLDA constructs a scatter matrix using the inverse nearest neighbor algorithm, which is an effective unsupervised outlier detection method that can eliminate "outliers" in the training set, resulting in better classification capabilities. nLDA directly defines the scatter matrix on the neighborhood, making it a novel local discriminator. Its smallest subclass can be considered as a neighborhood, thus solving the problem that the original LDA algorithm required samples to follow an independent and identically distributed (i.i.d.) pattern. Since nLDA is an extension of LDA, it is also a linear method. When encountering nonlinear data, the results are not ideal. Taking face recognition as an example, changes in facial expression, pose, or lighting often lead to nonlinear results. A common solution to nonlinear problems is to map the points of the original input data to a higher-dimensional Euclidean space, and then learn a classifier in this higher-dimensional space for discriminative learning. Therefore, to improve the performance of nLDA in handling nonlinear data, this invention proposes a kernelized inverse nearest neighbor discriminant analysis method (KRNDA). This method first maps the original data to a higher-dimensional Hilbert space using a Gaussian kernel mapping function, and then performs inverse nearest neighbor discriminant analysis in the higher-dimensional space. However, since the space of the Gaussian kernel mapping is not explicit, its implicit function cannot be explicitly expressed. Summary of the Invention
[0004] The purpose of this invention is to provide a kernelized inverse nearest neighbor discriminant analysis method. The original data is first mapped to a high-dimensional Hilbert space using a Gaussian kernel mapping function, and then inverse nearest neighbor discriminant analysis is performed in the high-dimensional space. Simultaneously, a kernelized derivation of the linear inverse nearest neighbor discriminant analysis is provided using kernel tricks. Furthermore, in the process of finding the inverse nearest neighbors of a sample, distances in the high-dimensional space are used to obtain high-dimensional inverse nearest neighbors, further effectively eliminating outliers.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A kernelized inverse nearest neighbor discriminant analysis method includes the following steps:
[0007] S1. Obtain training image samples, wherein the training image samples are X = [x1, x2, ..., x...]. N ];
[0008] S2. The input data is mapped to a high-dimensional space using a Gaussian kernel function, where the input data is training image samples;
[0009] S3. Using kernel tricks and the inverse nearest neighbor algorithm, the intra-class scatter matrix and inter-class scatter matrix representations of the training image samples in high-dimensional space are obtained;
[0010] S4. Learn the projection matrix that maximizes the inter-class scatter matrix and minimizes the intra-class scatter matrix through eigenvalue decomposition;
[0011] S5. Use the projection matrix to train and the test image samples to extract features;
[0012] S6. Use the nearest neighbor algorithm to classify the test samples.
[0013] Furthermore, the specific method of mapping the input data to a high-dimensional space using a Gaussian kernel function involves defining a mapping... Make each sample from the original feature space R d Mapping to a high-dimensional feature space F, it can be represented as: R d →F, The goal is to make linear subspaces in the feature space spanned by mapping vectors, where This means mapping sample x from the original feature space to a higher-dimensional feature space. It is the representation of the sample in a high-dimensional feature space. The i-th image vector x from the training samples... i and the j-th image vector x j The inner product is expressed in the following kernel function form:
[0014]
[0015] Where <·> represents the inner product of two vectors in the high-dimensional feature space; a typical and effective Gaussian kernel function k is chosen. g And perform inner product calculation, which is expressed as follows:
[0016]
[0017] Here, σ is an adjustable parameter in the Gaussian kernel function.
[0018] Furthermore, the method of using kernel tricks and inverse nearest neighbor algorithms to obtain the intra-class scatter matrix and inter-class scatter matrix representations of training image samples in high-dimensional space is as follows:
[0019] S301. Determine the inverse nearest neighbor of a sample in a high-dimensional kernel space. (Sample points in high-dimensional space...) A set of image samples after N mappings The k-th order inverse nearest neighbor is expressed as:
[0020]
[0021] in, Representing high-dimensional sample points The k-th order nearest neighbor;
[0022] To obtain an optimal projection matrix ω = [ω1, ω2, ..., ω] in high-dimensional space i ,…,ω h ], h represents the dimension of the final feature retention, which makes the inter-class divergence Maximize intra-class divergence Minimize, that is, maximize the following expression:
[0023]
[0024] In the high-dimensional space F, there are h projection vectors ω = [ω1, ω2, ..., ω]. i ,…,ω h Then the projection vector is linearly represented in the F-space spanned by the original high-dimensional data:
[0025]
[0026] Where, α ij N and N refer to the coefficients of the linear combination and the total number of training samples, respectively.
[0027] S302, The optimal projection moment ω was obtained, and the mapped high-dimensional samples were analyzed. By projecting, the dimensionality-reduced discriminative features can be obtained. but
[0028]
[0029] Using the linear expression and kernel trick above, in the above formula... Expressed as:
[0030]
[0031] Where α i =[α i1 ,α i2 ,…,α iN ] T ,because Then K m This represents the inner product vector between all trained high-dimensional samples and the m-th high-dimensional sample;
[0032] S303, Using Gaussian kernel function Calculate the kernel function k(x) i ,x m ), calculate the low-dimensional projection feature g m The discriminant features after dimensionality reduction are optimized, among which the low-dimensional projection feature g m Expressed as:
[0033]
[0034] Where α = [α1, α2, ..., α h ] T .
[0035] Furthermore, the specific steps for learning the projection matrix of maximizing the inter-class scatter matrix and minimizing the intra-class scatter matrix through eigenvalue decomposition are as follows: The problem of learning to maximize the inter-class scatter matrix and minimize the intra-class scatter matrix is transformed into a classic eigenvalue decomposition problem:
[0036] Bα=λWα
[0037] Where B and W are the inter-class scatter matrix and intra-class scatter matrix after mapping, respectively.
[0038] Furthermore, the classic eigenvalue decomposition problem is: during the solution process, by analyzing (W)... -1 B is subjected to eigenvalue decomposition, and the eigenvectors corresponding to the h largest eigenvalues are taken as α = [α1, α2, ..., α]. h And thus, the final solution is obtained.
[0039] Furthermore, the feature extraction using the projection matrix to train image samples specifically involves:
[0040] S501. For all training image samples X = [x1, x2, ..., x...] N Feature extraction;
[0041] S502. Obtain the training image sample set according to the high-dimensional mapping.
[0042] S503, Apply the formula to N high-dimensional training samples. Find the k-th order inverse nearest neighbors of the same type of sample, and when the number of inverse nearest neighbors is not less than t, calculate the mean of the inverse nearest neighbor samples, i.e. Projecting the high-dimensional mean vector yields the discriminative features. Based on the s3 discriminant features and low-dimensional projection features, proceed to g. m The derivation relationship between the linear combination coefficient α and the linear combination coefficient α can be obtained.
[0043] Finally, the discriminant features G = α of all training samples are obtained. T K, where
[0044] G = [g1, g2, ..., g M (M≤N), K=[K1,K2,…,K M ], where K is the kernel matrix.
[0045] S504, For a test sample x t According to the high-dimensional mapping, we get Projecting the high-dimensional sample yields the discriminative features. Get g t =α T K t .
[0046] Furthermore, the classification of the test image samples using the nearest neighbor algorithm specifically involves: based on g in S5... t With G = [g1, g2, ..., g M The inverse nearest neighbor mean of the M original samples in high-dimensional space is used to calculate the nearest neighbor distance using Euclidean distance, where g is... t The category is G = [g1, g2, ..., g M The class label of the nearest neighbor discriminant feature in the [].
[0047] The beneficial technical effects of the present invention are at least as follows:
[0048] The kernelized inverse nearest neighbor discriminant analysis (KRNDA) proposed in this invention effectively solves the performance degradation problem of the inverse nearest neighbor linear discriminant analysis (nLDA) method when processing nonlinear data (such as handwritten digits, faces, and multi-view target recognition problems). In the binary classification problem of identifying odd and even numbers on the USPS and MNIST handwritten digit datasets, the kernelized inverse nearest neighbor discriminant analysis method outperforms LDA, LFDA, ccLDA, and L... 2,1 - Improved LDA-based algorithms such as RLDA and nLDA. Achieve top recognition rates of 97.50% and 99.30% on the ORL face dataset and the coil-20 object recognition dataset, respectively. Attached Figure Description
[0049] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.
[0050] Figure 1 This is a flowchart of the kernelized inverse nearest neighbor discriminant analysis method according to an embodiment of the present invention.
[0051] Figure 2 This is a schematic diagram of digital samples in USPS and MNIST according to Embodiment 2 of the present invention.
[0052] Figure 3 This is a partial image illustration of the COIL-20 dataset from Embodiment 2 of the present invention.
[0053] Figure 4 This is a schematic diagram of a portion of the ORL face dataset from Embodiment 2 of the present invention. Detailed Implementation
[0054] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Specific implementation method one:
[0056] like Figure 1 As shown, the average value control method for the leveling system provided in this embodiment of the invention is a kernelized inverse nearest neighbor discriminant analysis method, which includes the following steps:
[0057] S1: Obtain training image samples X = [x1, x2, ..., x...] N ].
[0058] S2: Use the Gaussian kernel function to map the input data to a high-dimensional space, requiring each sample to be mapped from the original feature space to the high-dimensional feature space. The goal is to make a linear subspace in the feature space spanned by a mapping vector. This involves two image vectors x... i ,x j The inner product is expressed in the following kernel function form:
[0059]
[0060] Where <·> represents the inner product of two vectors in the high-dimensional feature space. Many mapping functions do not have explicit expressions, but their inner products can be expressed by formulas. Selecting a suitable kernel function to implicitly compute the inner product of two vectors in the high-dimensional feature space is a crucial step in all kernel methods. This invention selects a typical and effective Gaussian kernel function for inner product calculation, expressed as follows:
[0061]
[0062] σ is an adjustable parameter in the Gaussian kernel function. This allows the inner product of two samples in high-dimensional space to be expressed explicitly.
[0063] S3: Utilize kernel tricks and the inverse nearest neighbor algorithm to obtain the intra-class and inter-class scatter matrices of the training samples in high-dimensional space. First, determine the inverse nearest neighbors of the samples in the high-dimensional kernel space. Then, the sample points in the high-dimensional space... A set of image samples after N mappings The k-th order inverse nearest neighbor can be expressed as:
[0064]
[0065] in Representing high-dimensional sample points The k-th order nearest neighbor.
[0066] The goal of the algorithm is to find an optimal projection matrix ω = [ω1, ω2, ..., ω] in a high-dimensional space. h ], h is the eigenvector corresponding to the h largest eigenvalues, such that the between-class scatter matrix Maximize, while also maximizing the intra-class scatter matrix Minimize, that is, optimize the following expression:
[0067]
[0068] In the high-dimensional space F, there are h projection vectors ω = [ω1, ω2, ..., ω]. h It can be linearly represented in the F-space spanned by the original high-dimensional data:
[0069]
[0070] Where α ij and N refer to the coefficients of the linear combination and the total number of training samples, respectively. Suppose we obtain the optimal projection matrix ω, and then apply this to the high-dimensional samples after mapping. By projecting, the dimensionality-reduced discriminative features can be obtained. So there is
[0071]
[0072] Using the linear expression and kernel trick above, in the above formula... This can be expressed as:
[0073]
[0074] Where α i =[α i1 ,α i2 ,…,α iN ] T ,because Then use K m The kernel function k(x) represents the inner product vector between all training high-dimensional samples and the m-th high-dimensional sample.i ,x m The calculation method uses the Gaussian kernel function. conduct.
[0075] Calculate the low-dimensional projection feature g m The discriminant features after dimensionality reduction are optimized, among which the low-dimensional projection feature g m This can be expressed as:
[0076]
[0077] J KRNDA (ω)
[0078] Where α = [α1, α2, ..., α h ] T Using the above relationships, the optimization formula can be optimized. Rewritten as:
[0079]
[0080] in
[0081]
[0082] It is the number of local samples obtained from the reverse nearest neighbor.
[0083] In this context, t is the threshold for the number of inverse nearest neighbor samples, i.e., when... A local neighborhood is constructed only when the number of inverse neighbors reaches t, and this local neighborhood is used to calculate the mean of the high-dimensional sample. This refers to labels and samples x j The same set of image samples.
[0084] Optimize formula J KRNDA (ω) Rewritten as:
[0085]
[0086] in
[0087]
[0088] In summary,
[0089]
[0090] S4: Through eigenvalue decomposition learning, we can maximize the projection matrix of the between-class scatter matrix and minimize the projection matrix of the within-class scatter matrix. The optimization problem of maximizing the above formula can be transformed into a classic eigenvalue decomposition problem, i.e.
[0091] Bα=λWα
[0092] During the solution process, by analyzing (W) -1 B is subjected to eigenvalue decomposition, and the eigenvectors corresponding to the h largest eigenvalues are taken as α = [α1, α2, ..., α]. h ], thus obtaining the final solution.
[0093] S5: Use the projection matrix to extract features from the samples. First, for all training samples X = [x1, x2, ..., x...] N Extract features and obtain them according to high-dimensional mapping. Using the formula for N high-dimensional training samples Find the k-th order inverse nearest neighbors of the same type of sample, and when the number of inverse nearest neighbors is not less than t, calculate the mean of the inverse nearest neighbor samples, i.e. Projecting the high-dimensional mean vector yields the discriminative features. Based on the S3 discriminant features and low-dimensional projection features g m The derivation relationship between the linear combination coefficient α and the linear combination coefficient α can be obtained. Finally, the discriminant features G = α of all training samples are obtained. T K, where G = [g1, g2, ..., g M (M≤N), K=[K1,K2,…,K M ], K is the kernel matrix; for a test sample x t According to the high-dimensional mapping, we get Projecting the high-dimensional sample yields the discriminative features. Get g t =α T K t .
[0094] S6: Classify the test samples using the nearest neighbor algorithm, specifically by taking g from step S5. t With G = [g1, g2, ..., g M The inverse nearest neighbor mean of the M original samples in high-dimensional space is used to calculate the nearest neighbor distance using Euclidean distance. Then, g... t The category is G = [g1, g2, ..., g M The class label of the nearest neighbor discriminant feature in the []. Specific Implementation Method Two:
[0096] To demonstrate the effectiveness of the algorithm of this invention, this embodiment implements the proposed algorithm in handwritten digit recognition, face recognition, and object recognition datasets. The datasets used include the MNIST and USPS handwritten digit datasets, the ORL face dataset, and the coil-20 object recognition dataset. This invention is compared with the non-kernelized nLDA algorithm on the above datasets to highlight the effectiveness of this invention. In addition, related linear discriminant algorithms: LDA, LFDA, and L... 2,1 - A comparison of RLDA and ccLDA. LFDA is a linear discriminator built using local information. 2,1 -RLDA is a linear discriminant that utilizes L1-norm regularization. In ccLDA, the intra-cluster and inter-cluster scatter matrices from linear discriminant analysis are used to normalize the intra-class and inter-class scatter matrices, thus solving the overfitting problem caused by small sample sizes in LDA.
[0097] Figure 2 The image shows some handwritten font images from the MNIST and USPS datasets, with odd numbers on the left and even numbers on the right.
[0098] On the handwritten dataset, this embodiment divides the classification problem into a binary classification problem, namely, determining whether a digit is odd or even. The odd class consists of 1, 3, 5, 7, and 9. The even class consists of 2, 4, 6, 8, and 10. Obviously, based on the similarity of the font shape, both odd and even numbers actually contain 5 subclasses. The linear discriminant analysis based on inverse nearest neighbors (nLDA) method has good recognition performance for this multi-subclass image classification task. Different numbers of samples were randomly selected for experiments, such as MNIST1000, MNIST6000, MNIST10000, and MNIST15000. This means that we randomly selected 1000, 6000, 10000, and 15000 samples from the 60000 training samples in the MNIST dataset for training, and randomly selected test data from the 10000 test samples at a ratio of approximately 6:1. The USPS dataset contains a total of 9298 handwritten images. In this implementation, the training set contains 7291 images, and the test set contains 2007 images. The recognition results are shown in Table 1, with the best results highlighted in bold. It is easy to see that the kernelized inverse nearest neighbor discriminant analysis (KRNDA) algorithm proposed in this invention has better recognition results, demonstrating its ability to process nonlinear images.
[0099] Table 1. Comparison of accuracy rates for USPS and MNIST digit recognition problems (unit: %)
[0100] Dataset LDA LFDA ccLDA <![CDATA[L 2,1 -RLDA]]> nLDA KRNDA USPS 89.83 94.47 89.93 96.71 96.66 97.65 Mnist1000 87.35 78.91 76.50 87.95 87.95 92.17 Mnist6000 81.90 86.60 81.00 96.40 96.30 96.60 Mnist10000 82.29 86.31 82.11 97.05 95.73 97.60 Mnist15000 82.16 87.96 82.68 97.08 96.36 98.04
[0101] According to Table 1, we can see that KRNDA achieves recognition rates of 97.65%, 92.17%, 96.60%, 97.60%, and 98.04% on USPS and MNIST 1000, MNIST 6000, MNIST 10000, and MNIST 15000, respectively. On all USPS and MNIST 1000, MNIST 6000, MNIST 10000, and MNIST 15000, KRNDA significantly outperforms LDA, LFDA, ccLDA, and L... 2,1 - RLDA and nLDA. As the amount of MNIST training data increases, our recognition rate also increases, and its performance is also better than other linear classifiers.
[0102] Next, this invention uses the coil-20 (Columbia Object Image Database) dataset and the ORL dataset to evaluate the method. The coil-20 dataset consists of 20 different object categories, each with 72 images from different viewpoints, such as cars, cats, cups, erasers, etc. These images were manually resized and normalized to 20×20 pixels. 50% of the images for each category were randomly selected for training, and the remaining images were used for testing. Figure 3 As shown, some image examples from this dataset are presented.
[0103] The ORL face dataset contains 400 images of 40 individuals. Some images were taken at different times and exhibit variations, including facial expressions (eyes open or closed, smiling or not smiling) and facial details (wearing glasses or not). Face images extracted from the ORL dataset were resized and normalized to 20×20 pixels. Similarly, 50% of the images were randomly selected for training in each class, while the remaining images were used for testing. Figure 4 As shown, some image examples from this dataset are presented.
[0104] As shown in Table 2, the proposed KRNDA outperforms the original nLDA algorithm, achieving recognition rates of 97.50% and 99.30% on the ORL and coil-20 datasets, respectively. Furthermore, compared to other linear discriminant algorithms, this invention also exhibits significant performance advantages. The above embodiments illustrate the effectiveness of this invention.
[0105] Table 2. Comparison of recognition rates on ORL and coil-20 datasets (unit: %)
[0106] Dataset LDA FLDA ccLDA <![CDATA[L 2,1 -RLDA]]> nLDA KRNDA ORL 94.00 92.00 90.00 95.00 95.00 97.50 coil-20 93.88 96.66 91.50 97.92 95.14 99.30
[0107] The above embodiments of the present invention extend the inverse nearest neighbor linear discriminant analysis (nLDA) method to high-dimensional space for the first time to solve the classification problem of nonlinear data. It proposes to use a Gaussian kernel function for high-dimensional mapping, kernelize the nLDA algorithm, and use kernel tricks to derive the non-explicit mapping. In the space after Gaussian function mapping, a high-dimensional inverse nearest neighbor is established and the scatter matrix is obtained. The kernel trick is used to realize the distance expression in high-dimensional space.
[0108] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
Claims
1. A kernelized inverse nearest neighbor discriminant analysis method, characterized in that... It includes the following steps: S1. Obtain training image samples, wherein the training image samples are ; S2. The input data is mapped to a high-dimensional space using a Gaussian kernel function, where the input data is training image samples; S3. Using kernel tricks and the inverse nearest neighbor algorithm, the intra-class scatter matrix and inter-class scatter matrix representations of the training image samples in high-dimensional space are obtained; S4. Learn the projection matrix that maximizes the between-class scatter matrix and minimizes the within-class scatter matrix through eigenvalue decomposition: S5. Use the projection matrix to train and test image samples for feature extraction; S6. Classify the test image samples using the nearest neighbor algorithm; The specific method of using the Gaussian kernel function to map the input data to a high-dimensional space involves defining a mapping... This allows each sample to be derived from the original feature space. Mapping to a high-dimensional feature space F, it can be represented as: The purpose is to make the linear subspace in the feature space spanned by the mapping vector, where This means mapping sample x from the original feature space to a higher-dimensional feature space. It is the representation of sample x in a high-dimensional feature space; The method of using kernel tricks and inverse nearest neighbor algorithms to obtain the intra-class and inter-class scatter matrices of training image samples in high-dimensional space is as follows: S301. Determine the inverse nearest neighbor of a sample in a high-dimensional kernel space. (Sample points in high-dimensional space...) A set of image samples after N mappings The k-th order inverse nearest neighbor is expressed as: in, Representing high-dimensional sample points The k-th order nearest neighbor; To obtain an optimal projection matrix in high-dimensional space h represents the dimension of the final feature retention, such that the inter-class scatter matrix... Maximize the intra-class scatter matrix Minimize, i.e. maximize This can be expressed by the following formula: h projection vectors in high-dimensional space F Then the projection vector is linearly represented in the F-space spanned by the original high-dimensional data: in, N and N refer to the linear combination coefficient and the total number of training samples, respectively; S302, The optimal projection matrix was obtained. By mapping the high-dimensional samples By projecting, the dimensionality-reduced discriminative features can be obtained. ,but Using the linear expression and kernel trick above, in the above formula... Expressed as: in ,because ,but This represents the inner product vector between all trained high-dimensional samples and the m-th high-dimensional sample; S303, Using Gaussian kernel function Calculate kernel function k Calculate the low-dimensional projection features The discriminant features after dimensionality reduction are optimized, including the low-dimensional projection features. Expressed as: in .
2. The inverse nearest neighbor discriminant analysis method for nucleation according to claim 1, characterized in that... , the i-th image vector in the training samples and the j-th image vector The inner product is expressed in the following kernel function form: in, It is the inner product of two vectors in a high-dimensional feature space; a typical and effective Gaussian kernel function is selected. And perform inner product calculation, which is expressed as follows: in, It is an adjustable parameter in the Gaussian kernel function.
3. The nucleated inverse nearest neighbor discriminant analysis method according to claim 1, characterized in that... The specific steps for learning the projection matrix that maximizes the between-class scatter matrix and minimizes the within-class scatter matrix through eigenvalue decomposition are as follows: The problem of learning the projection matrix that maximizes the between-class scatter matrix and minimizes the within-class scatter matrix is transformed into a classic eigenvalue decomposition problem. Where B and W are the inter-class scatter matrix and intra-class scatter matrix after mapping, respectively.
4. The inverse nearest neighbor discriminant analysis method for nucleation according to claim 3, characterized in that... The classic eigenvalue decomposition problem is: during the solution process, by... B is subjected to eigenvalue decomposition, and the eigenvectors corresponding to the h largest eigenvalues are taken as α=[ , ,…, And thus, the final solution is obtained.
5. The nucleated inverse nearest neighbor discriminant analysis method according to claim 4, characterized in that... The specific steps of using the projection matrix for training and the test image samples for feature extraction are as follows: S501, For all training image samples Feature extraction; S502. Obtain the training image sample set according to the high-dimensional mapping. ; S503, Apply the formula to N high-dimensional training samples. Find the k-th order inverse nearest neighbors of the same type of sample, and when the number of inverse nearest neighbors is not less than t, calculate the mean of the inverse nearest neighbor samples, i.e. The discriminant features are obtained by projecting the high-dimensional mean vector. Based on the S3 discriminant features and low-dimensional projection features and linear combination coefficients The derivation relationship can be obtained as follows: Finally, the discriminative features of all training samples are obtained. M≤N , Kernel matrix; S504, For a test sample According to the high-dimensional mapping, we get The discriminant features are obtained by projecting the high-dimensional sample. ,get .
6. The inverse nearest neighbor discriminant analysis method for nucleation according to claim 5, characterized in that... The classification of the test image samples using the nearest neighbor algorithm specifically involves: according to S5... With G= The inverse nearest neighbor mean of the M original samples in high-dimensional space is used to calculate the nearest neighbor distance using Euclidean distance. The category is G= The class label of the discriminant feature that is the nearest neighbor in the middle.