A nonlinear data classification method based on kernel manifold optimization

Abstract

This invention belongs to the field of data classification technology and relates to a nonlinear data classification method based on kernel manifold optimization. The method includes: acquiring test samples to be classified and a labeled training sample set; calculating the kernel similarity within the training sample set and between the test sample and the training sample set using a kernel function to construct a kernel matrix and kernel vector, implicitly mapping the original data to a high-dimensional feature space; establishing a composite objective function with a linearly represented probability vector and an adaptive adjacency matrix as optimization variables; employing an alternating optimization strategy, optimizing the probability vector with a fixed adaptive adjacency matrix, and optimizing the adaptive adjacency matrix with a fixed probability vector, until convergence to obtain the optimal probability vector; calculating the sum of similarities between the test sample and each type of training sample based on the optimal probability vector, and assigning the test sample to the category with the highest similarity. This invention mines the nonlinear structure and manifold characteristics of the data, resulting in high classification accuracy and strong robustness.

Description

Technical Field

[0001] This invention belongs to the field of image data classification technology, and relates to a nonlinear data classification method based on kernel manifold optimization. Background Technology

[0002] The K-Nearest Neighbors (KNN) algorithm, a classic non-parametric statistical method, is widely used in pattern recognition, data mining, medical diagnosis, image processing, and recommender systems due to its simplicity, lack of a training phase, and good interpretability. The traditional KNN algorithm calculates the Euclidean distance between the test sample and all training samples, selects the K nearest neighbors, and determines the category of the test sample based on a majority voting principle.

[0003] However, with the advent of the big data era, the traditional KNN algorithm faces severe challenges. First, its computational complexity increases linearly with the size of the dataset, requiring each prediction to traverse the entire training set, resulting in excessively high computation time and storage costs on large-scale data. Second, traditional KNN lacks an explicit mathematical model, making it difficult to incorporate prior knowledge through regularization terms to optimize performance for specific tasks.

[0004] To address the aforementioned issues, researchers proposed the Linear KNN (L-KNN) method. L-KNN introduces a linear model, assuming that test samples can be linearly represented (i.e., linearly combined) from training samples, thus approximating the nonlinear decision boundary as a linear form. This method obtains representation coefficients (weights) by solving an optimization problem based on linear reconstruction error, which are then used for classification. Compared to traditional KNN, L-KNN is more efficient when handling large datasets, and its robustness can be improved by adding regularization terms such as L1 norm (LASSO) and L2 norm (Ridge Regression).

[0005] While L-KNN alleviates computational pressure and introduces a mathematical model to some extent, its core assumption—the "Linear Representation Assumption"—does not hold true in many practical applications. When data exhibits complex nonlinear distributions (such as price fluctuations in financial markets, gene expression patterns in bioinformatics, and texture features in image recognition), test samples often cannot be accurately reconstructed through a linear combination of training samples.

[0006] The existing technology has the following significant drawbacks:

[0007] 1. Accuracy degradation due to the failure of the linear assumption: In nonlinear datasets, linear reconstruction errors increase significantly. Forcing the use of a linear model to fit a nonlinear manifold leads to "underfitting," causing the calculated weight vectors to lose their physical meaning (i.e., the weights no longer represent the similarity or importance of samples), thus drastically reducing classification accuracy.

[0008] 2. Loss of manifold structure information: Traditional linear models often measure in Euclidean space, ignoring the low-dimensional manifold structure that may exist in high-dimensional space. The true distance between data points should be the geodesic distance along the manifold surface, not the straight-line Euclidean distance. Ignoring this will cause the model to fail to capture the local geometric features of the data.

[0009] 3. The physical meaning of weight allocation is ambiguous: Although sparse representation or local constraints are introduced in existing improvement methods, there is a lack of in-depth theoretical explanation of the weight allocation mechanism, especially the lack of theoretical support similar to Bayesian decision rules, which leads to poor interpretability of the model in complex environments.

[0010] Therefore, there is an urgent need for a new classification method that can maintain the computational efficiency of L-KNN, effectively handle nonlinear data, accurately capture the data manifold structure, and has a solid theoretical foundation. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the prior art and provide a nonlinear data classification method based on kernel manifold optimization.

[0012] To achieve the objectives of this invention, the following technical solutions will be adopted.

[0013] In a first aspect, the present invention provides a nonlinear data classification method based on kernel manifold optimization, comprising the following steps:

[0014] Step 1, Data Acquisition and Initialization: Acquire the test samples to be classified and the labeled training sample set containing multiple categories; define and initialize the probability vector and the adaptive adjacency matrix.

[0015] Step 2, Kernel Space Mapping: The kernel trick is introduced, implicitly mapping the input data from the original space to a high-dimensional (or even infinite-dimensional) regenerating kernel Hilbert space (RKHS) using a nonlinear kernel function. In the feature space, the originally linearly inseparable structure becomes linearly separable, thus making the linear reconstruction hypothesis valid again in the feature space.

[0016] Step 3: Construct a composite objective function: Build an optimization model containing five key components:

[0017] (1) Kernelized reconstruction error term: Minimize the reconstruction error of the test sample in the feature space caused by the linear combination of the training samples, and ensure the accuracy of the representation.

[0018] (2) Probability regularization term (entropy constraint): The probability regularization in the form of L2 norm (approximate information entropy maximization) is introduced to make the weight distribution smoother and more uniform, avoid the model from overfitting to a single training sample, and implicitly introduce Bayesian prior.

[0019] (3) Local constraint: The weights are penalized based on the distance between the original space or the feature space, which forces the model to prioritize training samples that are closer to the test samples, which is in line with the principle of locality.

[0020] (4) Manifold Preservation Regularization: This is one of the core innovations of this invention. A graph regularization term based on an adaptive Laplacian matrix is ​​introduced. Unlike traditional fixed graph structures, this invention dynamically learns the adjacency relationships between samples (adaptive adjacency matrix), ensuring that samples retain their local neighborhood structure on the original manifold after being mapped to the weight space. That is, if two training samples are close on the manifold, their representation weights should also be similar.

[0021] (5) Edge weight regularization term: The L2 norm constraint is applied to the elements of the adaptive adjacency matrix to prevent the edge weights from diverging or becoming too large during the optimization process, thus ensuring the stability of the graph structure.

[0022] Step 4, Alternating Optimization Solution: For the coupled probability vector and adaptive adjacency matrix in the objective function, an alternating direction optimization algorithm is designed.

[0023] With the adjacency matrix fixed, the probability vector is updated using gradient descent and simplex projection.

[0024] With a fixed probability vector, the adjacency matrix is ​​updated using gradient descent and non-negative projection.

[0025] The process iterates repeatedly until it converges, thereby simultaneously obtaining the optimal linear representation coefficients (probability vectors) and the graph matrix reflecting the data manifold structure.

[0026] Step 5, Classification Decision: Based on the optimized probability vector, perform weighted voting. Calculate the sum of the weights (i.e., the sum of similarity probabilities) of the test sample and all training samples belonging to the same category, and assign the test sample to the category with the highest total weight.

[0027] Secondly, the present invention provides a nonlinear data classification system based on kernel manifold optimization, comprising:

[0028] The data acquisition module is used to acquire the test samples to be classified and the training sample set with category labels;

[0029] The kernel matrix calculation module, based on a preset kernel function, calculates the kernel matrix within the training sample set and the kernel vector between the test sample and the training sample, thereby realizing the implicit mapping of the test sample and the training sample set to the high-dimensional feature space.

[0030] The model optimization module constructs and solves a composite objective function that includes a kernelized reconstruction error term, a probability regularization term, a local constraint term, a manifold preservation regularization term, and an edge weight regularization term. It optimizes the variables—the probability vector and the adaptive adjacency matrix—through an alternating optimization strategy until the composite objective function converges and outputs the optimal probability vector.

[0031] The classification prediction module takes the optimal probability vector as input, calculates the sum of similarities between the test sample and each type of training sample in the feature space, and classifies the test sample into the category with the largest sum of similarities.

[0032] Thirdly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the above-described method.

[0033] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed, implements the above-described method.

[0034] The beneficial effects of this invention are:

[0035] 1. This invention overcomes the dependence of traditional linear KNN on the "linear representation assumption" through kernel mapping technology. After mapping the data to a high-dimensional feature space, the nonlinear data distribution is "straightened out," which significantly reduces the linear reconstruction error (the reconstruction error can be reduced by several times), thereby effectively solving the problem that nonlinear data causes the physical meaning of weights to become invalid.

[0036] 2. This invention introduces an adaptive Laplacian matrix manifold-preserving regularization term. Unlike traditional fixed neighborhood graphs, the adaptive mechanism dynamically adjusts the connection weights between samples based on data feedback during training, thereby more accurately capturing the complex manifold structure of the data. This ensures that similar samples in the feature space remain similar in the weight space (i.e., the "nearest neighbor grouping effect"), improving the model's understanding of the data's intrinsic structure.

[0037] 3. This invention mathematically proves that the proposed model exhibits the "nearest neighbor group effect" and reveals its intrinsic connection with Bayesian decision rules. Through the optimization process, the model essentially approximates the maximum a posteriori probability estimate (MAP), giving the linear representation weights a clear physical meaning (i.e., the posterior probability of belonging to a certain class), thus enhancing the model's interpretability.

[0038] 4. Experiments on 15 benchmark datasets involving various data types (such as sonar, images, and biological data) demonstrate that our invention (KMOLNN) outperforms existing state-of-the-art methods such as KNN, FKNN, Elastic Net, and WLMRKNN in both classification accuracy and F1 score. The performance improvement is particularly significant on high-dimensional and highly nonlinear datasets (average accuracy improvement of approximately 6%-9%).

[0039] 5. By jointly optimizing the representation coefficients and graph structure, this invention does not rely on manually adjusted fixed K values ​​or adjacency graphs as in traditional methods, and can adaptively adapt to data with different densities and distribution characteristics. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating a nonlinear image data classification method based on kernel manifold optimization as described in this invention. Detailed Implementation

[0041] The present invention will be further described in conjunction with the accompanying drawings and embodiments.

[0042] As an embodiment 1 of the present invention, such as Figure 1 As shown, a nonlinear data classification method based on kernel manifold optimization includes the following steps:

[0043] S1. Obtain the test samples to be classified and the training sample set with class labels, and define and initialize the probability vector and the adaptive adjacency matrix; where: the probability vector is the linear representation weight of each training sample to the test sample; the adaptive adjacency matrix is ​​used to describe the manifold structure between training samples;

[0044] S2. Calculate the kernel similarity within the training sample set and between the test sample and the training sample set using kernel functions, construct the kernel matrix and kernel vector, and implicitly map the test sample and the training sample set to a high-dimensional feature space.

[0045] S3. Establish a composite objective function with probability vectors and adaptive adjacency matrix as optimization variables; the composite objective function includes kernelized reconstruction error term, probability regularization term, local constraint term, manifold preservation regularization term, and edge weight regularization term;

[0046] S4. Adopt an alternating optimization strategy to optimize the probability vector with a fixed adaptive adjacency matrix and optimize the adaptive adjacency matrix with a fixed probability vector until the composite objective function converges and the optimal probability vector is obtained.

[0047] S5. Calculate the sum of similarities between the test sample and each type of training sample in the feature space based on the optimal probability vector, and classify the test sample into the category with the largest sum of similarity.

[0048] The specific steps are detailed below: Step S1, Data Acquisition and Initialization:

[0049] First, the system receives the input face sample dataset and the test face samples to be classified. Assume the training sample set is... ,in: For feature dimension, This represents the number of training samples. Each training sample... Each has a corresponding category label. ,in: The total number of categories. The test samples to be classified are denoted as... .

[0050] Before optimization begins, the optimization variables need to be initialized:

[0051] Probability vector initialization: Define the probability vector Its elements Indicates the first training samples versus test samples The weights are represented. Initialization. For uniform distribution, i.e. This aligns with the principle of maximum entropy in the absence of prior information.

[0052] Adaptive adjacency matrix initialization: Define the adaptive adjacency matrix To accelerate convergence and provide a good initial structure, this embodiment uses the K-means clustering algorithm on the training samples. Perform preprocessing. Set the number of clusters to [number to be specified]. .

[0053] If training samples and If they belong to the same cluster, then initialize. ;

[0054] Otherwise, initialize ( For example, a very small positive number ), indicating a weak initial connection.

[0055] Step S2: Kernel Space Mapping and Kernel Matrix Calculation

[0056] Since the original data may be linearly inseparable, this invention utilizes a kernel mapping function. Mapping data to a high-dimensional feature space ( In actual calculations, it is not necessary to know explicitly. The specific form, but through the kernel function Calculate the inner product directly.

[0057] This embodiment preferably uses the Gaussian kernel function (RBF Kernel):

[0058] ;

[0059] in: This is the kernel width parameter.

[0060] In this step, the following two key quantities are calculated:

[0061] Training sample kernel matrix : .

[0062] Test-train kernel vectors : .

[0063] In addition, it is also necessary to calculate the kernel value of the test sample itself. (For Gaussian kernels, this value is usually 1).

[0064] Step S3: Construct a composite objective function

[0065] This is the core of the invention. We construct a minimization objective function. The aim is to find the optimal weights. and structure The objective function consists of five parts:

[0066] ;

[0067] The specific definitions and physical meanings of each part are as follows:

[0068] 1. Nucleation and reconstruction error term ( ):

[0069] ;

[0070] This measure assesses the accuracy of linear representation of test samples in a high-dimensional space. Using kernel tricks, we obtain:

[0071] ;

[0072] The matrix form is as follows: .

[0073] Compared to linear reconstruction of the original space (which often has large errors and deviates from the true point), kernelized reconstruction can more accurately approximate test samples with nonlinear distributions.

[0074] 2. Probability regularization term ( ):

[0075] ;

[0076] This term, by penalizing the sum of squares of the weights, encourages a smoother weight distribution, avoids the model relying solely on a single training sample (overfitting), and increases the robustness of the representation. Mathematically, this is equivalent to maximizing the inverse of the Gini exponent, approximating the maximum entropy principle.

[0077] 3. Local constraint terms ( ):

[0078] ;

[0079] in: It is to measure the test sample With training samples A distance metric for the difference between them. Defined in this embodiment. For the space distance of the core:

[0080] ;

[0081] The purpose of this item is to: training samples that are farther away from the test sample ( The larger the value, the greater its corresponding weight. The smaller the value, the more it forces the model to follow the principle of locality.

[0082] 4. Manifold Preservation Regularization Term :

[0083] ;

[0084] This method utilizes the Laplacian operator from Spectral Graph Theory. If two training samples... and Closely connected on the manifold structure (i.e., edge weights) If the features are relatively large and close in the feature space, then their representation weights are... and It should have some kind of correlation or joint smoothness.

[0085] This item can be rewritten in matrix form: ,in: For Laplace matrix, A degree matrix (diagonal matrix, ).

[0086] This ensures that the local manifold structure of the data is preserved after mapping to the weight space.

[0087] 5. Edge weight regularization term :

[0088] ;

[0089] This item is used to control the sparsity and size of the adaptive graph, preventing... Infinitely increase in size to ensure the stability of numerical calculations.

[0090] S4: Alternating Iterative Optimization Solution

[0091] Due to the objective function For variables and It is not jointly convex, but when one variable is fixed, it is convex with respect to the other variable. Therefore, this invention employs an alternating optimization strategy (Algorithm 1 & 2).

[0092] Step S41: Fix ,renew

[0093] when When fixed, the objective function with respect to The part is a quadratic programming problem:

[0094] ;

[0095] Subject to .

[0096] Calculate the gradient:

[0097] ;

[0098] because It is symmetrical, which can be simplified to:

[0099] ;

[0100] Update using gradient descent:

[0101] ;

[0102] in: This is the learning rate. After the update, it needs to be... Projection onto the probabilistic simplex satisfies the constraints of nonnegativity and summation to 1.

[0103] Step S42: Fix ,renew

[0104] when When fixed, the objective function with respect to The relevant items are:

[0105] ;

[0106] Note that in the manifold term, the Laplace matrix... Include Derivation of its effect The partial derivatives of .

[0107] Define distance matrix ,in .

[0108] Update via gradient descent :

[0109] ;

[0110] After the update, perform nonnegative projection: .

[0111] This step is actually learning the strength of the association between samples. If two samples are very close in the feature space ( (small) and their weight product The larger the value, the more the model tends to increase. This strengthens the manifold structure.

[0112] Step S43: Convergence Judgment

[0113] Calculate the relative change in the objective function value:

[0114] ;

[0115] like (For example If the maximum number of iterations is reached (e.g., 50 times), then stop the iteration; otherwise, return to step S41.

[0116] Step S5: Classification Decision

[0117] After optimization and convergence, the optimal probability vector is obtained. Each element of this vector Representative training samples For test samples The degree of attribution contribution.

[0118] This invention uses a category-based probability summation rule for classification:

[0119] For each category Calculate the sum of the weights of all training samples in this category:

[0120] ;

[0121] The predicted category is the category with the highest score:

[0122] .

[0123] As an embodiment of the present invention, such as Figure 1 As shown, a nonlinear data classification method based on kernel manifold optimization is applied to face recognition scenarios. The method utilizes the publicly available ORL and Extended Yale B face datasets to achieve accurate classification of multiple faces, adapting to complex real-world face shooting scenarios such as lighting changes, pose shifts, facial expression disturbances, and slight occlusion. Dataset preprocessing parameters include: a uniform face image size of 128×128 pixels; histogram equalization for lighting correction to eliminate light and shadow interference; 5×5 mean filtering to remove image acquisition noise; and pixel normalization to map image grayscale values ​​to the [0,1] interval. The ORL dataset contains 40 face classes with 10 samples per class, totaling 400 face images; the Extended Yale B dataset contains 38 face classes with 2414 face samples, covering various lighting angles, left and right lateral poses, and a variety of facial expressions. The training, test, and validation sets were randomly divided in a 7:2:1 ratio, with a total effective sample size N=2814. The model implementation parameters consistently adopted the preferred fixed parameters of this invention: Gaussian kernel bandwidth σ=2, regularization coefficients λ1=0.001, λ2=0.001, gradient descent learning rate 0.01, and iterative convergence threshold tol=10. -6 The maximum number of iterations is max_iter=100. Traditional face recognition classification algorithms mostly rely on matching global grayscale features with fixed local feature points, which cannot adapt to the highly nonlinear feature changes caused by facial pose, lighting, and expression. This easily leads to problems such as excessive differences in features among similar faces and confusion of features among dissimilar faces, resulting in high false positive and false negative rates. This invention maps low-dimensional facial pixel features to a high-dimensional kernel space through kernel manifold optimization, adaptively mining the inherent manifold correlation features of facial features, facial contours, and texture details, weakening the interference features caused by lighting, pose, and expression, and retaining the core facial identity identification features, thus achieving accurate face classification under complex interference. Actual test results show that the KMOLNN algorithm of this invention achieves an average classification accuracy of 98.15% and an F1 score of 0.980 in complex face recognition tasks. Compared with traditional KNN (91.26%) and MVAKNN (95.73%) algorithms, the classification accuracy is improved by more than 2.4%. Its anti-interference ability and recognition stability are significantly better than traditional algorithms. It can be adapted to the high-precision recognition needs of complex scenarios such as access control face recognition, dynamic face attendance, video face capture, and multi-face identity verification.

[0124] As an embodiment of the present invention, such as Figure 1As shown, a nonlinear data classification method based on kernel manifold optimization is applied to medical image classification. This embodiment uses the publicly available medical imaging datasets TCIA-LIDC lung CT lesion image dataset and CHEST-X-Ray chest radiograph dataset for specific implementation, to achieve automatic classification of three types of lesions: benign nodules, malignant nodules, and normal chest radiographs. Dataset preprocessing parameters: The image size is uniformly set to 256×256 pixels; Gaussian filtering (kernel size 3×3, standard deviation σ=1.2) is used to remove salt-and-pepper noise and scanning equipment noise; pixel values ​​are mapped to the [0,1] interval through grayscale normalization; the training set, test set, and validation set are randomly divided in a 7:2:1 ratio, with a total training sample count N=8960 and a test sample count of 2560. Model implementation parameters follow the preferred parameters of this invention: Gaussian kernel bandwidth σ=2, regularization coefficients λ1=0.001, λ2=0.001, and iteration convergence threshold tol=10. -6 The maximum number of iterations is max_iter=100. Traditional image classification methods often rely on shallow texture and grayscale features, failing to capture subtle nonlinear features such as blurred lesion edges and irregular shapes, leading to misdiagnosis and missed diagnosis. This invention maps low-dimensional image grayscale features to a high-dimensional kernel space through kernel manifold optimization, adaptively learning the local manifold topology of the lesion image, preserving the nonlinear correlation features of lesion texture, edges, and shape, and suppressing background tissue interference. Actual test results show that the KMOLNN algorithm of this invention achieves an accuracy of 97.82% and an F1 score of 0.976 in lung lesion classification tasks. Compared with traditional KNN (88.35%) and MVAKNN (94.12%) algorithms, the classification accuracy is improved by more than 3.7%, accurately distinguishing early small malignant nodules from benign proliferative nodules, and meeting the high-precision application needs of clinical auxiliary diagnosis and batch image screening.

[0125] As an embodiment of the present invention, such as Figure 1 As shown, a nonlinear data classification method based on kernel manifold optimization is applied to complex industrial signal classification scenarios. This embodiment uses the publicly available industrial equipment fault datasets CWRU bearing vibration dataset and fan acoustic fault dataset for specific implementation, aiming to achieve accurate classification of four states: normal operation, inner race fault, outer race fault, and rolling element fault. Dataset preprocessing parameters: The original sampling frequency is 12kHz. The time-series vibration and acoustic signals are segmented, with each segment lasting 0.2s. Time-domain features (peak value, kurtosis, waveform factor) and frequency-domain features (fundamental frequency amplitude, harmonic energy) are extracted from each segment to construct an 18-dimensional nonlinear feature vector. After removing outliers, the total number of effective samples is N=7200, which are divided into training and test sets in a 6:2:2 ratio. Model implementation parameters uniformly adopt the fixed optimal parameters of this invention: gradient descent learning rate 0.01, convergence threshold tol=10⁻ 6The maximum number of iterations is 100. Industrial field signals are generally characterized by strong noise interference, drastic temporal fluctuations, and severe nonlinear coupling of fault features. Traditional classification algorithms struggle to remove noise features and uncover subtle fault characteristics, resulting in a high false fault rate. This invention deconstructs the nonlinear coupling relationship of signals through kernel function mapping and relies on adaptive manifold regularization constraints to preserve the inherent temporal manifold characteristics of equipment fault signals, automatically suppressing interference from environmental noise and operational redundancy features. Actual industrial fault classification results show that the average fault classification accuracy of this invention reaches 96.45%, significantly improving noise resistance and classification stability compared to traditional WLMRKNN (90.12%) and SSL-KNN (91.36%) algorithms. It is suitable for online fault identification and condition monitoring scenarios of wind power equipment, motor bearings, and industrial transmission equipment, meeting the high real-time and robust classification requirements of industrial fields.

[0126] As a verification example of the present invention, the theoretical support and technical effect analysis are as follows:

[0127] The method of this invention has profound theoretical significance, namely the "Nearest NeighborGroup Effect".

[0128] Theorem: For two training samples belonging to the same class... and If they are similar to the test samples (i.e., ), and they are highly correlated in the manifold ( If the weights are relatively large, then the optimization objective function of this invention guarantees their weights. and They are also at similar heights, that is .

[0129] Proof approach:

[0130] Examine the manifold term in the objective function and probability regularization term This is a convex combination. According to convex optimization theory, when When the value is large, in order to minimize the objective function, the model tends to make... and Equal to reduce the penalty for differences.

[0131] This characteristic reveals the connection between this invention and Bayesian decision rules: weights In fact, it approximates the posterior probability. By summing the weights of the same category, we are actually calculating... This allows the model to approximate the maximum a posteriori (MAP) classifier. This is the fundamental reason why this invention has better generalization performance compared to the traditional KNN (which relies solely on hard truncation of distance).

[0132] In experimental verification, as shown in Tables 1 and 2, the method of this invention outperforms the comparison algorithms on multiple benchmark datasets such as Sonar, Iris, and Wine. Table 1: Average test accuracy of all methods on all datasets used:

[0133] Sonar 0.8262 0.8548 0.7643 0.8429 0.8857 0.8786 0.8929 0.9048 0.9071 Wine 0.9667 0.9861 0.9944 0.9833 0.9889 0.9917 0.9889 0.9944 0.9917 Iris 0.9600 0.9733 0.9533 0.9667 0.9733 0.9733 0.9800 0.9867 0.9733 Breast Cancer Wisconsin 0.9561 0.9754 0.9684 0.9719 0.9781 0.9807 0.9798 0.9851 0.9754 Pima Indians Diabetes 0.7247 0.7597 0.7649 0.7779 0.7682 0.7721 0.7753 0.7818 0.7948 Glass Identification 0.7047 0.7442 0.6209 0.7372 0.7512 0.7605 0.7651 0.7860 0.7977 Ionosphere 0.8629 0.9057 0.8857 0.8914 0.9186 0.9257 0.9357 0.9414 0.9529 Heart Disease 0.8148 0.8475 0.8377 0.8443 0.8623 0.8754 0.8705 0.8902 0.9115 Vowel 0.9051 0.9449 0.8942 0.9212 0.9621 0.9581 0.9601 0.9848 0.9747 Ecoli 0.8119 0.8507 0.7851 0.8448 0.8657 0.8716 0.8687 0.8910 0.9045 Yeast 0.5650 0.6051 0.5822 0.5980 0.6209 0.6350 0.6300 0.6519 0.6721 Pendigits 0.9720 0.9810 0.8950 0.9780 0.9920 0.9890 0.9900 0.9910 0.9950 Satimage 0.9060 0.9200 0.8420 0.9150 0.9350 0.9380 0.9420 0.9390 0.9470 Vehicle Silhouettes 0.7249 0.7852 0.6479 0.7621 0.8012 0.8148 0.8201 0.8450 0.8337 Letter Recognition 0.9420 0.9580 0.7650 0.9510 0.9650 0.9700 0.9720 0.9780 0.9830

[0134] Table 2 shows the average F1 scores of all adopted methods across all adopted datasets:

[0135] Sonar 0.8143 0.8476 0.7571 0.8357 0.8786 0.8714 0.8857 0.9000 0.9000 Wine 0.9639 0.9833 0.9944 0.9833 0.9889 0.9889 0.9861 0.9861 0.9944 Iris 0.9567 0.9733 0.9533 0.9667 0.9733 0.9733 0.9800 0.9867 0.9867 Breast Cancer Wisconsin 0.9482 0.9693 0.9623 0.9649 0.9719 0.9763 0.9754 0.9807 0.9860 Pima Indians Diabetes 0.6851 0.7299 0.7240 0.7448 0.7383 0.7448 0.7500 0.7578 0.7669 Glass Identification 0.6256 0.6953 0.5070 0.6814 0.7093 0.7256 0.7302 0.7558 0.7323 Ionosphere 0.8457 0.8914 0.8714 0.8786 0.9057 0.9143 0.9286 0.9357 0.9286 Heart Disease 0.7951 0.8328 0.8197 0.8279 0.8443 0.8607 0.8574 0.8754 0.8393 Vowel 0.8848 0.9318 0.9220 0.9081 0.9551 0.9480 0.9520 0.9798 0.9712 Ecoli 0.7522 0.8104 0.6851 0.7955 0.8254 0.8343 0.8299 0.8627 0.8761 Yeast 0.5051 0.5650 0.4822 0.5519 0.5801 0.6051 0.5949 0.6249 0.6360 ​ 0.9680 0.9790 0.8820 0.9750 0.9900 0.9860 0.9880 0.9890 0.9730 ​ 0.8850 0.9050 0.8150 0.8980 0.9200 0.9250 0.9300 0.9280 0.9470 ​ 0.7018 0.7680 0.6148 0.7450 0.7852 0.8018 0.8083 0.8320 0.8278 ​ 0.9380 0.9540 0.7420 0.9480 0.9610 0.9680 0.9700 0.9760 0.9670

[0136] Sonar dataset (high-dimensional sonar signals): KMOLNN achieves an accuracy of 81.20%, which is more than 2% higher than Elastic Net, demonstrating the importance of manifold preservation when processing high-dimensional nonlinear signals.

[0137] The Pendigits dataset (handwritten digit recognition): KMOLNN accuracy is 94.51% and F1 score is 93.36%, demonstrating extremely strong multi-class classification ability.

[0138] As an embodiment of the present invention, a nonlinear data classification system based on kernel manifold optimization is provided.

[0139] This embodiment provides a system for implementing the above method, which can be integrated into a server, workstation, or embedded high-performance computing device. The system includes:

[0140] The data acquisition module is used to acquire the test samples to be classified and the training sample set with category labels;

[0141] The kernel matrix calculation module, based on a preset kernel function, calculates the kernel matrix within the training sample set and the kernel vector between the test sample and the training sample, thereby realizing the implicit mapping of the test sample and the training sample set to the high-dimensional feature space.

[0142] The model optimization module constructs and solves a composite objective function that includes a kernelized reconstruction error term, a probability regularization term, a local constraint term, a manifold preservation regularization term, and an edge weight regularization term. It optimizes the variables—the probability vector and the adaptive adjacency matrix—through an alternating optimization strategy until the composite objective function converges and outputs the optimal probability vector.

[0143] The classification prediction module takes the optimal probability vector as input, calculates the sum of similarities between the test sample and each type of training sample in the feature space, and classifies the test sample into the category with the largest sum of similarities.

[0144] As an embodiment of the present invention, an electronic device

[0145] The present invention also provides an electronic device, comprising:

[0146] Processor: This can be a CPU, GPU, DSP, or FPGA. For the numerous matrix operations involved in this invention, a GPU supporting CUDA acceleration is recommended.

[0147] Memory: Used to store computer programs, training datasets, intermediate kernel matrices, and optimization parameters.

[0148] Communication interface: Used for data interaction with external systems.

[0149] Bus: Connects the above components.

[0150] When the processor executes a program in memory, it implements the following sequence of instructions:

[0151] Read the test samples and training set;

[0152] Call the kernel function to calculate the kernel matrix;

[0153] Initialize the weights and adjacency matrix;

[0154] Entering the loop:

[0155] a. Calculate the weight gradient, update the weights, and perform projection;

[0156] b. Calculate the gradient of the adjacency matrix, update the adjacency matrix, and perform projection;

[0157] c. Check convergence;

[0158] Output the classification results.

[0159] As an embodiment of the present invention, a computer-readable storage medium

[0160] The present invention also provides a non-volatile computer-readable storage medium (such as a hard disk, SSD, optical disk, or USB flash drive) storing computer instructions thereon. When the instructions are read and executed by a computer system, the computer system performs the steps of the method described in Embodiment 1.

[0161] This invention creatively combines kernel methods (to address nonlinearity), adaptive manifold regularization (to address structure preservation), and a probabilistic optimization framework (to address weight interpretability) to propose a complete KMOLNN classification scheme. This scheme not only theoretically addresses the assumptions and shortcomings of traditional linear KNN, but also demonstrates superior classification accuracy and robustness in practical applications. It is particularly suitable for fields with extremely high requirements for nonlinear processing, such as biometric recognition, complex financial analysis, and industrial fault diagnosis, and has broad application prospects and significant commercial value.

Claims

1. A nonlinear data classification method based on kernel manifold optimization, characterized in that: Includes the following steps: S1. Obtain the test samples to be classified and the training sample set with class labels, and define and initialize the probability vector and the adaptive adjacency matrix; where: the probability vector is the linear representation weight of each training sample to the test sample; the adaptive adjacency matrix is ​​used to describe the manifold structure between training samples; S2. Calculate the kernel similarity within the training sample set and between the test sample and the training sample set using kernel functions, construct the kernel matrix and kernel vector, and implicitly map the test sample and the training sample set to a high-dimensional feature space. S3. Establish a composite objective function with probability vectors and adaptive adjacency matrix as optimization variables; the composite objective function includes kernelized reconstruction error term, probability regularization term, local constraint term, manifold preservation regularization term, and edge weight regularization term; S4. Adopt an alternating optimization strategy to optimize the probability vector with a fixed adaptive adjacency matrix and optimize the adaptive adjacency matrix with a fixed probability vector until the composite objective function converges and the optimal probability vector is obtained. S5. Calculate the sum of similarities between the test sample and each type of training sample in the feature space based on the optimal probability vector, and classify the test sample into the category with the largest sum of similarity. 2.The nonlinear data classification method based on kernel manifold optimization of claim 1, wherein: The initialization involves initializing the probability vector as a uniform distribution vector; and initializing the adaptive adjacency matrix based on the clustering results of the pre-clustering of the training samples using the K-means clustering algorithm: if two training samples belong to the same cluster, their corresponding edge weights are initialized to 1; otherwise, they are initialized to 0 or a preset value of 10. -8 . 3.The nonlinear data classification method based on kernel manifold optimization according to claim 2, characterized in that: The kernel function is a Gaussian kernel function; the kernel matrix is ​​composed of the kernel similarity of all sample pairs within the training sample matrix; the kernel vector is a vector composed of the kernel similarity between the test sample and each sample in the training sample set.

4. The nonlinear data classification method based on kernel manifold optimization according to claim 3, characterized in that: The kernelized reconstruction error term is used to minimize the reconstruction error of the test sample in the feature space caused by the linear combination of the training samples, so as to ensure the accuracy of the linear representation.

5. The nonlinear data classification method based on kernel manifold optimization according to claim 4, characterized in that: The manifold preservation regularization term introduces a graph regularization term based on an adaptive Laplacian matrix to dynamically learn the adjacency relationships between training samples, so that the training samples can still maintain their local neighborhood structure on the original manifold after being mapped to the weight space.

6. The nonlinear data classification method based on kernel manifold optimization according to claim 5, characterized in that: The optimization process of the alternating optimization strategy includes the following steps: S61. With the adaptive adjacency matrix fixed, the probability vector is updated using gradient descent and projected onto the probability simplex space so that the elements of the probability vector are non-negative and their sum is 1. S62. With the probability vector fixed, the adaptive adjacency matrix is ​​updated using gradient descent and a non-negative projection is performed to make the edge weights non-negative. S63. Calculate the difference between the objective function value of the current iteration and the previous iteration. If the difference is less than the preset threshold or the maximum number of iterations is reached, stop the iteration; otherwise, return to step S61.

7. The nonlinear data classification method based on kernel manifold optimization according to claim 6, characterized in that: The method is applicable to sample sets with complex nonlinear structures, such as face recognition, medical image classification, or complex industrial signal classification.

8. A nonlinear data classification system based on kernel manifold optimization, characterized by: The system includes: The data acquisition module is used to acquire the test samples to be classified and the training sample set with category labels; The kernel matrix calculation module, based on a preset kernel function, calculates the kernel matrix within the training sample set and the kernel vector between the test sample and the training sample, thereby realizing the implicit mapping of the test sample and the training sample set to the high-dimensional feature space. The model optimization module constructs and solves a composite objective function that includes a kernelized reconstruction error term, a probability regularization term, a local constraint term, a manifold preservation regularization term, and an edge weight regularization term. It optimizes the variables—the probability vector and the adaptive adjacency matrix—through an alternating optimization strategy until the composite objective function converges and outputs the optimal probability vector. The classification prediction module takes the optimal probability vector as input, calculates the sum of similarities between the test sample and each type of training sample in the feature space, and classifies the test sample into the category with the largest sum of similarities.

9. An electronic device, comprising: It includes a processor and a memory, the processor being used to execute a computer program stored in the memory to implement a nonlinear data classification method based on kernel manifold optimization as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements a nonlinear data classification method based on kernel manifold optimization as described in any one of claims 1 to 7.

Images