High-dimensional data dimension reduction system and method based on semi-supervised local adaptive discriminant analysis

Abstract

The invention belongs to the technical field of artificial intelligence, and discloses a high-dimensional data dimension reduction system and method for semi-supervised local adaptive discriminant analysis, and the method comprises the following specific steps: 1), learning two independent projection matrixes for labeled data and unlabeled data; 2) setting residual constraints through the two projections to approach each other, and jointly converging to a unified low-dimensional space capable of distinguishing categories and reflecting an internal structure of data; (3) the projection of the labeled data can keep category discrimination information to the greatest extent, and the projection of the non-labeled data can keep the global manifold structure of the data optimally; the method is realized through a unified and solvable optimization objective function, and the method simultaneously utilizes marked and unmarked data, adaptively captures a local structure, is high in anti-noise capability, is suitable for semi-supervised high-dimensional data processing scenes such as face / handwritten digital recognition and gene data analysis, and shows remarkable advantages in classification on a standard data set.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, machine learning, data mining, and pattern recognition. In particular, it relates to a high-dimensional data dimensionality reduction method that combines a labeled-unlabeled double transformation matrix with adaptive intra-class divergence in a semi-supervised scenario. It is also applicable to high-dimensional data processing scenarios where labeled samples are scarce and unlabeled samples are abundant, such as computer vision, bioinformatics, and pattern recognition. Background Technology

[0002] In the era of big data, high-dimensional data (such as image features and gene expression data with dimensions of thousands or even tens of thousands) are widespread. However, the "curse of dimensionality" leads to increased data sparsity and a surge in computational complexity, severely impacting the performance of subsequent machine learning models. Dimensionality reduction techniques, by mapping high-dimensional data to a low-dimensional subspace, reduce data dimensionality while preserving key information, and have become a core means of solving this problem.

[0003] Among existing dimensionality reduction methods, supervised methods (such as Linear Discriminant Analysis (LDA) rely on a large number of labeled samples, resulting in limited performance in scenarios where labeled data is scarce; unsupervised methods (such as Principal Component Analysis (PCA)) do not utilize class information, leading to insufficient discriminative power of the reduced features; semi-supervised methods, while combining labeled and unlabeled data, mostly ignore the local structure of the data or employ fixed neighborhood relationships (such as kNN), making it difficult to adaptively capture the inherent manifold distribution of the data. Traditional methods typically calculate the intra-class scatter matrix based on the original spatial distance, neglecting the local structure in the projected subspace, leading to a difficulty in balancing the local compactness and inter-class discriminative power of the reduced data.

[0004] Furthermore, existing semi-supervised dimensionality reduction methods (such as Semi-Supervised Discriminant Analysis (SDA)) typically preserve the manifold structure of unlabeled data through graph Laplacian regularization. However, this approach essentially seeks a single projection that is a "compromise" between the two types of data. When the distributions of labeled and unlabeled data differ, or when their optimal projection directions are inconsistent, a single projection may not simultaneously and optimally preserve discriminative and structural properties. Therefore, a more flexible framework is urgently needed that can integrate semi-supervised information, adaptively capture local structure, and balance discriminative and local preservation in high-dimensional data dimensionality reduction, thereby obtaining superior feature representations. Summary of the Invention

[0005] To address the technical problems existing in the prior art, this invention provides a method and system for dimensionality reduction of high-dimensional data using semi-supervised local adaptive discriminant analysis. By constructing a locally adaptive intra-class scatter matrix and an objective function that fuses semi-supervised information, effective dimensionality reduction of high-dimensional data is achieved.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A high-dimensional data dimensionality reduction method using semi-supervised local adaptive discriminant analysis, S1: Data preprocessing: obtaining the original high-dimensional data matrix. , including the labeled data matrix and unlabeled data matrix .in , , Let these represent the total number of samples, the number of labeled samples, and the number of unlabeled samples, respectively. The dimension of the total number of samples is... All samples are standardized and normalized to eliminate scale differences between different feature dimensions and obtain a unified metric space.

[0008] S2: Construct a semi-supervised local structure weight matrix. Calculate the local similarity weights between samples based on the category information of the labeled samples.

[0009] S3: Calculate the discriminant divergence matrix, including:

[0010] (1) Total scatter matrix based on labeled samples To characterize the dispersion of labeled samples ,in .

[0011] (2) Total scatter matrix based on unlabeled samples To characterize the dispersion of unlabeled samples, ,in .

[0012] (3) The local adaptive intra-class scatter matrix is ​​expressed as follows: Where c is the number of categories, Let be the number of labeled samples in class i. , For the j-th and k-th labeled samples in the i-th class, for and Local similarity weights;

[0013] S4: Construct the objective function to maximize trace ratio

[0014] (1) Construct the numerator of the objective function: ;

[0015] (2) Construct the denominator of the objective function: ,in and For regularization parameters, For unit array;

[0016] (3) Construct the trace ratio maximization objective function

[0017] (3);

[0018] S5: Solve the objective function using an alternating iterative optimization algorithm.

[0019] (1) Fix the weight matrix of the tth time. Calculate the adaptive within-class scatter matrix with original distance weighting. ;

[0020] (2) Transform the objective function value into a generalized eigenvalue problem:

[0021] (4);

[0022] By using eigenvalue decomposition, the first... The eigenvectors corresponding to the largest eigenvalue are used to construct projection matrices, thus obtaining the th largest eigenvalue. Projection matrix in the next iteration and ;

[0023] (3) Fixed and Calculate the subspace distance of labeled samples Update # Secondary weight:

[0024] (5);

[0025] S6: Convergence Judgment: Calculate the... The objective function value of the next iteration ,like ( If the convergence threshold is reached or the maximum number of iterations is reached, the iteration stops, and the optimal projection matrix is ​​output. and .

[0026] S7: Data Dimensionality Reduction: Labeled data is mapped to... using a projection matrix. and For each unlabeled sample, find k nearest neighbors in the low-dimensional space containing labeled samples, and assign pseudo-labels through voting.

[0027] A semi-supervised local adaptive discriminant analysis system for dimensionality reduction of high-dimensional data that implements the above method includes:

[0028] 1. Data Input and Preprocessing Module: This module receives high-dimensional raw data, performs data cleaning, format unification, and basic preprocessing, and provides high-quality input data, including labeled and unlabeled data, for subsequent modules, thus resolving redundancy and format inconsistency issues in high-dimensional data.

[0029] 2. The supervised information local structure learning module constructs an intra-class local graph (connecting only labeled samples of the same class) based on the category labels of the labeled samples, calculates the similarity weights of intra-class samples (e.g., through adaptive adjustment via distance), and preserves intra-class compactness.

[0030] 3. Discriminative divergence matrix calculation module, used to calculate the total divergence of labeled samples, the total divergence of unlabeled samples, and the local adaptive intra-class divergence matrix.

[0031] 4. Objective function construction module, used to integrate the divergence matrix, weight matrix and projection matrix to construct the objective function in trace ratio form with the objectives of "semi-supervised discriminability" and "local adaptation", and to clarify the optimization direction of "maximizing discriminative information".

[0032] 5. The objective function optimization module transforms the trace ratio objective function into a solvable mathematical problem. Through alternating iterative optimization of the projection matrix and weight matrix, it achieves convergence of the objective function and outputs the optimal projection matrix. , and weight matrix .

[0033] 6. High-dimensional data dimensionality reduction module: Used to map high-dimensional data to a low-dimensional subspace using the optimal projection matrix and output the dimensionality reduction result.

[0034] 7. Results Output and Evaluation Module: Used to output low-dimensional features and evaluate the dimensionality reduction effect through quantitative indicators.

[0035] The semi-supervised local adaptive discriminant analysis method and system for dimensionality reduction of high-dimensional data in this invention have the following advantages:

[0036] (1) Semi-supervised information deep fusion: It can maintain good performance even in scenarios where labeled data is scarce by simultaneously utilizing the category information of labeled samples and the distribution characteristics of unlabeled samples.

[0037] (2) Local adaptive structure learning: By iteratively optimizing the local weight matrix, the intra-class divergence calculation is adapted to the structure of the projected subspace, thereby improving the ability to preserve local compactness.

[0038] (3) Balance between discriminability and stability: The objective function simultaneously maximizes the inter-class difference and minimizes the intra-class scatter matrix, and combines regularization terms to avoid overfitting, thereby improving the discriminability and numerical stability of the dimensionality reduction features. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a semi-supervised local adaptive discriminant analysis method for dimensionality reduction of high-dimensional data, provided in an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of a semi-supervised local adaptive discriminant analysis high-dimensional data dimensionality reduction system provided in an embodiment of the present invention. Detailed Implementation

[0042] Figure 1 This invention provides a flowchart of a semi-supervised local adaptive discriminant analysis method for dimensionality reduction of high-dimensional data, as provided in an embodiment of the invention. The embodiment of the invention provides a semi-supervised local adaptive discriminant analysis method for dimensionality reduction of high-dimensional data, including:

[0043] S1. Data Preprocessing: Obtaining the Original High-Dimensional Data Matrix , including the labeled data matrix and unlabeled data matrix .in , , Let these represent the total number of samples, the number of labeled samples, and the number of unlabeled samples, respectively. The dimension of the total number of samples is... All samples are standardized and normalized to eliminate scale differences between different feature dimensions and obtain a unified metric space.

[0044] S2. Construct a semi-supervised local structure weight matrix. Calculate the local similarity weights between samples based on the category information of the labeled samples.

[0045] S3. Calculate the discriminant divergence matrix, including:

[0046] (1) Based on the total scatter matrix of the labeled samples, characterize the dispersion of the labeled samples. ;

[0047] (2) Based on the total scatter matrix of unlabeled samples, characterize the dispersion of unlabeled samples. ;

[0048] (3) The local adaptive intra-class scatter matrix is ​​expressed as follows: Where c is the number of categories, Let be the number of labeled samples in class i. , For the j-th and k-th labeled samples in the i-th class, for and Local similarity weights.

[0049] S4. Construct the trace ratio maximization objective function:

[0050] (1) Construct the numerator of the objective function: ;

[0051] (2) Construct the denominator of the objective function: ,in and For regularization parameters, For unit array;

[0052] (3) Construct the trace ratio maximization objective function:

[0053] (6)

[0054] S5. Solve the objective function using an alternating iterative optimization algorithm.

[0055] (1) Fix the weight matrix of the tth time. Calculate the adaptive within-class scatter matrix with original distance weighting. ;

[0056] (2) Transform the objective function value into a generalized eigenvalue problem:

[0057] (7)

[0058] By using eigenvalue decomposition, the first... The eigenvectors corresponding to the largest eigenvalue are used to construct projection matrices, thus obtaining the th largest eigenvalue. Projection matrix in the next iteration and ;

[0059] (3) Fixed and Calculate the subspace distance of labeled samples Update # Secondary weight

[0060] (8);

[0061] S6. Convergence Judgment: Calculate the... The objective function value of the next iteration ,like ( If the convergence threshold is reached or the maximum number of iterations is reached, the iteration stops, and the optimal projection matrix is ​​output. and .

[0062] S7. Data Dimensionality Reduction: Labeled data is mapped to... using a projection matrix. and For each unlabeled sample, find k nearest neighbors in the low-dimensional space containing labeled samples, and assign pseudo-labels by voting using the KNN algorithm.

[0063] Figure 2 This is a schematic diagram of a semi-supervised local adaptive discriminant analysis high-dimensional data dimensionality reduction system provided in an embodiment of the present invention; the embodiment of the present invention provides a semi-supervised local adaptive discriminant analysis high-dimensional data dimensionality reduction system, including:

[0064] 1. Data Input and Preprocessing Module: This module receives high-dimensional raw data, performs data cleaning, format unification, and basic preprocessing, and provides high-quality input data, including labeled and unlabeled data, for subsequent modules, thus resolving redundancy and format inconsistency issues in high-dimensional data.

[0065] 2. The supervised information local structure learning module constructs an intra-class local graph (connecting only labeled samples of the same class) based on the category labels of the labeled samples, calculates the similarity weights of intra-class samples (e.g., through adaptive adjustment via distance), and preserves intra-class compactness.

[0066] 3. Discriminative divergence matrix calculation module, used to calculate the total divergence of labeled samples, the total divergence of unlabeled samples, and the local adaptive intra-class divergence matrix;

[0067] 4. Objective function construction module, used to integrate the divergence matrix, weight matrix and projection matrix to construct the objective function in trace ratio form with the objectives of "semi-supervised discriminability" and "local adaptation", and to clarify the optimization direction of "maximizing discriminative information".

[0068] 5. The objective function optimization module transforms the trace ratio objective function into a solvable mathematical problem. Through alternating iterative optimization of the projection matrix and weight matrix, it achieves convergence of the objective function and outputs the optimal projection matrix. , and weight matrix .

[0069] 6. High-dimensional data dimensionality reduction module: Used to map high-dimensional data to a low-dimensional subspace using the optimal projection matrix and output the dimensionality reduction result;

[0070] 7. Results Output and Evaluation Module: This module outputs low-dimensional features, evaluates the dimensionality reduction effect using quantitative indicators, and labels unlabeled data.

[0071] Experiments and Analysis

[0072] A simulation experiment was conducted using a semi-supervised local adaptive discriminant analysis method for dimensionality reduction of high-dimensional data, as proposed in this invention. This example was performed using MATLAB software on a Windows 11 operating system with an Intel Core i7-12700K CPU and 32GB of memory. The MSRA25 and CMUPIE datasets were used, and their information is shown in Table 1. 50% of the data from each class was randomly selected as training samples, and the remaining portion was used as test samples. In the training samples, 10 samples from each class were randomly selected as labeled samples, and the rest were used as unlabeled samples. Accuracy (ACC) was used to evaluate the classification performance, and the experimental results are shown in Table 2. As can be seen from Table 2, the classification performance of this invention achieved high-quality results.

[0073] Table 1 Dataset Information Table

[0074]

[0075] Table 2 ACC Results Table

[0076]

[0077] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0078] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A high-dimensional data dimensionality reduction system for semi-supervised local adaptive discriminant analysis, characterized in that, include: The first module, Data Input and Preprocessing, receives high-dimensional raw data, performs data cleaning, format unification, and basic preprocessing, and provides high-quality input data, including labeled and unlabeled data, for subsequent modules, thus solving the problems of redundancy and inconsistent formats in high-dimensional data. II. The supervised information local structure learning module constructs an intra-class local graph (connecting only labeled samples of the same class) based on the category labels of the labeled samples, calculates the similarity weights of intra-class samples (e.g., through adaptive adjustment by distance), and preserves intra-class compactness. III. Discriminant divergence matrix calculation module, used to calculate the total divergence of labeled samples, the total divergence of unlabeled samples, and the local adaptive intra-class divergence matrix; IV. Objective Function Construction Module: This module integrates the divergence matrix, weight matrix, and projection matrix to construct an objective function in trace ratio form, aiming at "semi-supervised discriminability" and "local adaptation," thus clarifying the optimization direction of "maximizing discriminative information." V. Objective Function Optimization Module: This module transforms the trace ratio objective function into a solvable mathematical problem. By iteratively optimizing the projection matrix and weight matrix, it achieves convergence of the objective function and outputs the optimal projection matrix. , and weight matrix ; VI. High-dimensional data dimensionality reduction module, which uses the optimal projection matrix to map high-dimensional data to a low-dimensional subspace and outputs the dimensionality reduction result; VII. Result Output and Evaluation Module: This module outputs low-dimensional features and evaluates the dimensionality reduction effect using quantitative indicators.

2. A method for dimensionality reduction of high-dimensional data using semi-supervised local adaptive discriminant analysis, characterized by utilizing the system described in claim 1, with the following specific steps: S1. Data Preprocessing: Obtaining the Original High-Dimensional Data Matrix , including the labeled data matrix and unlabeled data matrix .in , , Let these represent the total number of samples, the number of labeled samples, and the number of unlabeled samples, respectively. The dimension of the total number of samples is... All samples are standardized and normalized to eliminate scale differences between different feature dimensions and obtain a unified metric space. S2. Construct a semi-supervised local structure weight matrix, and calculate the local similarity weights between samples based on the category information of the labeled samples; S3. Calculate the discriminant divergence matrix, including: (1) Based on the total scatter matrix of the labeled samples, characterize the dispersion of the labeled samples. ; (2) Based on the total scatter matrix of unlabeled samples, characterize the dispersion of unlabeled samples. ; (3) The local adaptive intra-class scatter matrix is ​​expressed as follows: Where c is the number of categories, Let be the number of labeled samples in class i. , For the j-th and k-th labeled samples in the i-th class, for and Local similarity weights; S4. Construct the objective function to maximize the trace ratio; (1) Construct the numerator of the objective function: ; (2) Construct the denominator of the objective function: ,in and For regularization parameters, For unit array; (3) Construct the trace ratio maximization objective function （1）； S5. Solve the objective function using an alternating iterative optimization algorithm; (1) Fix the weight matrix of the tth time. Calculate the adaptive within-class scatter matrix with original distance weighting. ; (2) Transform the objective function value into a generalized eigenvalue problem: （2）； By using eigenvalue decomposition, the first... The eigenvectors corresponding to the largest eigenvalue are used to construct projection matrices, thus obtaining the th largest eigenvalue. Projection matrix in the next iteration and ; (3) Fixed and Calculate the subspace distance of labeled samples Update # Secondary weight ; S6. Convergence Judgment: Calculate the... The objective function value of the next iteration ,like ( If the convergence threshold is reached or the maximum number of iterations is reached, the iteration stops, and the optimal projection matrix is ​​output. and ; S7. Data Dimensionality Reduction: Labeled data is mapped to... using a projection matrix. and For each unlabeled sample, find k nearest neighbors in the low-dimensional space containing labeled samples, and assign pseudo-labels by voting.

3. The high-dimensional data dimensionality reduction method of semi-supervised local adaptive discriminant analysis according to claim 2, characterized in that, The high-dimensional data mentioned in step S1 includes, but is not limited to, facial image data, handwritten digit image data, and gene expression data, with the ratio of labeled samples to unlabeled samples being 1:5 to 1:

100.

4. The high-dimensional data dimensionality reduction method of semi-supervised local adaptive discriminant analysis according to claim 2, characterized in that, The preprocessing process in step S1 includes feature normalization (using methods such as Z-Score or Min-Max).

5. The high-dimensional data dimensionality reduction method of semi-supervised local adaptive discriminant analysis according to claim 2, characterized in that, The method for initializing local similarity weights in step S2 is as follows: ,in .

6. The high-dimensional data dimensionality reduction method of semi-supervised local adaptive discriminant analysis according to claim 2, characterized in that, In step S3, (1) Total scatter matrix of labeled samples The centralized matrix , To label the number of samples; (2) Total scatter matrix of unlabeled samples Centralization , This represents the number of unlabeled samples.

7. The high-dimensional data dimensionality reduction method of semi-supervised local adaptive discriminant analysis according to claim 2, characterized in that, In step S4 and The range of values ​​is .

8. The high-dimensional data dimensionality reduction method of semi-supervised local adaptive discriminant analysis according to claim 2, characterized in that, In step S4, when maximizing the trace ratio objective function, since the objective function affects the projection matrix... and The scaling remains invariant. Normalization constraints are introduced to simplify the objective function, which is then solved using generalized eigenvalue decomposition, transforming the objective function into a generalized eigenvalue problem. Eigenvalue decomposition is performed on the eigenvalues, selecting the first... The eigenvectors corresponding to the largest eigenvalues ​​constitute the optimal... ,in The dimension for dimensionality reduction; Decomposition yields: Projection matrix : The former Rows; projection matrix : After OK.

9. The high-dimensional data dimensionality reduction method of semi-supervised local adaptive discriminant analysis according to claim 2, characterized in that, In step S5, when solving the generalized eigenvalue problem, 1e is added to the denominator matrix. -9 Perturbations are used to ensure positive definiteness and avoid numerical singularities.

10. The high-dimensional data dimensionality reduction method of semi-supervised local adaptive discriminant analysis according to claim 2, characterized in that, In step S5 The range of values ​​is The maximum number of iterations is 30 to 100.

Images