A fully supervised local analysis text data dimension reduction method
By employing a fully supervised local analysis method, utilizing the K-nearest neighbor rule and local objective function decomposition, the problem of insufficient classification ability of unsupervised manifold learning algorithms in text dimensionality reduction is solved, achieving efficient data dimensionality reduction and improved classification capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGDONG POWER GRID CO LTD
- Filing Date
- 2022-11-17
- Publication Date
- 2026-05-15
AI Technical Summary
Existing unsupervised manifold learning algorithms have poor classification and discrimination capabilities in text dimensionality reduction and are difficult to handle external data.
A fully supervised local analysis method is adopted. The high-dimensional dataset is divided by the K-nearest neighbor rule, a fully supervised local objective function is constructed, the local permutation matrix is solved, and the global projection matrix is obtained by eigenvalue decomposition, so as to realize the mapping from high-dimensional data to low-dimensional data.
It improves the classification ability in the text dimensionality reduction process, effectively solves the problem of foreign samples, ensures that data points of the same type are close and data points of different types are separated, and enhances the system's distinguishing ability.
Smart Images
Figure CN115757781B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text dimensionality reduction technology, and in particular to a method for dimensionality reduction of text data based on fully supervised local analysis. Background Technology
[0002] With the continuous advancement of computer network and data storage technologies, massive amounts of information are collected and stored. Today, organizations, enterprises, and the internet have accumulated enormous amounts of data, and this volume continues to grow at an unprecedented rate. The pressure on computers to process this data is becoming increasingly immense, thus creating an urgent need for dimensionality reduction of text. This is beneficial for subsequent data mining tasks such as classification and clustering. Furthermore, since text contains a large amount of noise and redundant information, dimensionality reduction can eliminate this noise, effectively reducing the processing scale and significantly improving the speed and accuracy of classifiers. Therefore, research on text dimensionality reduction has considerable theoretical significance.
[0003] Dimensionality reduction algorithms are widely used in many fields. These algorithms are mainly divided into two parts: linear dimensionality reduction algorithms and nonlinear dimensionality reduction algorithms. Linear methods assume that the internal structure of the data follows a linear distribution. Among the many linear dimensionality reduction algorithms, PCA and LDA are the two most well-known. Although linear dimensionality reduction algorithms are easy to implement, they ignore the nonlinear structure of the data.
[0004] To address this, manifold learning emerged, its fundamental assumption being that high-dimensional data samples are embedded on a smooth manifold in a low-dimensional Euclidean space. Based on this crucial assumption, the goal of manifold learning methods is to map high-dimensional data to a low-dimensional space, allowing the low-dimensional data to reflect the structural features of the original high-dimensional data. Numerous manifold learning algorithms have been proposed, such as LLE, ISOMAP, LE, and LTSA. These algorithms can effectively preserve data characteristics for nonlinear manifolds. However, these manifold-based dimensionality reduction algorithms are unsupervised, resulting in poor discriminative ability for classification. Furthermore, popular learning algorithms struggle with external data challenges. Summary of the Invention
[0005] This invention provides a fully supervised local analysis text data dimensionality reduction method, which solves the technical problems of poor classification ability and difficulty in handling external data.
[0006] In view of this, the present invention provides a method for dimensionality reduction of text data based on fully supervised local analysis, comprising the following steps:
[0007] Obtain text data and construct a high-dimensional dataset;
[0008] Based on the K-nearest neighbor rule, find the k nearest points of each high-dimensional sample point in the high-dimensional dataset to form the nearest neighbor local area;
[0009] A fully supervised local objective function is constructed based on a fully supervised local analysis algorithm, and the local permutation matrix is solved.
[0010] The global permutation matrix is obtained by using the local permutation matrix, and the fully supervised local objective function is simplified to obtain the simplified fully supervised local objective function.
[0011] Based on the fully supervised local projection algorithm, the global projection matrix is obtained by performing eigenvalue decomposition on the simplified fully supervised local objective function, and the low-dimensional data after dimensionality reduction of the high-dimensional dataset is solved based on the global projection matrix.
[0012] Preferably, the step of finding the k nearest points to each high-dimensional sample point in the high-dimensional dataset based on the K-nearest neighbor rule to form a local nearest neighbor specifically includes:
[0013] Let the high-dimensional dataset be X = [x1…x2] N ]∈R D×N x i ∈R D (i = 1, ..., N), where x i Let X represent the i-th sample point, N represent the number of samples, R represent all real numbers, and D represent the dimension. Let the dataset after dimensionality reduction from the high-dimensional dataset X be Y = [y1…y2]. i …y N ]∈R d×N y i Let d represent the i-th sample point, where d << D;
[0014] Based on the K-nearest neighbor rule, the Euclidean distance between each sample point in the high-dimensional dataset and other sample points is calculated sequentially. All sample points corresponding to the center point are then sorted in ascending order according to their Euclidean distances. The first k sample points are selected to form the corresponding nearest neighbor local area, denoted as . x ik Indicates x i The k-th nearest neighbor local area is defined using Euclidean distance from the center point.
[0015] Preferably, the steps of constructing a fully supervised local objective function based on a fully supervised local analysis algorithm and solving for the local permutation matrix specifically include:
[0016] The fully supervised local objective function is constructed based on the fully supervised local analysis algorithm as follows:
[0017]
[0018] In the formula, c represents the nearest neighbor local data points with c categories, and N l Indicates the number of class l. This represents the mean vector of all points in the q-th class. Let $\mathbf{p}$ be the mean vector of all points in the $p$-th class, where $\mathbf{p}$.
[0019]
[0020] In the formula, T represents the transpose sign, and N q Y represents the number of samples with local class q. i =YS i , among which, S i To select the matrix, S i ∈R N×N e q This represents the location of the sample point of class q, where,
[0021]
[0022] In the formula, N q N represents the number of classes q. c This indicates the number of classes c, and k+1 represents the size of the local area; e q Let x be a row vector of size k+1, where all positions in category q are 1. Then the sample point x i Location of similar sample points for,
[0023]
[0024] The position e of the q-class sample points q and sample point x i Location of similar sample points Substituting into the fully supervised local objective function and simplifying it, we obtain the following:
[0025]
[0026] The local permutation matrix L is obtained through a fully supervised local objective function. i for,
[0027]
[0028] Preferably, the step of obtaining the global permutation matrix through the local permutation matrix and simplifying the fully supervised local objective function to obtain the simplified fully supervised local objective function specifically includes: obtaining the global permutation matrix from the local permutation matrix using the following formula,
[0029]
[0030] The local permutation matrix L i Substituting this into the simplified fully supervised local objective function, we obtain the relay simplified fully supervised local objective function as follows:
[0031]
[0032] Substituting the global permutation matrix L into the simplified fully supervised local objective function, we obtain the simplified fully supervised local objective function as follows:
[0033]
[0034] Preferably, based on the fully supervised local projection algorithm, the steps of obtaining the global projection matrix by performing eigenvalue decomposition on the simplified fully supervised local objective function, and solving for the low-dimensional data after dimensionality reduction of the high-dimensional dataset based on the global projection matrix specifically include:
[0035] Based on the fully supervised local projection algorithm, the simplified fully supervised local objective function is simplified to obtain the projection simplified objective function as follows:
[0036]
[0037] In the formula, Y = W T X, W∈R D×d W is the global projection matrix;
[0038] Setting W T W = I, where I represents the identity matrix, then by taking XLX T The eigenvectors corresponding to the first d largest eigenvalues are the global projection matrix W;
[0039] Based on the global projection matrix W, through Y = W T X is the high-dimensional dataset X reduced to low-dimensional data Y.
[0040] As can be seen from the above technical solutions, the present invention has the following advantages:
[0041] This invention segments high-dimensional datasets using the K-nearest neighbor rule, forming multiple overlapping local segments. By exploring the geometric information of different categories of data within these local neighborhoods, a fully supervised local objective function is constructed using a fully supervised local analysis algorithm. This algorithm solves for the local permutation matrix, ensuring that similar data points within a local segment are as close as possible to each other and different classes are as far apart as possible during dimensionality reduction. This effectively learns the geometric structure of local neighbors, improving the system's classification ability. Furthermore, by comprehensively considering the geometric permutation information of each local segment, the global projection matrix is obtained using eigenvalue calculation. Consequently, the low-dimensional data after dimensionality reduction from the high-dimensional dataset can be obtained, effectively solving the problem of foreign samples. Attached Figure Description
[0042] Figure 1 A flowchart of a fully supervised local analysis text data dimensionality reduction method provided in an embodiment of the present invention. Detailed Implementation
[0043] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0044] For easier understanding, please refer to Figure 1 This invention provides a method for dimensionality reduction of text data based on fully supervised local analysis, comprising the following steps:
[0045] S1. Obtain text data and construct a high-dimensional dataset;
[0046] S2. Based on the K-nearest neighbor rule, find the k nearest points of each high-dimensional sample point in the high-dimensional dataset to form the nearest neighbor local area.
[0047] Among them, the K-nearest neighbor rule is used to classify the sample points.
[0048] S3. Construct a fully supervised local objective function based on the fully supervised local analysis algorithm, and solve for the local permutation matrix.
[0049] It should be noted that since high-dimensional data is divided into local segments using KNN, and because it is a fully supervised local segment, meaning the categories of the sample data are clear, the fully supervised local segmentation algorithm aims to make the data after dimensionality reduction as dispersed as possible between different categories and as close as possible between samples of the same category.
[0050] Among them, the local permutation matrix is a process of rearranging, representing a permutation matrix from high-dimensional to low-dimensional.
[0051] S4. Obtain the global permutation matrix through the local permutation matrix, and simplify the fully supervised local objective function to obtain the simplified fully supervised local objective function.
[0052] S5. Based on the fully supervised local projection algorithm, the global projection matrix is obtained by performing eigenvalue decomposition on the simplified fully supervised local objective function, and the low-dimensional data after dimensionality reduction of the high-dimensional dataset is solved based on the global projection matrix.
[0053] This invention provides a fully supervised local analysis-based text data dimensionality reduction method. It segments a high-dimensional dataset using the K-nearest neighbor rule, forming multiple overlapping local segments. By exploring the geometric information of different categories of data within these local neighborhoods, a fully supervised local objective function is constructed using a fully supervised local analysis algorithm. This algorithm solves for the local permutation matrix, ensuring that similar data points within a local segment are as close as possible to each other, while different classes are separated as much as possible. This effectively learns the geometric structure of local neighbors, improving the system's classification ability. Finally, by comprehensively considering the geometric permutation information of each local segment, an eigenvalue calculation method is used to obtain the global projection matrix. This allows for the calculation of the low-dimensional data after dimensionality reduction from the high-dimensional dataset, effectively addressing the problem of foreign samples.
[0054] In one specific embodiment, step S2 specifically includes:
[0055] S201. Let the high-dimensional dataset be X = [x1...x2...x3...x4...x5...x6...x7 ... N ]∈R D×N x i ∈R D (i = 1, ..., N), where x i Let X represent the i-th sample point, N represent the number of samples, R represent all real numbers, and D represent the dimension. Let the dataset after dimensionality reduction from the high-dimensional dataset X be Y = [y1…y2]. i …y N ]∈R d×N y i Let d represent the i-th sample point, where d << D;
[0056] S202. Based on the K-nearest neighbor rule, calculate the Euclidean distance between each sample point in the high-dimensional dataset and other sample points, taking each sample point as the center point. Sort all sample points corresponding to the center point in ascending order according to their Euclidean distances, and take the first k sample points to form the corresponding nearest neighbor local area, denoted as . x ik Indicates x i The k-th nearest neighbor local area is defined using Euclidean distance from the center point.
[0057] In one specific embodiment, step S3 specifically includes:
[0058] S301. Based on the fully supervised local analysis algorithm, the fully supervised local objective function is constructed as follows:
[0059]
[0060] In the above formula, the first term represents making neighboring points of the same class as close as possible in low dimension, while the second term represents making the distance between sample data of different classes as large as possible.
[0061] In the formula, c represents the nearest neighbor local data points with c categories, and N l Indicates the number of class l. This represents the mean vector of all points in the q-th class. Let $\mathbf{p}$ be the mean vector of all points in the $p$-th class, where $\mathbf{p}$.
[0062]
[0063] In the formula, T represents the transpose sign, and N q Y represents the number of samples with local class q. i =YS i , among which, S i To select the matrix, S i ∈R N×N e q This represents the location of the sample point of class q, where,
[0064]
[0065] In the formula, N q N represents the number of classes q. c This indicates the number of classes c, and k+1 represents the size of the local area; e q Let x be a row vector of size k+1, where all positions in category q are 1. Then the sample point x i Location of similar sample points for,
[0066]
[0067] S302, Position the q-class sample points e q and sample point x i Location of similar sample points Substituting into the fully supervised local objective function and simplifying it, we obtain the following:
[0068]
[0069] S303. Obtain the local permutation matrix L through a fully supervised local objective function. i for,
[0070]
[0071] In one specific embodiment, step S4 specifically includes:
[0072] S401. The global permutation matrix is obtained from the local permutation matrix using the following formula:
[0073]
[0074] The above formula takes into account the overall structure, so the global permutation matrix is obtained by performing operations on the local permutation matrix.
[0075] S402, arrange the local permutation matrix L i Substituting this into the simplified fully supervised local objective function, we obtain the relay simplified fully supervised local objective function as follows:
[0076]
[0077] S403. Substituting the global permutation matrix L into the relay simplified fully supervised local objective function, we obtain the simplified fully supervised local objective function as follows:
[0078]
[0079] In one specific embodiment, step S5 specifically includes:
[0080] S501. Based on the fully supervised local projection algorithm, the simplified fully supervised local objective function is simplified to obtain the projection simplified objective function.
[0081]
[0082] In the formula, Y = W T X, W∈R D×d W is the global projection matrix;
[0083] S502, Setting W T W = I, where I represents the identity matrix, then by taking XLX T The eigenvectors corresponding to the first d largest eigenvalues are the global projection matrix W;
[0084] S503, Based on the global projection matrix W, through Y = W T X is the high-dimensional dataset X reduced to low-dimensional data Y.
[0085] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A dimensionality reduction method for text data based on fully supervised local analysis, characterized in that, Includes the following steps: Obtain text data and construct a high-dimensional dataset; Based on the K-nearest neighbor rule, find the nearest neighbors of each high-dimensional sample point in the high-dimensional dataset. A set of points forms the nearest neighbor local area, including: Let the high-dimensional dataset be... , ,in, Let X represent the i-th sample point, N represent the number of samples, R represent all real numbers, and D represent the dimension. Let the high-dimensional dataset X be the dataset after dimensionality reduction. , Let i represent the i-th sample point, where... ; Based on the K-nearest neighbor rule, the Euclidean distance between each sample point in the high-dimensional dataset and other sample points is calculated sequentially. All sample points corresponding to the center point are then sorted in ascending order according to their Euclidean distances. The first k sample points are selected to form the corresponding nearest neighbor local area, denoted as . , Indicates Divide the digits by Euclidean distance from the center point. The nearest neighbor local area; A fully supervised local objective function is constructed based on a fully supervised local analysis algorithm, and the local permutation matrix is solved, including: The fully supervised local objective function is constructed based on the fully supervised local analysis algorithm as follows: In the formula, Represents the nearest neighbor local area Types of data points, Indicates the number of class l. This represents the mean vector of all points in the q-th class. Let $\mathbf{p}$ be the mean vector of all points in the $p$-th class, where $\mathbf{p}$. In the formula, T represents the transpose sign. This represents the number of samples with the local class q. , among which, S i To select a matrix, e q This represents the location of the sample point of class q, where, In the formula, Indicates the number of classes q. Indicates the number of class C. Indicates the size of a local area; It is A row vector of size, in category If all positions are 1, then the sample point Location of similar sample points for, The position e of the q-class sample points q and sample points Location of similar sample points Substituting into the fully supervised local objective function and simplifying it, we obtain the following: The local permutation matrix is obtained through a fully supervised local objective function. for, ； The global permutation matrix is obtained by using the local permutation matrix, and the fully supervised local objective function is simplified to obtain the simplified fully supervised local objective function. Based on the fully supervised local projection algorithm, the global projection matrix is obtained by performing eigenvalue decomposition on the simplified fully supervised local objective function, and the low-dimensional data after dimensionality reduction of the high-dimensional dataset is solved based on the global projection matrix.
2. The method for dimensionality reduction of text data based on fully supervised local analysis according to claim 1, characterized in that, The steps of obtaining the global permutation matrix through the local permutation matrix and simplifying the fully supervised local objective function to obtain the simplified fully supervised local objective function specifically include: The global permutation matrix is obtained from the local permutation matrix using the following formula: Local permutation matrix Substituting this into the simplified fully supervised local objective function, we obtain the relay simplified fully supervised local objective function as follows: Substituting the global permutation matrix L into the simplified fully supervised local objective function, we obtain the simplified fully supervised local objective function as follows: 。 3. The method for dimensionality reduction of text data based on fully supervised local analysis according to claim 2, characterized in that, Based on the fully supervised local projection algorithm, the steps of obtaining the global projection matrix by performing eigenvalue decomposition on the simplified fully supervised local objective function, and solving for the low-dimensional data after dimensionality reduction of the high-dimensional dataset based on the global projection matrix specifically include: Based on the fully supervised local projection algorithm, the simplified fully supervised local objective function is simplified to obtain the projection simplified objective function as follows: In the formula, , W is the global projection matrix; set up Let I represent the identity matrix, then by taking... forward The eigenvectors corresponding to the largest eigenvalues are the global projection matrices. ; Based on the global projection matrix pass Obtain high-dimensional dataset Low-dimensional data after dimensionality reduction .