A method and system for unsupervised feature selection based on multi-clustering

Abstract

The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for unsupervised feature selection based on multi-clustering. The selection method comprises: obtaining a plurality of data points to be clustered in a high-dimensional space to obtain a feature matrix; embedding the feature matrix in the high-dimensional space into a low-dimensional space using an isometric mapping algorithm to obtain an embedding matrix in the low-dimensional space; normalizing the feature matrix to obtain a reference feature matrix; fitting a feature vector composed of features in each dimension of the embedding matrix using the reference feature matrix and the embedding matrix to obtain a sparse coefficient vector of each feature vector in the embedding matrix; selecting the maximum value of the sparse coefficient of each feature in the sparse coefficient vector as a contribution value, selecting features corresponding to multiple maximum contribution values ​​according to a preset number of features to be selected to form a new feature matrix, and performing clustering based on the new feature matrix. The method has low computational complexity and accurate clustering.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for unsupervised feature selection based on multi-clustering. Background Art

[0002] Machine learning algorithms often struggle when faced with hundreds or thousands of high-dimensional data points. The best solution is to collect more data or process it using feature engineering. However, data collection often requires significant human and material resources, making it a costly option. Feature selection, a highly effective method for processing features, selects the features most helpful for the task and removes unnecessary ones, thereby reducing the dimensionality of the data for easier processing and preventing overfitting. In most feature selection tasks, labeled data is often difficult to obtain. This is because manual data labeling requires significant effort, which complicates the task. Unsupervised feature selection, however, is used to process unlabeled data and make it more applicable to subsequent tasks.

[0003] The unsupervised feature selection algorithm MCFS (Multi-Cluster Feature Selection) effectively preserves the clustering structure of data, making it well-suited for multi-classification tasks. MCFS is an unsupervised feature selection algorithm based on manifold learning. It first uses the Laplacian Eigenmaps (LE) algorithm to embed high-dimensional manifold data into a low-dimensional space. The embedded feature matrix is ​​then processed. The LE algorithm uses Euclidean distance when constructing the p-proximity graph.

[0004] In practice, the inventors found that the above-mentioned prior art has the following defects:

[0005] The distance metrics between two points on a hypersurface in high-dimensional space include Euclidean distance and geodesic distance. Euclidean distance is the most commonly used distance metric, while geodesic distance is the shortest distance between two points on a hypersurface along the surface of the hypersurface in high-dimensional space. Figure 1 As shown, the Euclidean distance is Figure 1 The black straight line in (a), and the geodesic distance is the red curve in Figure 1(a). When processing hypersurfaces in high-dimensional space, if the hypersurface is close to a plane, the distance between two points in low-dimensional space is approximately the Euclidean distance. When the hypersurface in high-dimensional space is more curved, using Euclidean distance to embed data will result in large errors. Summary of the Invention

[0006] In order to solve the above technical problems, the purpose of the present invention is to provide a method and system for unsupervised feature selection based on multi-clustering. The technical solutions adopted are as follows:

[0007] In the first aspect, an embodiment of the present invention provides a method for unsupervised feature selection based on multi-clustering, which includes: obtaining multiple data points to be clustered in a high-dimensional space to obtain a feature matrix; using an isometric mapping algorithm to embed the feature matrix of the high-dimensional space into a low-dimensional space to obtain an embedding matrix in the low-dimensional space; normalizing the feature matrix to obtain a baseline feature matrix; using the baseline feature matrix and the embedding matrix to fit the feature vector composed of features in each dimension in the embedding matrix to obtain a sparse coefficient vector of each feature vector in the embedding matrix; selecting the maximum value of the sparse coefficient of each feature in the sparse coefficient vector as the contribution value, and selecting the features corresponding to the largest multiple contribution values ​​according to the preset number of features to be selected to form a new feature matrix, so as to perform clustering based on the new feature matrix.

[0008] Furthermore, the dimension of the embedding matrix is ​​equal to the number of clusters to be clustered.

[0009] Furthermore, the method of selecting features corresponding to the largest multiple contribution values ​​according to the preset number of features required to be selected to form a new feature matrix is: sorting the multiple contribution values ​​in descending order, and selecting the largest multiple features according to the preset number of features required to be selected to form a new feature matrix.

[0010] Furthermore, the steps of the isometric mapping algorithm include: calculating the distance between data points in the high-dimensional space, and then using the MDS algorithm to calculate the coordinates of the data points in the low-dimensional space.

[0011] Furthermore, the method for calculating the distance between data points in a high-dimensional space includes: for any data point in the high-dimensional space as a data point to be reduced in dimensionality, obtaining the neighboring points of the data point to be reduced in dimensionality, and obtaining an adjacency matrix, setting the distance between the neighboring points in the adjacency matrix and the data point to be reduced in dimensionality as the Euclidean distance, and calculating the shortest distance between all points in the adjacency matrix through a shortest path algorithm, wherein the shortest distance is the distance between the data points.

[0012] Furthermore, the number of the neighboring points is 5.

[0013] Furthermore, the shortest path algorithm adopts the classic Dijkstra or Floyd algorithm.

[0014] Furthermore, the fitting method for obtaining the sparse coefficient vector of each eigenvector in the embedding matrix by fitting the reference feature matrix and the embedding matrix is: fitting by using a LASSO regression algorithm.

[0015] In the second aspect, another embodiment of the present invention provides a multi-cluster unsupervised feature selection system based on geodesic distance, characterized in that it includes a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, it implements the steps of any one of the above methods.

[0016] The present invention has the following beneficial effects:

[0017] An embodiment of the present invention provides a method for feature selection, which maps high-dimensional features to a low-dimensional space and fits a sparse coefficient vector. Feature selection is performed based on the sparse coefficient vector. The clustering result obtained by using this feature selection method is more accurate. The method has low computational complexity and high computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a comparison diagram of Euclidean distance and geodesic distance;

[0020] Figure 2 A flowchart of a method for unsupervised feature selection based on multi-clustering provided by one embodiment of the present invention;

[0021] Figure 3 A flow chart of the feature selection principle of a method for unsupervised feature selection based on multi-clustering provided by an embodiment of the present invention;

[0022] Figure 4 The effect diagram of clustering results of four algorithms on five data sets;

[0023] Figure 5 A graph showing the relationship between the nearest neighbor parameter from 1 to 20 and the clustering performance in a method for unsupervised feature selection based on multi-clustering provided by an embodiment of the present invention;

[0024] Figure 6 A graph showing the relationship between the parameter embedding matrix sizes from 1 to 30 and the clustering performance of a multi-clustering based unsupervised feature selection method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] To further illustrate the technical means and effectiveness of the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a method and system for unsupervised feature selection based on multi-clustering proposed by the present invention. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0026] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0027] The following describes in detail a method and system for unsupervised feature selection based on multi-clustering provided by the present invention with reference to the accompanying drawings.

[0028] See also Figure 1 and Figure 2 , which shows a flow chart of a method for unsupervised feature selection based on multi-clustering and a flow chart of feature selection principles provided by an embodiment of the present invention, the method comprising the following steps:

[0029] Step S001: Acquire multiple data points to be clustered in a high-dimensional space to obtain a feature matrix; and use an isometric mapping algorithm to embed the feature matrix in the high-dimensional space into a low-dimensional space to obtain an embedding matrix in the low-dimensional space.

[0030] The general method of feature selection is to first set a standard that can better distinguish useful features from useless features, and then use this standard to filter the feature matrix and remove redundant features. For example: given a sample point Indicates that this data point has n features, where b i Represents the i-th feature; if k (k < n) features are selected through feature selection, the data points after feature selection are b k represents the kth feature, Represents a k-dimensional real number space. After feature selection, the features of the sample points change from n to k.

[0031] Specifically, in an embodiment of the present invention, before feature selection, the data in the high-dimensional space needs to be mapped to the low-dimensional space. The isometric mapping algorithm (Isomap) uses the property that the manifold data is locally homeomorphic to the Euclidean space. It first calculates the distance between the data points in the high-dimensional space, and then uses the MDS algorithm to calculate the coordinates of each data point in the low-dimensional space.

[0032] The method for obtaining the distance between data points in a high-dimensional space is as follows: for any data point in the high-dimensional space as a data point to be reduced in dimension, obtain the neighboring points of the data point to be reduced in dimension, and obtain the adjacency matrix, set the distance between the neighboring points in the adjacency matrix and the data point to be reduced in dimension as the Euclidean distance, and calculate the shortest distance between all points in the adjacency matrix using the shortest path algorithm, and the shortest distance is the distance between the data points. Specifically, assuming that for a feature matrix composed of a certain column of data points in an N×M-dimensional high-dimensional space D in Represents the N×M-dimensional real number space, x i Represents the i-th data point in the high-dimensional space, where i∈[1,N], and the feature matrix X is an N-dimensional feature matrix; first determine the data point x i The p nearest neighbor points in the adjacency matrix Lieutenant General x i The distance to the nearest p neighbor points is set as its Euclidean distance in high-dimensional space, x i The distances to other points are set to be unreachable; then the adjacency matrix W is fed into the shortest path algorithm to find the shortest distances between all points and update the adjacency matrix W. For the element w in the updated adjacency matrix W ij Represents the i-th data point x i With the jth data point x j The shortest path algorithm uses the classic Dijkstra or Floyd algorithm. By approximating the true geodesic distance through the shortest path algorithm, this method can well preserve the global structure of the manifold data, ensuring that the global structure is maintained to the greatest extent after the data is embedded in the low-dimensional space.

[0033] The method of using the MDS algorithm to calculate the coordinates of each data point in the low-dimensional space is as follows: first calculate the inner product matrix of the embedding matrix, then perform eigenvalue decomposition on the inner product matrix to obtain the eigenvalue matrix and eigenvector matrix, select the first K eigenvalues ​​and their eigenvectors to form a new eigenvalue matrix and a new eigenvector matrix, and calculate the embedding matrix based on the new eigenvalue matrix and the new eigenvector matrix. Specifically, the embedding matrix obtained by reducing the N-dimensional feature matrix X in the high-dimensional space to the K-dimensional low-dimensional space is denoted as where y j Represents the jth data point in the low-dimensional space; assuming that the distances between the corresponding data points are equal before and after dimensionality reduction, the data point x i With data point x j The distance in low-dimensional space is denoted as d ij , d ij =‖y i -y j ‖,‖y i -y j ‖ represents yi and y j The L2 norm between the known data points x i With data point x j The distance in high-dimensional space is w ij , then w ij =d ij =‖y i -y j ‖. Let the inner product matrix of the embedded matrix be Then for the element b in the inner product matrix B ij There is b ij =y i T y j , then Then we get:

[0034]

[0035] in,

[0036] That is, according to the data point x i With data point x j The distance in high-dimensional space is w ij Can get the element b in the inner product matrix ij , that is to say, the inner product matrix B can be obtained according to the adjacency matrix W. The eigenvalue decomposition of the inner product matrix is ​​B = VΛV T , we get the diagonal matrix Λ=diag(λ1,λ2,...,λ M ) and the eigenvector matrix V, where the eigenvalues ​​in the diagonal matrix are arranged in descending order; then the first k eigenvalues ​​and the corresponding eigenvectors are selected to form a new diagonal matrix and its eigenvector matrix, and the new diagonal matrix is ​​recorded as Λ′=diag(λ1,λ2,...,λ M ), the corresponding new eigenvector matrix is ​​recorded as V′, then the embedding matrix is ​​obtained according to the new diagonal matrix and its eigenvector matrix The embedding matrix Y is the coordinate matrix, or feature matrix, of the manifold data in the high-dimensional space in the low-dimensional space.

[0037] The feature matrix in the high-dimensional space is embedded into the low-dimensional space through the method of step S001, and each dimension in the low-dimensional space corresponds to a clustering structure.

[0038] Step S002: normalizing the feature matrix to obtain a reference feature matrix; fitting the feature vectors composed of the features in each dimension of the embedding matrix using the reference feature matrix and the embedding matrix to obtain a sparse coefficient vector of each feature vector in the embedding matrix;

[0039] The purpose of this step is to fit the embedding matrix Y through LASSO regression to evaluate the importance of various features in the feature matrix.

[0040] The method of fitting the embedding matrix using LASSO regression includes: after obtaining the embedding matrix Y through step S001, first normalizing the feature matrix X to obtain the normalized feature matrix, and the normalized feature matrix is ​​called the reference feature matrix, and the reference feature matrix is ​​recorded as X0, so that the feature metrics in the feature matrix are consistent; then for each column y′ in the embedding matrix Y k The sparse coefficient vector is obtained by LASSO regression fitting, where each column of the embedding matrix is ​​composed of multiple features. Each column is called a dimension, and the vector composed of multiple features in each column is called the feature vector of that dimension.

[0041] Specifically, the kth column in the embedding matrix is ​​recorded as y′ k , the sparse coefficient vector of the embedding matrix Y is recorded as a k , the sparse coefficient vector is an M-dimensional coefficient vector, then the minimization objective function of LASSO regression is:

[0042]

[0043] Among them, ‖a k ‖1 represents a k The L1 norm of express L2 norm. Due to the penalty of L1 norm regularization, when β is large enough, the sparse coefficients of certain features will drop exactly to 0. Due to the sparsity of LASSO regression, a sparse matrix will be automatically obtained in the process of solving LASSO regression. Since the feature metrics are unified before LASSO regression, the larger the coefficients of certain features, the greater their contribution to resolving clustering structures. Due to the sparsity of LASSO regression, certain features will be automatically filtered out, leaving some features that are more helpful in resolving clustering structures. Moreover, since the combination of several relatively weaker features can better distinguish different clusters, evaluating these features individually will ignore this characteristic, and in the process of LASSO regression, the combination of various features is solved instead of evaluating a single feature. Therefore, the LASSO regression method is selected, and the minimization objective function can be equivalently expressed as:

[0044]

[0045] Since the parameter γ of the LASSO regression algorithm is not easy to accurately control the sparsity of the coefficient matrix, the LARs algorithm can be used to solve the equivalent minimization objective function, and the LARs algorithm can be used to input ak The number of non-zero entries in controls the sparsity of the coefficient matrix.

[0046] Through step S002, each column vector y′ in the embedding matrix Y k Solve the LASSO regression problem and obtain K sparse coefficient vectors

[0047] Step S003: Select the maximum value of each feature at the corresponding position of all the sparse coefficient vectors as the contribution value, and select the features corresponding to the largest multiple contribution values ​​according to the preset number of features to be selected to form a new feature matrix, so as to perform clustering based on the new feature matrix.

[0048] Obtain K sparse coefficient vectors a by the method in step S002 k ,k∈(1,2,3,...,K),a k There are M dimensions, each a k corresponds to a cluster, and a k Each item in represents a feature. Since the data is normalized first, a k,j The larger the value, the greater the contribution of the jth feature to the kth cluster, and each feature contributes differently to different clustering structures. Therefore, for each feature, its maximum contribution value is selected as the selection criterion. The maximum contribution value of a feature is recorded as MCFS, and the specific definition of the maximum contribution value is as follows:

[0049]

[0050] where a k,j is the sparse coefficient vector a k The jth element of .

[0051] After obtaining the maximum contribution value MCFS of all features, the MCFS of all features are sorted in descending order, and the d features with the largest maximum contribution value MCFS are selected as the final selected features.

[0052] In step S003, the maximum value of each feature at the corresponding position of all sparse vectors is selected as the maximum contribution value MCFS of each feature, and d features with the largest maximum contribution value MCFS are selected to form a new feature matrix X.

[0053] Furthermore, the calculation method of the complexity of feature selection is as follows: IsoMap algorithm builds p proximity graph based on BallTree, and its computational complexity is approximately O(Mlog(p)Nlog(N)), where O() function represents the function of time complexity, M represents the feature dimension, and N represents the sample dimension. Dijkstra algorithm is used to build the shortest path graph, and its computational complexity is approximately O(N2 (p+log(N))). In MDS, we first find the inner product matrix of the embedding matrix Y. It takes O(N 2 ), using the Lanczos algorithm to find the first K eigenvalues ​​of a matrix requires O(KN 2 Solving LASSO regression with LARs algorithm requires O(M 3 +NM 2 ), so solving K LASSO regressions requires O(KM 3 +KNM 2 Calculating the MCFS score for all features requires O(KM). Selecting the top K features requires O(Mlog(M)). Since K << M, p << N, and ignoring constants, the total time complexity of MCFS-I is: O(Mlog(p)Nlog(N)+N 2 (p+log(N))+KN 2 + KM 3 +KNM 2 +KM+Mlog(M))≈O(NMlog(N)+N 2 log(N)+N 2 +M 3 +M+ Mlog(M)).

[0054] In summary, an embodiment of the present invention provides a method for feature selection, which maps high-dimensional features to low-dimensional space and fits them to obtain a sparse coefficient vector. Feature selection is performed based on the sparse coefficient vector, and the clustering result obtained by using this feature selection method is more accurate. This method has low computational complexity, small error, and high computational efficiency.

[0055] Furthermore, this feature selection method is used for clustering. The method provided in this embodiment of the present invention is referred to as the MCFS-I algorithm to distinguish it from the MSFS algorithm in the prior art. This method was implemented using PyCharm Community Edition 2019.3.3 for Windows, running on a PC running Windows 10 Home Edition with an Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz 2.81GHz and 8GB of RAM. Based on this experimental environment, the MCFS algorithm, Laplacian Score, and Nonnegative Discriminative Feature Selection (NDFS) in the prior art were compared with the MCFS-I algorithm provided in this embodiment of the present invention. In the MCFS-I algorithm, the number of neighbors and clusters was set to 5. The MCFS algorithm uses the Leap-Evolution (LE) algorithm to process manifold data and is also applicable to multi-cluster structures. The Laplacian Score algorithm selects data that best preserves the local manifold structure. NDFS also utilizes spectral clustering to enhance the unsupervised feature selection process.

[0056] The following will cluster the evaluation indicators used in the evaluation algorithm, the data set targeted by the algorithm, and the running algorithm, and analyze the advantages of the MCFS-I algorithm provided by the embodiment of the present invention over the existing algorithms.

[0057] First, the embodiment of the present invention uses the normalized mutual information (NMI) metric to evaluate clustering results. Specifically, after clustering using the selected features, assuming T is the true label of the data and T′ is the result of the clustering algorithm, their MI(T, T′) (mutual information) is defined as follows:

[0058]

[0059] where p(t i ) represents selecting a point from the data set that belongs to t i The probability of p(t′ i ) represents selecting a point from the data set that belongs to t′ i The probability of p(t i ,t′ i ) represents selecting a data from the data set that also belongs to t i and t′ i The specific definition of NMI is:

[0060]

[0061] Where H(T) represents the entropy of T, H(T′) represents the entropy of T′, and NMI(T,T′) shows that the value range of NMI is 0 to 1. When the value of NMI is closer to 1, the clustering result is more similar to the true classification of the data, and when NMI is closer to 0, the deviation between the clustering result and the true classification of the data is greater.

[0062] Secondly, five commonly used datasets are selected as the data sets for testing the algorithm. The five datasets are from the scikit-learn machine learning library and the scikit-feature feature engineering library. The dataset summary information is shown in Table 1:

[0063] Table 1 Dataset information

[0064] Samples Features Classes lung_small 72 325 7 MNIST 10000 784 10 warpPIE10P 210 2420 10 Yale 165 1024 15 digits 1797 64 10

[0065] The lung_small dataset is a small biological dataset from the scikit-feature library. It has a total of 72 sample points, each with 325 features, and these samples are divided into 7 categories. MNIST is a classic dataset of handwritten digits 0-9. Each sample point is a 28×28 grayscale image of a handwritten digit. The MNIST dataset has a total of 60,000 sample points, which are divided into a training set of 50,000 samples and a test set of 10,000 samples through geometric sampling. Due to the large number of sample points in the original dataset, only the test set was used for evaluation in this experiment. The warpPIE10P dataset is a small face dataset from the scikit-feature library. It has a total of 210 sample points, each with 2420 features, and the entire dataset is divided into 10 categories. The Yale dataset is a face dataset from the scikit-feature library, containing 165 grayscale images, each 32×32 in size. The dataset includes 15 individuals, each with 11 images in the following states: center-lit, with glasses, happy, left-lit, without glasses, normal, right-lit, sad, sleepy, surprised, and blinking. The digits dataset is a handwritten digit dataset from the scikit-learn library, containing 1797 samples. The handwritten digits are categorized into ten classes, ranging from 0 to 9, with approximately 180 grayscale images per class, and each image is an 8×8 image.

[0066] Then, we ran the algorithm for clustering, running it five times for each test. Each time, we ran the k-means clustering algorithm five times using random starting points and evaluated it using the NMI metric. The mean NMI of the five clustering runs was used as the result for that run, and the mean of the five runs was used as the result for that test. We performed clustering tests on the five datasets listed in Table 1 using default settings, and also tested the effects of different MCFS-I parameters on the results.

[0067] For details, please refer to Figure 4 , which respectively shows the clustering results of the four algorithms on five data sets. Through the analysis of the clustering results, it can be seen that the clustering performance of the MCFS-I algorithm on each data set is better than that of the MCFS algorithm, and the clustering results of the MCFS-I algorithm gradually reach the optimal value when the number of features is selected between 50 and 100. Since the number of features in the digits data set is only 64, the clustering evaluation results when only 30 features are selected are given.

[0068] Table 2 shows the clustering results of various feature selection algorithms when the number of selected features is 100 / 30 (digits). The MCFS-I algorithm achieved the best clustering results on all three datasets except the MNIST dataset, where it was second only to the NDFS algorithm. When the number of selected features is 100, the MCFS-I algorithm achieved an average improvement of approximately 3.206% over the MCFS algorithm on all five datasets, an average improvement of approximately 14.822% over the lap_score algorithm, and an average improvement of approximately 7.704% over the NDFS algorithm.

[0069] Table 2 Clustering results of each algorithm in each data set when 100 / 30 (digits) features are selected (%)

[0070] Algorithm Lung_small warpPIE10P MNIST Yale digits MCFS-I 78.23 40.31 41.99 50.45 72.31 MCFS 72.22 36.25 38.00 50.18 70.61 NDFS 72.44 25.99 42.56 41.15 62.63 Lap_score 59.92 32.42 9.73 49.07 58.04 All Features 73.14 23.98 50.20 49.73 74.38

[0071] Although the NDFS algorithm performed slightly better than the MCFS-I algorithm on the MNIST dataset, the average runtime per test was significantly longer than that of MCFS-I. As shown in Table 3, MCFS-I averaged approximately 189 seconds on the MNIST dataset, while NDFS averaged 6600 seconds. NDFS also significantly outperformed MCFS-I on the other two datasets. The time difference between the MCFS and MCFS-I algorithms on both large and small datasets was relatively small and within an acceptable range. However, MCFS-I achieved significantly higher accuracy than MCFS. Because the lung_small dataset was too small and the lap_score algorithm was too simple, no comparison was made here.

[0072] Table 3 Comparison of average running time of each algorithm

[0073] MNIST warpPIE10P Yale digits MCFS-I 189s 429ms 195ms 6.89s MCFS 98s 419ms 190ms 2.83s NDFS 6600s 18000ms 2000ms 6.38s

[0074] Finally, combine Figure 5 Further analysis of the parameters, Figure 5 The relationship between the MCFS-I algorithm's clustering performance and the nearest neighbor parameter, ranging from 1 to 20, is shown. The figure shows that clustering performance approaches optimal performance when the nearest neighbor parameter is 5. However, as the nearest neighbor parameter increases, the performance fluctuates around the mean, sometimes even declining. Clustering results are relatively stable in the MNIST and digits datasets as the parameter increases, while they fluctuate more significantly in the other three datasets. It can be seen that varying the n_neighbors parameter results in very stable performance in the MNIST and digits datasets, while significantly impacting the results in the Yale and warpPIE10P face datasets and the lung_small biological dataset. Therefore, the standard deviation and coefficient of variation of the NMI for different algorithms in these datasets were tested for n_neighbors values ​​ranging from 1 to 20 to determine their sensitivity to this parameter.

[0075] Table 4 shows the degree of dispersion of the results of the MCFS algorithm and the MCFS-I algorithm for different choices of the parameter n_neighbors. The coefficient of variation (CV) is the standard deviation of the sample population σ divided by the mean of the sample population μ. The coefficient of variation is defined as This eliminates the effects of different measurement scales and dimensions and is used to compare the degree of dispersion of different sample populations. Table 4 shows that both algorithms perform very stably on the digits handwriting dataset, but are unstable on the other three datasets. This indicates that the effect of the parameter n_neighbors on the results is not due to changes in the geodesic distance. Instead, the effect of the parameter n_neighbors on the clustering results is more pronounced in the warpPIE10P dataset, where the coefficient of variation is much greater than that of the other datasets.

[0076] Table 4. Dispersion of NMI in various datasets for MCFS and MCFS-I with different choices of parameter n_neighbors

[0077]

[0078] Figure 6The relationship between the MCFS-I algorithm's parameter embedding matrix size, ranging from 1 to 30, and clustering performance is shown. Clustering performance is optimal when the embedding matrix size is equal to the number of clusters in the dataset. As the embedding matrix increases, clustering performance fluctuates slightly. Compared to the nearest neighbor parameter, increasing the embedding matrix size results in less fluctuation in clustering results.

[0079] The above comparison shows that across all five datasets, MCFS-I, an improvement based on the MCFS algorithm, outperforms the other three algorithms on average. MCFS-I significantly reduces processing time for large datasets compared to the slightly more performant NDFS, which takes a relatively long time to process large datasets. Selecting 50-100 features from the data yields excellent results for MCFS-I, surpassing the clustering results obtained when all features are used. The choice of the n_neighbors parameter for the MCFS-I algorithm has minimal impact on the clustering results for the digits and MNIST handwritten digit datasets, but significantly affects the clustering results for the lung_small, Yale, and warpPIE10P datasets.

[0080] In summary, this paper introduces geodesic distance into the unsupervised feature selection algorithm MCFS and combines IsoMap with MCFS to propose the MCFS-I algorithm. This algorithm is better able to handle hypersurfaces in high-dimensional spaces and better preserve the global structure. Compared with the lap_score, NDFS, and MCFS algorithms, MCFS-I demonstrates significantly superior performance. MCFS-I is more effective when the number of selected features is between 50 and 100. The number of nearest neighbors p varies significantly, but generally achieves good results when p = 5. Furthermore, compared with NDFS, MCFS-I is faster on large datasets and generally performs better when the embedding matrix size K is equal to the number of clusters in the data.

[0081] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. The above description is of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0082] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-clustering based unsupervised feature selection method for grayscale image processing, characterized in that: The feature selection methods include: Acquire multiple data points to be clustered in a high-dimensional space to obtain a feature matrix; embed the feature matrix in the high-dimensional space into a low-dimensional space using an isometric mapping algorithm to obtain an embedding matrix in the low-dimensional space; Normalizing the feature matrix to obtain a reference feature matrix; fitting the feature vectors composed of the features in each dimension of the embedding matrix using the reference feature matrix and the embedding matrix using a LASSO regression algorithm to obtain a sparse coefficient vector of each feature vector in the embedding matrix; Selecting the maximum value of the sparse coefficient of each feature in the sparse coefficient vector as the contribution value, sorting the multiple contribution values ​​in descending order, selecting the features corresponding to the largest multiple contribution values ​​according to a preset number of features to be selected to form a new feature matrix, and performing clustering based on the new feature matrix; The steps of the isometric mapping algorithm include: calculating the distance between data points in the high-dimensional space, and then calculating the coordinates of the data points in the low-dimensional space using the MDS algorithm; The method for calculating the distance between data points in a high-dimensional space includes: for any data point in the high-dimensional space as a data point to be reduced in dimensionality, obtaining the neighboring points of the data point to be reduced in dimensionality, and obtaining an adjacency matrix, setting the distance between the neighboring points in the adjacency matrix and the data point to be reduced in dimensionality as the Euclidean distance, and calculating the shortest distance between all points in the adjacency matrix through a shortest path algorithm, wherein the shortest distance is the distance between the data points.

2. The method for unsupervised feature selection based on multi-clustering according to claim 1, characterized in that: The dimension of the embedding matrix is ​​equal to the number of clusters to be clustered.

3. The method for unsupervised feature selection based on multi-clustering according to claim 1, characterized in that: The number of the neighboring points is 5.

4. The method for unsupervised feature selection based on multi-clustering according to claim 1, characterized in that: The shortest path algorithm adopts the classic Dijkstra or Floyd algorithm.

5. A multi-cluster unsupervised feature selection system based on geodesic distance, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 4 when executing the computer program.

Images