Fast fuzzy spectral clustering method based on anchor point diagram

Abstract

The invention discloses a fast fuzzy spectral clustering method based on an anchor point diagram, and is applied to the technical field of fast spectral clustering. Comprising the following steps: acquiring an original data matrix, a category number and an anchor point number, generating anchor points, and constructing a kernel-free similar matrix and a kernel-free adjacent matrix; initializing an original data class guidance matrix and an anchor point data class guidance matrix; updating the class guidance matrix of the original data; updating the anchor point data class guidance matrix; and repeatedly updating until convergence, and outputting a fuzzy membership matrix. According to the method, the membership relationship between each original data point and different clustering centers can be identified more accurately, the interpretability of clustering is enhanced, the sensitivity of clustering to the original clustering centers is reduced, and the clustering precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of fast spectral clustering, and more particularly to a fast fuzzy spectral clustering method based on anchor graph. Background Art

[0002] Spectral clustering is a widely used clustering algorithm. Compared with the traditional K-Means algorithm, spectral clustering has stronger adaptability to data distribution, excellent clustering effect, and much less clustering calculation. Among the existing spectral clustering algorithms, the fast spectral clustering algorithm based on anchor graph has achieved good clustering effect, but there are still some limitations. First, in the process of eigendecomposition of the normalized Laplace matrix, when selecting the eigenvector corresponding to the minimum eigenvalue to form the class guidance matrix, negative values ​​may be generated; secondly, the class guidance matrix in the traditional spectral clustering adopts a "hard label" clustering mode, and the membership of the data point to the cluster center is either 0 or 1, while a large number of pixels in the hyperspectral image are located in the common area between various objects. In this case, using fuzzy clustering labels to identify the membership of each data point to different cluster centers (with a value range of 0-1) is crucial to improving clustering accuracy. Therefore, how to provide a fast fuzzy spectral clustering method based on anchor graph is a problem that technicians in this field need to solve urgently. Summary of the invention

[0003] In view of this, the present invention provides a fast fuzzy spectral clustering method based on anchor graph to solve the problems in the existing fast spectral clustering algorithm.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A fast fuzzy spectral clustering method based on anchor graph includes the following steps:

[0006] S1. Get the original data matrix The number of categories c and the number of anchor points m;

[0007] S2, use BKHK point selection algorithm to generate m anchor points;

[0008] S3, construct the non-core similarity matrix Z and the non-core adjacency matrix W;

[0009] S4. Initialize the original data class guidance matrix and anchor data class guidance matrix

[0010] S5, updating the class guidance matrix F of the original data;

[0011] S6, update the anchor data class guidance matrix G;

[0012] S7. Repeat S5-S6 until convergence, and output the fuzzy membership matrix F.

[0013] Optionally, the optimization objective function of the fast fuzzy spectral clustering method is:

[0014]

[0015] Where F is the class guidance matrix, Z is the kernel-free similarity matrix, s is the constraint parameter, and λ is the regularization parameter.

[0016] Optionally, the optimization process of the objective function is as follows:

[0017] The original objective function is:

[0018]

[0019] Add a non-negative matrix constraint to the class guidance matrix F of the original objective function, limit the sum of the values ​​in each row of the class guidance matrix F to 1, and then add a regular constraint term to obtain:

[0020]

[0021] By using the method of constructing anchor graph, m anchor points are selected from m original data points to participate in constructing the non-core similarity matrix Z. The non-core similarity matrix Z is used to represent the similarity between the original data points and the selected anchor points, thereby replacing the similarity matrix S with the non-core similarity matrix Z. A new balanced regularization term Tr(F T F+G T G), we get:

[0022]

[0023] Introduce constraint parameter s to maximize the sum of similarities within the class, using Z-s11 T Substituting the kernel-free similarity matrix Z, we get:

[0024]

[0025] Where S is the similarity matrix.

[0026] Optionally, S5 is specifically:

[0027] Fix the anchor data class guidance matrix G, optimize the class guidance matrix F of the original data; use the Lagrange multiplier method to solve the objective function:

[0028]

[0029] K=(1 / 2λ1)AG

[0030] A=Z-s11 T

[0031] Where λ1 is the regularization parameter.

[0032] Optionally, S6 is specifically:

[0033] Fix the class guidance matrix F of the original data, optimize the class guidance matrix G of the anchor data; use the Lagrange multiplier method to solve the objective function:

[0034]

[0035] M=(1 / 2λ2)A T F

[0036] A=Z-s11 T

[0037] Where λ2 is the regularization parameter.

[0038] Through the above technical solution, it can be known that compared with the prior art, the present invention provides a fast fuzzy spectral clustering method based on anchor graph, which has the following beneficial effects: by constructing a fuzzy clustering model, providing different membership degrees for each data, introducing a new balanced regularization term in the objective function of the model, introducing non-negative matrix constraints in the class indicator matrix, and finally using a convergent alternating iterative optimization algorithm to solve the objective function. The present invention can be used more accurately to identify the membership relationship between each original data point and different cluster centers, enhance the interpretability of clustering, reduce the sensitivity of clustering to the original cluster center, and improve clustering accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0040] Figure 1 This is a flow chart of the fast fuzzy spectral clustering method based on anchor graph of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The embodiment of the present invention discloses a fast fuzzy spectral clustering method based on anchor graph, such as Figure 1 As shown, the following steps are included:

[0043] S1. Get the original data matrix The number of categories c and the number of anchor points m;

[0044] S2, use BKHK point selection algorithm to generate m anchor points;

[0045] S3, construct the non-core similarity matrix Z and the non-core adjacency matrix W;

[0046] S4. Initialize the original data class guidance matrix and anchor data class guidance matrix

[0047] S5, updating the class guidance matrix F of the original data;

[0048] S6, update the anchor data class guidance matrix G;

[0049] S7. Repeat S5-S6 until convergence, and output the fuzzy membership matrix F.

[0050] In the embodiment of the present invention, S3 is specifically:

[0051] By using the kernel-free similarity matrix To represent the similarity between data points, the (i, j)th element Z ij Defined as:

[0052]

[0053] In the formula, Represents data point x i The index of the k nearest neighbors, K(x i ,x s ) is the Gaussian kernel function, which is further expressed as:

[0054]

[0055] Where σ is the thermal core parameter.

[0056] The similarity Z between data points ij Solve by the following formula:

[0057]

[0058] In the formula, represents the i-th row element of Z, γ is the regularization parameter, and n is the number of data points; the constraint condition of the objective function is that the sum of the elements in each row of the matrix is ​​1, and each element is greater than or equal to 0, so as to ensure that the obtained solution is non-negative and sparse. However, there are noise, abnormal pixels or mixed pixels in the hyperspectral image. The above formula does not comprehensively consider the spatial information of the hyperspectral image, which will lead to poor clustering accuracy. In order to make full use of the spatial information of the hyperspectral image, the above formula can be further improved as follows:

[0059]

[0060] In the formula, Represents the data point x i The average value of adjacent pixels in the neighborhood space. The parameter α is used to adjust the balance between the original data points in the hyperspectral image and the corresponding mean-filtered data points. For ease of calculation, assume that the square of the Euclidean distance between data points is Right now The mean of a data point and the square of the Euclidean distance between data points is Right now and The relationship is:

[0061]

[0062] represents a column vector, then the above formula can be converted into a vector form, namely:

[0063]

[0064] It can be intuitively found that the unknown quantity in the above formula is vector z i and parameter γ, the purpose is to learn to obtain a sparse vector z containing k non-zero i , the parameter γ can be obtained by optimization, because the non-core similarity matrix Z is sparse, which can greatly reduce the computational burden of subsequent spectral clustering analysis; the Lagrangian function of the above formula is:

[0065]

[0066] Among them, η and β i ≥0 are all Lagrange multipliers. The optimal solution z can be obtained by taking the derivative of the above formula. * , and let z i is equal to zero, and the solution is:

[0067]

[0068] where z * The jth element of is:

[0069]

[0070] According to the definition of Karush-Kuhn-Tucker (KKT) conditions, the optimal solution z * The following conditions must be met:

[0071]

[0072] Solve for z * The jth element of is:

[0073]

[0074] Among them, (x) + represents max(x,0), that is, x≥0, according to the vector z i Constraints The solution for parameter η is:

[0075]

[0076] The parameter γ satisfies the inequality:

[0077]

[0078] The optimal solution of the objective function is a vector containing k non-zeros Then the solution for the parameter γ is:

[0079]

[0080] The optimal solution of the non-core similarity matrix Z is obtained as:

[0081]

[0082] The non-core similarity matrix Z is successfully constructed. The non-core similarity matrix Z reflects the similarity between data and can be used to construct the adjacency matrix in the spectral clustering algorithm. Based on the construction of the non-core similarity matrix Z, the adjacency matrix W can be obtained. The calculation formula is as follows:

[0083] W=ZΛ -1 Z T .

[0084] Furthermore, the optimization objective function of the fast fuzzy spectral clustering method is:

[0085]

[0086] Where F is the class guidance matrix, Z is the kernel-free similarity matrix, s is the constraint parameter, and λ is the regularization parameter.

[0087] Furthermore, the optimization process of the objective function is as follows:

[0088] The original objective function is:

[0089]

[0090] The traditional spectral clustering algorithm may generate negative values ​​when using eigenvectors to form the class guidance matrix F, which does not conform to the membership relationship between pixels and actual objects. Therefore, a non-negative matrix constraint is added to the class guidance matrix F of the original objective function to limit the sum of the values ​​of each row of the class guidance matrix F to 1, and then a regular constraint term is added to obtain:

[0091]

[0092] By using the method of constructing anchor graph, m anchor points are selected from m original data points to participate in constructing the non-core similarity matrix Z. The non-core similarity matrix Z is used to represent the similarity between the original data points and the selected anchor points, thereby replacing the similarity matrix S with the non-core similarity matrix Z. A new balanced regularization term Tr(F T F+G T G), we get:

[0093]

[0094] The balanced regularization term helps to obtain a more accurate and balanced clustering effect; the constraint parameter s is introduced to maximize the sum of similarities within the class, using Z-s11 T Substituting the kernel-free similarity matrix Z, we get:

[0095]

[0096] In the formula, S is the similarity matrix. By adjusting the size of the constraint parameter s, the similarity of data in the same category is made greater, and the similarity of data in different categories is made smaller.

[0097] In an embodiment of the present invention, the first term of the optimization objective function expresses the sum of similarities within the class, and the second term is a constraint and adjustment to the first term to avoid obtaining an undesirable class guidance matrix F, in which the values ​​in F are all between 0 and 1, indicating the membership of each original data point to different categories. The idea of ​​fuzzy clustering is used to identify the membership of each original data point to different classes, which improves the accuracy and interpretability of the clustering results and reduces the impact of fuzzy clustering on the sensitivity of the initial clustering center. The strategy of constructing an anchor graph can reduce the time complexity of hyperspectral image clustering and realize rapid clustering of large-scale images. After the optimization objective function is constructed, a fast-converging iterative optimization method is used to solve it.

[0098] Furthermore, S5 is specifically:

[0099] Fix the anchor data class guidance matrix G, optimize the class guidance matrix F of the original data; use the Lagrange multiplier method to solve the objective function:

[0100]

[0101] K=(1 / 2λ1)AG

[0102] Where λ1 is the regularization parameter.

[0103] In the embodiment of the present invention, the derivation process of the objective function is:

[0104] When the anchor data class guidance matrix G is fixed, the objective function can be derived as:

[0105]

[0106] Where λ1 is the regularization parameter, and the formula can also be transformed into:

[0107]

[0108] Let A = Z-s11 T ,get:

[0109]

[0110] Assuming K = (1 / 2λ1)AG, we get:

[0111]

[0112] The above formula can be changed to:

[0113]

[0114] Multiplying the above formula by a certain quantity can turn it into:

[0115]

[0116] Furthermore, S6 is specifically:

[0117] Fix the class guidance matrix F of the original data, optimize the class guidance matrix G of the anchor data; use the Lagrange multiplier method to solve the objective function:

[0118]

[0119] M=(1 / 2λ2)A T F

[0120] Where λ2 is the regularization parameter.

[0121] In the embodiment of the present invention, the derivation process of the objective function is:

[0122] When the class index matrix F of the original data is fixed, the objective function can be derived as:

[0123]

[0124] Where λ2 is the regularization parameter, and the formula can also be transformed into:

[0125]

[0126] Let A = Z-s11 T ,get:

[0127]

[0128] The above formula can be changed to:

[0129]

[0130] It can also be written as:

[0131]

[0132] Let M = (1 / 2λ2)A T F, we can get:

[0133]

[0134] The above formula can be changed to:

[0135]

[0136] Multiplying the above objective function by a quantity can turn it into:

[0137]

[0138] In one embodiment of the present invention, the complexity of the algorithm of the present invention is calculated:

[0139] The time complexity of randomly selecting anchor points is O(1); the time complexity of constructing the similarity matrix Z is O(ndm); obtaining the category indicator matrices F and G of the original data and anchor point data requires O(ncdt1) and O(mcdt2) respectively, where t1 and t2 are the number of iterations; it can be concluded that the overall time complexity of the algorithm of the embodiment of the present invention is O(nmd+ncdt1+mcdt2+1). Since the number of anchor points is much smaller than the number of original data points, that is, m<<n, and the number of categories is much smaller than the data dimension, that is, c<<d, the overall time complexity of the algorithm of the embodiment of the present invention is about O(ndm).

[0140] In one embodiment of the present invention, the algorithm of the present invention is verified by experiments, specifically:

[0141] The data sets used in the experiment include the small data set IndianPines, the medium data set Salinas, and two large data sets PaviaUniversity and Botswana. The data sets are divided into three groups according to small, medium and large. Other algorithms involved in the comparison include K-means, FCM, FCM_S1, SC, RSFCM, SGCNR and FECBG algorithms.

[0142] First, a clustering experiment is conducted on a small dataset IndianPines. The parameters of the algorithm in the embodiment of the present invention are set as: m = 60, and the iteration parameters are: λ1 = 0.001, λ2 = 0.001, s = 0.015. The clustering maps of each algorithm are as follows: Figure 1 ,The clustering quantitative indicators are shown in Table 1. The data on the right side of the table are the ,user accuracy UA(%) of each feature after clustering. The best results of all indicators are ,indicated in bold;

[0143] Table 1 Quantitative clustering indicators of IndianPines dataset

[0144]

[0145] As can be seen from Table 1, the algorithm of the embodiment of the present invention achieved the best UA on five types of objects (Corn-notill, Grass-pasture, Grass-trees, Soybean-mintill, Wheat). The average accuracy AA is 42.28%, which is 4.17%-11.16% higher than other algorithms; the overall accuracy is 42.16%, which is 6.62%-45.07% higher than other comparison algorithms; the Kappa coefficient is 0.3415, which is also the best among the comparison algorithms; in terms of clustering speed, the K-means algorithm has the fastest clustering speed, which is only 0.9116s, but the clustering accuracy is the worst; the spectral clustering speed is the slowest, which takes 39.2537s; the algorithm of the embodiment of the present invention takes 4.1073s. Combining the clustering accuracy and speed, the algorithm of the embodiment of the present invention achieves the best clustering accuracy without the cost of too long clustering time, and has the best overall performance.

[0146] When clustering experiments were conducted on the medium-sized dataset Salinas, the number of selected anchor points also increased accordingly. The parameters of the algorithm in the embodiment of the present invention were set to m = 240, and the iteration parameters were: λ1 = 0.001, λ2 = 0.001, s = 0.015. The quantitative indicators of each algorithm are shown in Table 2. The data on the right side of the table are the user accuracy UA (%) of each feature after clustering;

[0147] Table 2 Quantitative indicators of clustering of Salinas dataset

[0148]

[0149] As can be seen from Table 2, the classification accuracy of the algorithm of the embodiment of the present invention on a single feature is not very outstanding, and the best UA is only achieved on two types of features (Brocoli weeds 1 and Vinyard trellis), and one type of feature (Fallow) is not identified. However, compared with other algorithms, the overall quantitative evaluation indicators of the algorithm of the embodiment of the present invention are the best, with an average accuracy AA of 72.13%, 7.21%-12.05% higher than other comparison algorithms; the overall accuracy OA reaches 69.65%, 4.86%-8.79% higher than other algorithms; the Kappa coefficient is 0.6643.

[0150] Clustering experiments were conducted on two large datasets, Pavia University and Botswana. First, clustering was performed on Pavia University. The parameters of the algorithm in the embodiment of the present invention were set to m = 270, k = 5, and the iteration parameters were: λ1 = 1.1, λ2 = 0.1. The quantitative indicators of each algorithm are shown in Table 3;

[0151] Table 3 Quantitative clustering indicators of Pavia University and Botswana datasets

[0152]

[0153] It can be seen that the embodiment of the present invention has achieved the best results in the three clustering accuracy indicators. The average accuracy AA is 57.33%, and the overall accuracy OA reaches 64.97%, which is 20.7%-24.88% higher than other algorithms; the Kappa coefficient is 0.5382, and the clustering time is 18.39s, which is not the fastest, but faster than FCM_S1, RSFCM and SGCNR.

[0154] Then clustering is performed on the Botswana dataset. In the experiment, the parameters of the algorithm of the embodiment of the present invention are set to m=270, k=5, and the iteration parameters are: λ1=1.1, λ2=0.1, s=0.1. The quantitative indicators of each algorithm are shown in Table 4;

[0155] Table 4 Quantitative clustering indicators of Botswana dataset

[0156]

[0157] It can be seen that the algorithm of the embodiment of the present invention achieves the highest UA on three types of landforms (Hippo grass, Floodplain grasses 1 and Acacia grasslands), and obtains the best results on the three overall clustering accuracy indicators, among which the average accuracy AA is 60.42%, which is 3.45%-11.99% higher than other comparison algorithms; the overall accuracy is 59.91%, which is 4%-10.65% higher than other comparison algorithms, and the kappa coefficient is 0.5662. The clustering time of the algorithm of the embodiment of the present invention is 60.41s. Considering the comprehensive clustering accuracy and clustering speed, the clustering efficiency of the algorithm of the embodiment of the present invention is the highest, and the best clustering accuracy is achieved within an acceptable clustering time.

[0158] Through four groups of experiments on small, medium and large data sets, it can be seen that the algorithm of the embodiment of the present invention is not only better than the classic K-means algorithm and FCM clustering algorithm in improving clustering accuracy, but also better than the three anchor graph-based clustering algorithms. It can be seen that the algorithm of the embodiment of the present invention combines the advantages of the anchor graph strategy and the fuzzy clustering strategy.

[0159] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0160] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fast fuzzy spectral clustering method based on anchor graph, characterized in that: The following steps are involved: S1. Get the original data matrix The number of categories c and the number of anchor points m; S2, use BKHK point selection algorithm to generate m anchor points; S3, construct the non-core similarity matrix Z and the non-core adjacency matrix W; S4. Initialize the original data class guidance matrix and anchor data class guidance matrix S5, updating the class guidance matrix F of the original data; S6, update the anchor data class guidance matrix G; S7. Repeat S5-S6 until convergence, and output the fuzzy membership matrix F.

2. The fast fuzzy spectral clustering method based on anchor graph according to claim 1, characterized in that: The optimization objective function of the fast fuzzy spectral clustering method is: Where F is the class guidance matrix, Z is the kernel-free similarity matrix, s is the constraint parameter, and λ is the regularization parameter.

3. The fast fuzzy spectral clustering method based on anchor graph according to claim 2, characterized in that: The optimization process of the objective function is as follows: The original objective function is: Add a non-negative matrix constraint to the class guidance matrix F of the original objective function, limit the sum of the values ​​in each row of the class guidance matrix F to 1, and then add a regular constraint term to obtain: By using the method of constructing anchor graph, m anchor points are selected from m original data points to participate in constructing the non-core similarity matrix Z. The non-core similarity matrix Z is used to represent the similarity between the original data points and the selected anchor points, thereby replacing the similarity matrix S with the non-core similarity matrix Z. A new balanced regularization term Tr(F T F+G T G), we get: Introduce constraint parameter s to maximize the sum of similarities within the class, using Z-s11 T Substituting the kernel-free similarity matrix Z, we get:

4. The fast fuzzy spectral clustering method based on anchor graph according to claim 1, characterized in that: S5 is as follows: Fix the anchor data class guidance matrix G, optimize the class guidance matrix F of the original data; use the Lagrange multiplier method to solve the objective function: K=(1 / 2λ1)AG A=Z-s11 T Where λ1 is the regularization parameter.

5. The fast fuzzy spectral clustering method based on anchor graph according to claim 1, characterized in that: S6 is specifically: Fix the class guidance matrix F of the original data, optimize the class guidance matrix G of the anchor data; use the Lagrange multiplier method to solve the objective function: M=(1 / 2λ2)A T F A=Z-s11 T Where λ2 is the regularization parameter.

Images