Adaptive L 2p Fuzzy clustering method based on adaptive L norm

Abstract

This invention belongs to the field of machine learning and data mining technology, and discloses an adaptive... L 2p Fuzzy clustering method based on norm: Step 1: Define the data samples to be clustered and construct the objective function of the clustering algorithm; Step 2: Initialize the parameters in the constructed objective function; Step 3: Use an alternating optimization method and the initialized data obtained in Step 2 to optimize and update each variable in the objective function until the objective function converges. Compared with existing methods based on... L 2p Unlike norm-based fuzzy clustering algorithms, this invention can adaptively learn. L 2p Parameters in norm p It can effectively solve the problem of parameter allocation in traditional algorithms. p The problem of poor clustering results due to manually setting fixed values ​​is addressed. Furthermore, to avoid trivial solutions in membership degrees, this invention introduces an entropy regularization term. Experimental results demonstrate that this invention achieves better clustering performance than traditional algorithms.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and data mining technology, and discloses an adaptive... L 2p Fuzzy clustering methods based on norms. Background Technology

[0002] In today's information-explosive era, massive amounts of data are generated and accumulated, making efficient analysis of this data a pressing issue for various industries. Traditional data analysis methods are often inefficient when processing large datasets, necessitating the development of more advanced technologies such as machine learning to address this challenge. Clustering is an important research area in unsupervised machine learning because it is not dependent on labels and is widely favored. To date, numerous clustering algorithms have been proposed and widely applied to tasks such as image segmentation, person re-identification, document clustering, and network analysis.

[0003] Traditional clustering analysis methods typically employ hard clustering models, assigning sample points to a single cluster. However, this approach has limitations when dealing with complex, multimodal, and ambiguous data in the real world. To overcome these limitations, soft clustering methods have gradually become a hot topic in data analysis. In hard clustering, we only know that a sample belongs to a certain cluster, but we don't know the relationship between that sample and other clusters. In other words, in hard clustering, the membership degree of a sample can only be 0 or 1. A typical example of hard clustering is the K-Means algorithm. In soft clustering, the membership degree varies between 0 and 1, which can better reveal the relationship between a sample and all clusters. The traditional Fuzzy C-Means (FCM) is a classic soft clustering algorithm, proposed by Ruspini and later improved by Dunn.

[0004] However, since a series of fuzzy clustering algorithms, represented by FCM, use squared Euclidean distance as the distance metric, their robustness is not very good. Then, Xu Jinglin et al. proposed a method based on... L 2p A norm-based fuzzy clustering algorithm, which exhibits better robustness compared to fuzzy clustering algorithms based on squared Euclidean distance. Although L 2p Norms can help improve the robustness of fuzzy clustering algorithms, but L 2p There is a parameter in the norm. p Traditional based L 2p In fuzzy clustering algorithms based on norm, this parameter is pre-set. p It's set to a fixed value. However, in the real world, data is complex and diverse, and manually set fixed values ​​are ineffective. pThe values ​​are often inappropriate and can negatively impact clustering results. Therefore, it is necessary to develop an adaptive learning method. L 2p Norm parameters p Fuzzy clustering methods are very meaningful. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive L 2p A norm-based fuzzy clustering method to solve the problems of traditional clustering methods. L 2p In the fuzzy clustering algorithm of norm, the parameters are manually set. p Setting a fixed value leads to poor clustering results.

[0006] To achieve the above objectives, the present invention employs the following technical solution: An adaptive L 2p The norm-based fuzzy clustering method specifically includes the following steps: Step 1: Let the data samples to be clustered be... The objective function for the clustering algorithm is as follows:

[0007] In the formula: n —Sample size; c —The number of clusters in the sample; d -sample x i Dimensions i =1, 2, ..., n ; u ij —Membership matrix The i row and number j The elements of the column represent sample values. x i Belongs to the j The membership degree of each cluster; j =1, 2, ..., c Membership matrix ; —No. j The center of a cluster; set of cluster centers ; —No. j The covariance coefficients of each cluster; the set of covariance coefficients of each cluster. ; p — L 2p Parameters in the norm; λ >0 — Entropy regularization hyperparameter; Step 2: Initialize the parameters in the objective function constructed in Step 1 to obtain the initialized data. The specific operations are as follows: Step 21, data samples to be clustered X Perform normalization preprocessing to obtain the preprocessed data; Step 22: Select the most suitable parameter values ​​for the entropy regularization hyperparameter from the reference parameter list using a grid search method. λ Initialization is performed, wherein the reference parameter list is [0.1, 0.5, 1, 2, 4, 6, 8, 12, 16, 20, 40, 100, 300]; Step 23, Initialization and p =1; Step 24, for the cluster center Random initialization is used; Step 3: Use the method of alternating optimization of variables and the initialized data obtained in Step 2 to optimize and update each variable in the objective function constructed in Step 1 until the objective function converges.

[0008] Furthermore, in step 21, the normalization preprocessing is performed using the min-max normalization method.

[0009] Furthermore, step 3 specifically includes the following steps: Step 3.1: Update according to the following formula u ij : ; Step 3.2.1: Construct intermediate variables according to the following formula D ij : ; Step 3.2.2: Update the following formula: j The center of each cluster : ; Step 3.2.3: Update the following formula: j Covariance coefficients of each cluster : ; Step 3.2.4: Judgment and If convergence has occurred, proceed to step 3.3; otherwise, repeat steps 3.2.1 to 3.2.3 until convergence is achieved. and Convergence, then proceed to step 3.3; Step 3.3: Update using Newton's method as shown in the following formula. p until p No longer changing;

[0010] in: ; Step 3.4: Determine whether the objective function constructed in Step 1 has converged. If it has, output the membership matrix U; otherwise, repeat steps 3.1 to 3.3 until the objective function constructed in Step 1 has converged, and then output the membership matrix U.

[0011] Compared to existing technologies, the method of this invention employs an adaptive approach. L 2p Norms are used to improve the clustering performance of algorithms. This is in contrast to existing methods based on... L 2p Unlike norm-based fuzzy clustering algorithms, traditional algorithms require manual configuration. L 2p Norm parameters p The value of the present invention allows for adaptive learning. L 2p Parameters in norm p It can effectively solve the problem of parameter allocation in traditional algorithms. p Manually setting fixed values ​​leads to poor clustering results. Furthermore, to avoid trivial solutions in membership degrees, the method of this invention introduces an entropy regularization term. Experimental results demonstrate that the algorithm of this invention achieves better clustering performance than traditional algorithms. Attached Figure Description

[0012] Figure 1 This is a convergence curve of the objective function for clustering on the Iris dataset in this invention.

[0013] Figure 2 This is the convergence curve of the objective function for clustering on the PalmData dataset in this invention.

[0014] Figure 3 This is a convergence curve of the objective function for clustering on the Yale dataset, as presented in this invention.

[0015] Figure 4 This is a convergence curve of the objective function for clustering on the COIL20 dataset, as presented in this invention. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments. The present invention includes, but is not limited to, the following embodiments.

[0017] The present invention provides an adaptive L 2p The norm-based fuzzy clustering method specifically includes the following steps: Step 1: Let the data samples to be clustered be... The objective function for the clustering algorithm is as follows:

[0018] In the formula: n —Sample size; c —The number of clusters in the sample; d -sample x i Dimensions i =1, 2, ..., n ; u ij —Membership matrix The i row and number j The elements of the column represent sample values. x i Belongs to the j The membership degree of each cluster; j =1, 2, ..., c Membership matrix ; —No. j The center of a cluster; set of cluster centers ; —No. j The covariance coefficients of each cluster; the set of covariance coefficients of each cluster. ; p — L 2p Parameters in the norm; λ >0 — Entropy regularization hyperparameter, which is used to avoid trivial solutions with membership degrees.

[0019] Step 2: Initialize the parameters in the objective function constructed in Step 1 to obtain the initialized data. The specific steps are as follows: Step 21, data samples to be clustered Normalization preprocessing is performed to obtain preprocessed data. In this embodiment, the min-max normalization method is preferred for normalization preprocessing. Step 22: Select the most suitable parameter values ​​for the entropy regularization hyperparameter from the reference parameter list using a grid search method. λ Perform initialization. The reference parameter list is [0.1, 0.5, 1, 2, 4, 6, 8, 12, 16, 20, 40, 100, 300]. Step 23, Initialization and p =1; Step 24, for the cluster center Random initialization is used.

[0020] Step 3: Use the alternating optimization method and the initialized data obtained in Step 2 to optimize and update each variable in the objective function constructed in Step 1 until the objective function converges. Specifically, this includes the following steps: Step 3.1: Update according to the following formula u ij : ; Step 3.2.1: Construct intermediate variables according to the following formula D ij ; ; Step 3.2.2: Update the following formula: j The center of each cluster : ; Step 3.2.3: Update the following formula: j Covariance coefficients of each cluster : ; Step 3.2.4: Judgment and If convergence has occurred, proceed to step 3.3; otherwise, repeat steps 3.2.1 to 3.2.3 until convergence is achieved. and Converge, then proceed to step 3.3.

[0021] Step 3.3: Update using Newton's method as shown in the following formula. p until p No longer changing;

[0022] in:

[0023] Step 3.4: Determine whether the objective function constructed in Step 1 has converged. If it has, output the membership matrix U; otherwise, repeat steps 3.1 to 3.3 until the objective function constructed in Step 1 has converged, and then output the membership matrix U.

[0024] To verify the feasibility and effectiveness of the method of the present invention, experiments were conducted on eight real datasets, and eight advanced algorithms were introduced for comparison.

[0025] The real datasets used in the experiments included Iris, wine, Segmentation, Digts, PalmData, Uspst, Yale32x32, and COIL20.

[0026] Comparison algorithms include K-Means (KM), Coordinate Descent K-Means (CDKM), Fuzzy C-Means (FCM), Fuzzy C-Means with Entropy Regularization (FCM-ER), Robust SparseFuzzy K-Means (RSFKM), Iteratively Re-Weighted Fuzzy C-Means (IRW-FCM), FuzzyK-Means with Pattern Shrinking (FKPS), Gaussian Mixture Model with SphericalCovariance (GMM-SC).

[0027] The method proposed in this invention is called Adaptive. L 2p -norm based Fuzzy C-Means (AL-FCM).

[0028] Table 1 provides detailed information on the eight real-world datasets used in the experiments. Tables 2 and 3 present the experimental results of the proposed method and eight comparative algorithms. To ensure fair results, the experiments were repeated ten times on each dataset, and the results were averaged and their standard deviations are given. Bold text indicates the best results, and underlined text indicates the second-best results. The evaluation criterion for clustering results in Table 1 is ACC, and the evaluation criterion for clustering results in Table 2 is NMI. It can be seen that the clustering effect of the proposed method is better than that of the traditional FCM.

[0029] Table 1

[0030] Table 2

[0031] Table 3

[0032] Figures 1-4 The graph shows the convergence curves of the objective function of the proposed method on four real datasets (Iris, PalmData, Yale, and COIL20). The vertical axis represents the objective function values, and the horizontal axis represents the number of algorithm iterations. It can be seen that the proposed method is indeed feasible.

[0033] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An adaptive-based L 2p The fuzzy clustering method based on norm is characterized by, Specifically, the steps include the following: Step 1: Let the data samples to be clustered be... The data samples to be clustered come from datasets Iris, PalmData, Yale, or COIL20; the objective function for constructing the clustering algorithm is as follows: In the formula: n —Sample size; c —The number of clusters in the sample; d -sample x i Dimensions i =1, 2, ..., n ; u ij —Membership matrix The i row and number j The elements of the column represent sample values. x i Belongs to the j The membership degree of each cluster; j =1, 2, ..., c Membership matrix ; —No. j The center of a cluster; set of cluster centers ; —No. j The covariance coefficients of each cluster; the set of covariance coefficients of each cluster. ; p — L 2p Parameters in the norm; λ >0 — Entropy regularization hyperparameter; Step 2: Initialize the parameters in the objective function constructed in Step 1 to obtain the initialized data. The specific operations are as follows: Step 21, data samples to be clustered X Perform normalization preprocessing to obtain the preprocessed data; Step 22: Select the most suitable parameter values ​​for the entropy regularization hyperparameter from the reference parameter list using a grid search method. λ Initialization is performed, wherein the reference parameter list is [0.1, 0.5, 1, 2, 4, 6, 8, 12, 16, 20, 40, 100, 300]; Step 23, Initialization and p =1; Step 24, for the cluster center Random initialization is used; Step 3: Use the method of alternating optimization of variables and the initialized data obtained in Step 2 to optimize and update each variable in the objective function constructed in Step 1 until the objective function converges.

2. The adaptive-based method as described in claim 1 L 2p The fuzzy clustering method based on norm is characterized by, In step 21, the normalization preprocessing is the min-max normalization method.

3. The adaptive-based method as described in claim 1 L 2p The fuzzy clustering method based on norm is characterized by, Step 3 specifically includes the following steps: Step 3.1: Update according to the following formula u ij : ； Step 3.2.1: Construct intermediate variables according to the following formula D ij : ； Step 3.2.2: Update the following formula: j The center of each cluster : ； Step 3.2.3: Update the following formula: j Covariance coefficients of each cluster : ； Step 3.2.4: Judgment and If convergence has occurred, proceed to step 3.3; otherwise, repeat steps 3.2.1 to 3.2.3 until convergence is achieved. and Convergence, then proceed to step 3.3; Step 3.3: Update using Newton's method as shown in the following formula. p until p No longer changing; in: ； Step 3.4: Determine whether the objective function constructed in Step 1 has converged. If it has, output the membership matrix U; otherwise, repeat steps 3.1 to 3.3 until the objective function constructed in Step 1 has converged, and then output the membership matrix U.

Images