A robust image clustering method based on discriminative embedding projective fuzzy clustering

Abstract

The present invention discloses a robust image clustering method based on discriminative embedded projected fuzzy clustering, which belongs to the field of image recognition and classification and pattern recognition. The present invention adopts a robust image clustering method based on discriminative embedded projected fuzzy clustering, embeds the optimal subspace projection learning into the fuzzy clustering algorithm for optimization, and embeds discriminative learning while removing redundant features, effectively enhancing the inter-class discriminability of samples in the projected optimal subspace, thereby suppressing the degradation of clustering performance under noise pollution. In addition, inspired by the entropy metric, a regularization term based on the maximum entropy principle is designed, and a dynamic information entropy graph is constructed to update the sample membership level distribution to mine a more reliable natural category division of the data. The present invention adaptively identifies noise-contaminated image data while robustly clustering, and eliminates its influence. It has high robustness to noise-contaminated image data and effectively improves the clustering performance of noise-contaminated data.

Description

[0001] Field

[0002] The invention relates to a robust image clustering method based on discriminative embedded projection fuzzy clustering, and belongs to the fields of image recognition and classification and pattern recognition. Background Art

[0003] In recent years, with the rapid development of sensors, the internet, and various digital devices, the amount of image data people are exposed to has exploded exponentially. While this image data contains rich information, it also presents challenges such as high complexity, data overload, and redundant feature information. Furthermore, data collected in an irregular manner or under harsh environments often exhibits high noise and low resolution. The poor availability of this data poses significant challenges to data analysis and model learning. Cluster analysis, a typical unsupervised learning method, not only mines the underlying structural information in data but also effectively reduces the workload of labeling large amounts of unlabeled data, providing powerful support for subsequent pattern recognition tasks such as image classification. Fuzzy clustering uses a partitioning matrix to evaluate the membership level of each sample and assign it to a cluster. Compared to hard-partition clustering, it provides a more flexible way to obtain natural clustering of data. Among them, fuzzy C-means clustering (FCM) is one of the most popular fuzzy clustering algorithms and has been widely used in image recognition and classification.

[0004] Pan Xingguang et al. (“Feature Reduction Fuzzy Clustering Algorithm Based on Marginal Kurtosis Measurement” Control and Decision, 2021, 36(11): 2665-2673.) used the marginal kurtosis measurement index to reduce relatively unimportant redundant features, aiming to delete features with weights less than a threshold from the feature level and retain only important features for clustering, thereby reducing the impact of redundant features on clustering. However, it treats all sample data including noise or abnormal data equally. Although the features of these abnormal samples are reduced, their harmful information is still retained. Therefore, it is more sensitive to noise-contaminated image data. In addition, it lacks consideration of discriminative information between data classes, so its ability to handle redundant features still needs to be improved, which further limits its clustering performance on complex real-world data. Summary of the Invention

[0005] To address the current challenges of complex and time-consuming image data label acquisition, noise pollution that degrades the performance of traditional fuzzy clustering algorithms by destroying the original data structure, and redundant image feature data that interferes with the discovery of underlying true data structures, the present invention employs a robust image clustering method based on discriminative embedded projection fuzzy clustering. This method embeds optimal subspace projection learning into the fuzzy clustering algorithm for optimization. This method removes redundant features while embedding discriminative learning, effectively enhancing the inter-class discriminability of samples in the projected optimal subspace, thereby suppressing the degradation of clustering performance in the presence of noise pollution. Furthermore, inspired by the entropy metric, a regularization term based on the maximum entropy principle is designed, and a dynamic information entropy graph is constructed to update the sample membership level assignment, thereby mining more reliable natural classification of the data. While robustly clustering, the present invention adaptively identifies and eliminates the effects of noise pollution on image data, demonstrating high robustness and effectively improving clustering performance on noise-contaminated data. Furthermore, the present invention is an unsupervised algorithm that avoids the time-consuming and labor-intensive acquisition of traditional image data labels and does not require any prior label data, enabling efficient and robust clustering of data containing noise pollution.

[0006] The present invention is achieved through the following technical solution, and its specific steps are as follows:

[0007] Step 1: Obtain image data information to construct a data matrix. For n images with resolution u×v, stretch each image to obtain a d×1 row vector, where d=u×v; convert the image data composed of n images into the target data matrix Each row of the matrix Represents an image, define x i is the original spatial sample data, each image is regarded as a sample; given the number of real categories c contained in the target data, the cluster centers of c clusters are randomly initialized, that is, the initial is the centroid of the jth cluster in the original space, and the centroid matrix of the original space clustering is

[0008] Step 2: Establish a robust discriminant embedded projection fuzzy clustering framework (Robust Discriminant Embedded Projection Fuzzy C-means Clustering, RDEPFCM). According to this framework, clustering operations are performed on the acquired noise-contaminated image data matrix. At the same time, in the optimal subspace of projection, the present invention can adaptively identify noise-contaminated images and mark them as outliers, thereby improving clustering performance, preventing abnormal data from distorting the membership level distribution, and avoiding the incorrect classification of outliers into normal sample clusters. Define the projection matrix Used to learn the optimal subspace information, the cluster centroid matrix M = H of the projection subspace can be obtained TΩ, the optimal subspace sample projection is given by H T x i Calculation is obtained. The main objective function of the framework is:

[0009]

[0010] stY≥0,Y1=1,S T 1=K,0≤S≤1,H T H=I

[0011] in, is the centroid of the jth ideal subspace cluster, and the centroid matrix of subspace clustering is Outlier constraint vector of the subspace It is used to adaptively identify nK outliers in the subspace and eliminate their influence, S i is the value of the i-th element of S, which can continuously correct the deviation of subspace prototype learning caused by outliers. λ and ρ are regularization parameters.

[0012] By regularizing the entropy of the second term of the objective function, a dynamic information entropy graph is constructed to more stably calculate the membership matrix. Maximize the retention of the original probability distribution information of the data, the element y in the matrix ij It represents the membership degree of the i-th sample in the optimal projection subspace to the j-th cluster. The third term of the objective function is used to minimize the sample reconstruction error of the projection subspace and embed the discriminative learning into the fuzzy clustering algorithm for optimization.

[0013] Step 3: Alternately iterate and optimize the objective function. Use the iterative reweighted optimization method to solve the four variables m, Y, H, and S in the objective function. First, initialize m, Y, and H and calculate S according to the formula. Then, fix S and m and optimize Y for both real samples and outliers. Then, fix S and Y and solve for m according to the formula. Finally, fix m, Y, and S to solve the subspace projection matrix H. Repeat this cycle until convergence.

[0014] The solution steps are as follows:

[0015] Step 3.1: Based on the randomly initialized cluster centers of c clusters, obtain the initial subspace cluster centroid vector Initialize all elements in Y to y ij =1 / c. Consider making the initial probability of the sample distribution in each class equal. Initialize the projection matrix The matrix satisfies (H (0) ) T H (0) =I. Define T(γ (i) ) is the sum of the membership-weighted projected subspace "sample-cluster centroid" residual and the sample projection reconstruction term.i) ) Sort them in ascending order and calculate the optimal solution of the outlier constraint vector S in the corresponding subspace:

[0016]

[0017] Step 3.2: Perform different optimizations on real samples and noise data to obtain Y.

[0018] In step 3.1 we get T(γ (i) ) sort the data sequence in ascending order and convert the corresponding original space sample x i Sorting can get the sorted data matrix Its corresponding sorted membership matrix

[0019] Step 3.2.1: For normal sample data that is not contaminated by noise. When the sample subspace projection H T x i The corresponding S i =1, the image data samples corresponding to the original space are considered to be normal image data not contaminated by noise. The optimal membership degree corresponding to the normal sample is obtained by optimization.

[0020]

[0021] in, for No. row vector, and Then it is the matrix No. The membership value of row j and column j.

[0022] Step 3.2.2: For the abnormal sample data adaptively identified during the optimization process. When the sample subspace projection H T x i The corresponding S i = 0, the image data samples corresponding to the original space are considered to be abnormal image data contaminated by noise. At this time, the optimal solution of the membership of all sample points can be obtained for:

[0023]

[0024] Step 3.3: Take the partial derivative of the objective function with respect to m and make it equal to 0, and you can get:

[0025]

[0026] Step 3.4: Define the outlier constraint matrix of the subspace is a diagonal matrix whose i-th diagonal element is S i . is a diagonal matrix, the jth diagonal element is Thus we obtain the matrix P=SY, where Defining the overall spread of real data And the intra-class divergence Outlier constraint matrix of subspace It is used to adaptively identify abnormal data and is a binary diagonal matrix that is only related to the data matrix. Therefore, we can get that minimizing the original objective function to obtain the optimal solution H is equivalent to solving the equivalent maximization problem:

[0027]

[0028] Since the projection matrix Satisfy the orthogonality constraint H T H=I, so the optimal solution of H is given by (ρS t -S W ) is composed of the eigenvectors corresponding to the first r largest eigenvalues of . It is worth emphasizing that this maximization problem can be described as maximizing the difference between the overall divergence and the intra-class divergence. Since the overall divergence is the sum of the inter-class divergence and the intra-class divergence, the above problem can be regarded as maximizing the inter-class divergence, which enhances the discriminative learning of the proposed framework for the inter-class division of samples. At this point, m, Y, H, and S are updated, and the next iterative operation is repeated until the objective function converges. At this time, x can be determined based on the maximum membership value. i Cluster markers Without any prior label information, the image sample x i Assign to corresponding cluster Thus, image clustering is achieved. According to the obtained sample cluster label λ i You can get the predicted label z of all image data p ,The predicted labels are only used to conveniently evaluate the clustering performance.

[0029] The present invention proposes a robust image clustering based on discriminative embedded projective fuzzy clustering, which can not only adaptively identify noise-contaminated image data and eliminate its influence during application, but also suppress the influence of redundant features and enhance the inter-class discriminative information of the sample projection space, thereby improving the robustness of the algorithm and the image clustering performance.

[0030] The beneficial effects of the method of the present invention mainly include:

[0031] (1) A robust fuzzy clustering algorithm based on discriminative embedding projection is proposed. In this algorithm, the outlier constraint vector of the optimal subspace is solved by iteratively optimizing the objective function, thereby assigning a binary weight to each sample subspace projection. Based on this, normal samples and noise-contaminated samples corresponding to the original space are adaptively identified, thereby enhancing the robustness of the algorithm.

[0032] (2) While performing robust clustering, the present invention constructs a maximization inter-class divergence problem that is equivalent to the original objective function, enhances the inter-class discrimination of samples in the optimal projection subspace, suppresses redundant features to obtain a more effective sample and feature distribution structure, avoids the influence of noise pollution on image data clustering, reduces the amount of data storage, reduces the amount of data calculation, and improves calculation efficiency.

[0033] (3) Inspired by the entropy metric, the present invention further maximizes the retention of the probability distribution information of the original data in a stable and conservative manner based on the maximum entropy principle, designs an entropy regularization term for membership, constructs a dynamic information entropy graph to more stably update the membership matrix, and mines more realistic potential cluster divisions of the data, further improving the image clustering performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above steps and other objects, features and advantages of the present invention will be apparent from the following detailed description, in which:

[0035] Figure 1 It is the algorithm flow chart;

[0036] Figure 2 Some examples of noise-contaminated dig1-10 handwritten character images:

[0037] in, Figure 2 (e), (f), (h), and (j) are noise-contaminated images, and the rest are original normal image data in the dig1-10 dataset.

[0038] Figure 3 The experimental results of the present invention on the dig1-10 handwritten character image dataset. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0040] The comprehensive model solution process of robust image clustering based on discriminative embedded projected fuzzy clustering in this invention is as follows: Figure 1As shown in the figure, the dig1-10 handwritten character image dataset is used for clustering. The dig1-10 dataset contains a total of 1797 handwritten character images, including rich character images such as handwritten letters, numbers, mathematical operators, etc. The resolution is 8×8, and each handwritten character image corresponds to a sample. The true labels corresponding to the 1797 images are The predicted labels of 1797 images obtained by the clustering algorithm are The true label z of the dataset itself t This is only used to verify the final clustering effect and is not used as any input information for the clustering algorithm. To test the algorithm, we add 10% noise contaminated samples as an example. At this time, the number of noise contaminated image samples in the dig1-10 dataset is recorded as p = 359. The specific implementation method includes the following steps:

[0041] Step 1: Input the data matrix consisting of dig1-10 images The number of true categories c, the number of noise-contaminated image samples p = 359, where each column of the matrix is a sample, n = 1797 is the number of samples, d = 8 × 8 = 64 is the dimension of the data matrix, and c = 10 is the number of true categories contained in the face data.

[0042] Step 2: Establish a robust discriminant embedded projection fuzzy C-means clustering framework (RDEPFCM). The main objective function of the framework is:

[0043]

[0044] stY≥0,Y1=1,S T 1=K,0≤S≤1,H T H=I

[0045] According to this framework, clustering operations are performed on the acquired noise-contaminated image data matrix. At the same time, in the optimal subspace of the projection, the present invention can adaptively calculate S to identify noise-contaminated images and mark them as outliers, thereby improving clustering performance, preventing abnormal data from distorting the membership level distribution, and avoiding outliers from being mistakenly classified into normal sample clusters. The second term of the objective function is the entropy regularization term of the membership, thereby constructing a dynamic information entropy graph in the optimization to more stably calculate the membership matrix. The original probability distribution information of the data is preserved to the maximum extent. The third term of the objective function is used to minimize the sample reconstruction error of the projected subspace and embed discriminative learning into the fuzzy clustering algorithm for optimization to suppress the impact of noise pollution on clustering performance.

[0046] Step 3: Alternately iterate and optimize the objective function. The detailed solution steps are as follows:

[0047] Step 3.1: Randomly initialize the cluster centers of c clusters, that is, obtain the initial m. Initialize all elements in Y to y ij =1 / c=1 / 10, K=np=1438. Initialize the projection matrix The matrix satisfies (H (0) ) T H (0) =I. Consider making the initial probability of the sample distribution in each class equal. That is, we can fix m, Y, H and calculate S. Fixing m, Y, H, the objective function is transformed into:

[0048]

[0049] s T 1=K,0≤S≤1

[0050] Define T(γ( i )) is the sum of the membership-weighted projected subspace "sample-cluster centroid" residual and the sample projection reconstruction term:

[0051]

[0052] T(γ( i) ) According to the order from small to large, we can get the outlier constraint vector S of the corresponding subspace in the constraint S T The optimal solution under the condition of 1=K is:

[0053]

[0054] According to the binary weight distribution of the sample subspace projection of the vector S, the normal samples and noise pollution samples corresponding to the original space can be adaptively identified, thereby eliminating the influence of noise pollution abnormal data on the objective function and correcting the clustering prototype learning error caused by abnormal values. i The value of is assigned to the noise-contaminated data as an abnormal label, providing preprocessing for unsupervised abnormal data identification for subsequent image classification and recognition.

[0055] Step 3.2: Optimize the normal sample and the noise contaminated sample data to obtain Y. In step 1, we get T(γ( i )) Sort the data sequence in ascending order and convert the corresponding original space sample x i Sorting can get the sorted data matrix Its corresponding sorted membership matrix

[0056] Step 3.2.1: For normal sample data that is not contaminated by noise. When the sample subspace projection H T x i The corresponding S i = 1, the image data samples corresponding to the original space are considered to be normal image data not contaminated by noise. i =1, the sub-problem of the objective function is:

[0057]

[0058] in, for No. row vector, and Then it is the matrix No. The membership value of row j and column j.

[0059] Construct the Lagrangian function of the above formula:

[0060]

[0061] By Lagrangian function and η i The derivatives of are all 0, which can eliminate the unknown Lagrange multiplier η i , and optimize the normal sample to obtain the optimal membership

[0062]

[0063] Step 3.2.2: For the abnormal sample data adaptively identified during the optimization process. When the sample subspace projection H T x i The corresponding S i = 0, the image data samples corresponding to the original space are considered to be abnormal image data contaminated by noise. According to the extreme value theorem, the optimal membership value corresponding to the noise contaminated data in the optimization process can be solved as follows:

[0064]

[0065] The optimal solution of the membership of all sample points can be obtained for:

[0066]

[0067] Step 3.3: Fix S, Y, and H and solve for m.

[0068] At this time, the objective function subproblem is rewritten as:

[0069]

[0070] Taking the partial derivative of the objective function subproblem with respect to m as 0, we can get the solution:

[0071]

[0072] Step 3.4: Fix S, Y, and m and solve for H.

[0073] At this point, the above objective function is transformed into:

[0074]

[0075] By converting the above equation into an equivalent equation, we can get the following equation:

[0076]

[0077] Among them, the outlier constraint matrix of the subspace is is a diagonal matrix whose i-th diagonal element is S i . is a diagonal matrix, the jth diagonal element is The matrix P = SY, and The overall dispersion of the data And the intra-class divergence

[0078] Outlier constraint matrix of subspace It is used to adaptively identify abnormal data and is a binary diagonal matrix that is only related to the data matrix. Therefore, we can get that minimizing the original objective function to obtain the optimal solution H is equivalent to solving the equivalent maximization problem:

[0079]

[0080] Since the projection matrix Satisfy the orthogonality constraint H T H=I, so the optimal solution of H is given by (ρS t -S W ) consists of the eigenvectors corresponding to the first r largest eigenvalues. It is worth emphasizing that this maximization problem can be described as maximizing the difference between the overall divergence and the intra-class divergence. Since the overall divergence is the sum of the inter-class divergence and the intra-class divergence, the above problem can be viewed as maximizing the inter-class divergence, which enhances the discriminative learning of the proposed ICDEFKM framework for classifying samples between classes.

[0081] At this point, m, Y, H, and S are updated, and the next iteration is performed again until the objective function converges. The binary weight distribution penalizes the abnormal data in the subspace, and finally adaptively identifies 359 noise-contaminated image samples corresponding to the original space. More accurate cluster prediction labels are obtained for 1797 handwritten character images. Improve image clustering performance. By comparing the real label z t And the predicted label z obtained by the clustering algorithm p To calculate relevant clustering performance evaluation indicators, such as clustering accuracy and clustering normalized mutual information. Take the dig1-10 handwritten character image dataset (1797 images, each with 8×8 pixels) as an example. When 10% noise is added, the clustering accuracy of FCM on the dig1-10 handwritten character image data is only 48.84%, and the clustering normalized mutual information is 45.54%. When 10% noise is added, the noise suppression image clustering based on robust fuzzy clustering proposed in the present invention has a clustering accuracy of 78.90% on the dig1-10 handwritten character image dataset, and the clustering normalized mutual information is 79.06%, which are improved by 30.06% and 33.52% respectively, significantly improving the clustering accuracy of handwritten character image data containing noise-contaminated images.

Claims

1. A robust image clustering method based on discriminative embedded projective fuzzy clustering, characterized by: Here are the steps: Step 1: Obtain image data information and construct a data matrix. For n images with resolution u×v, stretch each image to obtain a d×1 row vector, where d=u×v; convert the image data composed of n images into the target data matrix. Each row of the matrix Represents an image, define x i is the original spatial sample data, each image is regarded as a sample; given the number of real categories c contained in the target data, the cluster centers of c clusters are randomly initialized, that is, the initial is the centroid of the jth cluster in the original space, and the centroid matrix of the original space clustering is Step 2: Establish a robust discriminant embedded projection fuzzy clustering framework (Robust Discriminant Embedded Projection Fuzzy C-means Clustering, RDEPFCM) to perform clustering operations on the acquired noise-contaminated image data matrix. At the same time, in the optimal subspace of the projection, the noise-contaminated image is adaptively identified and marked as an outlier, thereby improving the clustering performance, preventing abnormal data from distorting the membership level assignment, and avoiding the outliers from being incorrectly divided into normal sample clusters. Define the projection matrix Used to learn the optimal subspace information, the cluster centroid matrix M = H of the projection subspace can be obtained T Ω, the optimal subspace sample projection is given by H T x i The main objective function of the calculation acquisition framework is: s.t.Y≥0,Y1=1,S T 1=K,0≤S≤1,H T H=I in, is the centroid of the jth ideal subspace cluster, and the centroid matrix of subspace clustering is Outlier constraint vector of the subspace It is used to adaptively identify nK outliers in the subspace and eliminate their influence, S i is the value of the i-th element of S, which continuously corrects the deviation λ of the subspace prototype learning caused by the outliers, and ρ is the regularization parameter By regularizing the entropy of the second term of the objective function, a dynamic information entropy graph is constructed to more stably calculate the membership matrix. Maximize the retention of the original probability distribution information of the data, the element y in the matrix ij The third term of the membership objective function, which indicates that the i-th sample in the optimal projection subspace belongs to the j-th cluster, is used to minimize the sample reconstruction error of the projection subspace and embed the discriminative learning into the fuzzy clustering algorithm for optimization. Step 3: Alternately iterate and optimize the objective function. Use iterative reweighted optimization to solve the four variables m, Y, H, and S in the objective function. First, initialize m, Y, and H, and calculate S according to the formula. Then, fix S and m, and optimize Y for both real samples and outliers. Then, fix S and Y, and solve for m according to the formula. Finally, fix m, Y, and S to solve the subspace projection matrix H, and repeat the cycle until convergence. The solution steps are as follows: Step 3.1: Based on the randomly initialized cluster centers of c clusters, obtain the initial subspace cluster centroid vector Initialize all elements in Y to y ij = 1 / c Considering that the initial probability of the sample distribution in each class is equal, the projection matrix is initialized. The matrix satisfies (H (0) ) T H (0) =I define T(Y (i) ) is the sum of the membership-weighted projection subspace "sample-cluster centroid" residual and the sample projection reconstruction term. (i) ) Sort them in ascending order and calculate the optimal solution of the outlier constraint vector S in the corresponding subspace: Step 3.2: Optimize the real sample and noise data to get Y In step 3.1, we get T( (i) ) sort the data sequence in ascending order and convert the corresponding original space sample x i Sort and get the sorted data matrix Its corresponding sorted membership matrix Step 3.2.1: For normal sample data not contaminated by noise, when the sample subspace projection H T x i The corresponding S i =1, the image data samples corresponding to the original space are considered to be normal image data not contaminated by noise. The optimal membership degree corresponding to the normal sample is obtained by optimization. in, for No. row vector, and Then it is the matrix No. Membership value of row j column Step 3.2.2: For the abnormal sample data adaptively identified during the optimization process, when the sample subspace projection H T x i The corresponding S i = 0, the image data samples corresponding to the original space are considered to be abnormal image data contaminated by noise. At this time, the optimal solution of the membership of all sample points is obtained. for: Step 3.3: Take the partial derivative of the objective function with respect to m and get it equal to 0. Step 3.4: Define the outlier constraint matrix of the subspace is a diagonal matrix whose i-th diagonal element is is a diagonal matrix, the jth diagonal element is Thus we obtain the matrix P=SY, where Defining the overall spread of real data And the intra-class divergence Outlier constraint matrix of subspace It is used to adaptively identify abnormal data and is a binary diagonal matrix related only to the data matrix. Therefore, minimizing the original objective function to obtain the optimal solution H is equivalent to solving the equivalent maximization problem: Since the projection matrix Satisfy the orthogonality constraint H T H=I, so the optimal solution of H is given by (ρS t -S W ) corresponds to the eigenvectors of the first r largest eigenvalues. It is worth emphasizing that the maximization problem is described as maximizing the difference between the overall divergence and the intra-class divergence. Since the overall divergence is the sum of the inter-class divergence and the intra-class divergence, the above problem can be regarded as maximizing the inter-class divergence, which enhances the discriminative learning of the proposed framework for the inter-class division of samples. At this point, m, Y, H, and S are updated, and the next iterative operation is repeated until the objective function converges. At this time, x is determined according to the maximum membership value. i Cluster markers Without any prior label information, the image sample x i Assign to corresponding cluster Thus, image clustering is realized, wherein, according to the obtained sample cluster label λ i Get the predicted label z for all image data p ,The predicted labels are only used to conveniently evaluate the clustering performance.

Images