Deep embedding subspace clustering method based on double depolarization contrast learning

Abstract

The invention relates to a deep embedded subspace clustering method based on double depolarization contrast learning. The method comprises the following steps: selecting a public image data set as a sample set and constructing a clustering model; the method comprises the following steps: firstly, pre-training an auto-encoder, then enhancing data of a data set, inputting the data into the encoder to obtain embedded representation of the data set, and clustering the data set by using a k-means method to obtain an initialized subspace base; secondly, training a clustering model, to be more specific, enhancing data of a data set, inputting the data into two encoders to obtain embedded representations of the enhanced data, performing debiased positive sampling and debiased negative sampling on the embedded representations, and reconstructing the embedded representations of the two enhanced data through a decoder; the distribution probability of each embedded representation in each subspace is calculated by utilizing a subspace base, so that the clustering category of each sample is determined. According to the method, the autoencoder is trained by optimizing the positive and negative sample structures and the sample reconstruction distance, and meanwhile, the subspace base is iteratively optimized, so that a high-quality clustering model is obtained.

Description

Technical Field

[0001] The present invention relates to the field of deep learning, and in particular to an image clustering method based on deep embedding subspace, which effectively improves clustering accuracy and robustness by constructing a multi-subspace representation and joint optimization framework. Background Art

[0002] With the development of deep learning, deep clustering methods have been widely used in fields such as images, text, and speech. Traditional clustering methods such as K-means and spectral clustering rely on shallow features and have difficulty capturing complex structures. In recent years, deep clustering models based on autoencoders, such as DEC and IDEC, have significantly improved clustering performance by combining feature learning with clustering optimization. On the other hand, subspace clustering methods such as DSC (Deep Subspace Clustering) have shown good results when processing high-dimensional data by learning sparse representations of samples in low-dimensional subspaces. However, existing deep subspace clustering methods mainly rely on minimizing reconstruction errors, ignoring the discriminative relationship between samples, and have difficulty in effectively dividing complex category boundaries. At the same time, contrastive learning, as a representative paradigm of unsupervised representation learning, has been widely used in fields such as images and speech. Methods such as SimCLR, MoCo, and BYOL effectively improve the discriminative ability of features by maximizing the consistency between different enhanced views, providing strong feature support for deep clustering. However, directly introducing contrastive learning into clustering scenarios still faces two key challenges:

[0003] There is a bias in the construction of positive and negative samples: Due to the lack of label information, data-augmented view pairs are often used as positive samples when constructing positive and negative samples, while the remaining samples are considered negative. This "view assumption" easily leads to the introduction of false negatives and false positives, weakening the effectiveness of contrastive learning and even interfering with the feature space. There is a lack of guidance from structural clustering objectives: Existing contrastive learning methods often focus on instance differentiation as their core goal, failing to incorporate the structural requirements of "complex clusters within the same cluster and separate clusters across different clusters" in clustering tasks. This leads to blurred cluster boundaries in the feature space, limiting the ultimate clustering effect.

[0004] In addition, some studies have attempted to introduce clustering structure constraints based on contrastive learning, such as SCN, CC, and PICA. However, most methods still have the following problems:

[0005] First, the potential subspace structure information between samples is not fully considered, and the low-dimensional substructure of the manifold where the samples are located in the high-dimensional space is ignored; second, the training goal focuses on feature consistency, and the impact of misclassified samples (especially false negative samples) on the final clustering effect is not explicitly considered; third, there is a lack of modeling and debiasing mechanism for the distribution of positive and negative samples, which can easily cause inter-class confusion and intra-class collapse during the optimization process.

[0006] In summary, current research on combining deep subspace clustering with contrastive learning is still in its infancy, facing key technical bottlenecks such as inaccurate construction of positive and negative samples, missing clustering targets, and ineffective modeling of sample distribution biases. Therefore, there is an urgent need for a deep clustering method that can jointly model subspace structure, debias the distribution of positive and negative samples, and impose structural clustering constraints. Summary of the Invention

[0007] In response to the above-mentioned problems existing in the prior art, the technical problem to be solved by the present invention is: how to improve the clustering accuracy of data based on deep embedding learning.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A deep embedding subspace clustering method based on double debiased contrastive learning includes the following steps:

[0010] S100: Select a public image dataset as a sample set, where each image in the sample set is a sample, and the sample set contains n samples; construct a clustering model M, where M includes an autoencoder network module and a subspace basis matrix module. The autoencoder network module includes two encoders and two decoders;

[0011] S200: Initialize the autoencoder network module and select the i-th sample x from the sample set i , and then use two independent data augmentation methods to i Processing to obtain enhanced data and The expression is as follows:

[0012]

[0013] Among them, T a and T b Represents two different data enhancement methods, and Represents the enhanced samples obtained by the corresponding data enhancement method;

[0014] S300: and As input to the autoencoder network module:

[0015] and Each is encoded by an encoder to obtain The corresponding latent feature embedding and The corresponding latent feature embedding Then and Then pass through two identical decoders to get The corresponding restored image and The corresponding restored image

[0016] in accordance with and Use mean square error MSE to construct the loss function L recon , the expression is as follows:

[0017]

[0018] Where n represents the total number of samples in the sample set;

[0019] The enhanced data and restored images of all samples are used as training data, and L recon Pre-train the autoencoder network module. When the number of pre-training times reaches the maximum or L recon When the minimum value is reached, the pre-training is stopped and the pre-trained autoencoder network module is obtained;

[0020] The pre-trained autoencoder network module and subspace basis matrix module are taken as M', and then a joint optimization objective function L is constructed to train M':

[0021] S400: and Perform linear combination calculation to get the embedded representation z i ; Traverse all samples in the sample set and obtain the embedded representation set Z of all samples;

[0022] The K-means clustering method is used to cluster the embedded representation in Z to obtain K categories, and the corresponding initialization subspace basis matrix set B is obtained through the K categories, B=[B (1) ,B (2) ,...,B (K) ], represents the j-th subspace basis matrix and j∈[1,K], b represents the number of basis vectors of all subspaces, and p represents the dimension of each basis vector;

[0023] Use B to define the subspace constraint loss function L sub-1 , L sub-1 The calculation expression is as follows:

[0024]

[0025] Among them, B T represents the transposed matrix of B, ⊙ represents element-by-element multiplication, I represents a diagonal matrix of size k×k, with diagonal elements set to 1 and the remaining elements set to 0; F represents the formula representation, and O represents the parameter for eliminating the self-influence of corner blocks;

[0026] S500: Calculate the affinity a of the i-th sample to the j-th subspace basis ij , through a ij Then calculate the normalized distribution probability q of the i-th sample to the j-th subspace basis ij , the calculation expression is as follows:

[0027]

[0028] Among them, γ is the adjustment term to avoid the denominator being 0, and b is the number of subspace basis vectors;

[0029] Using q ij Construct KL divergence loss function L sub-2 ,

[0030]

[0031] Among them, Q represents the subspace allocation matrix, P represents the redefined allocation probability, and p ij represents the refined subspace affinity vector;

[0032] S600: Constructing a double debiasing contrast loss function L dcl Used to ensure the stability of autoencoding network learning, the formula is as follows:

[0033]

[0034] Among them, p pos represents the positive sample set, p neg represents the negative sample set, F d-neg Debiasing negative sample loss, F d-pos Debiasing positive sample loss, For all possible sample pairs (z,z + ) performs expected calculation, z represents the anchor point sample, z - Represents the distribution of negative samples p neg The negative samples obtained by sampling, z + is from the positive sample distribution p pos The positive samples are obtained by sampling, s(·,·) represents the cosine similarity function, v represents the false negative samples, and W represents the weighting parameter;

[0035] S700: Joint Lrecon , L dcl , L sub-1 and L sub-2 As the loss function L of M', the calculation expression is as follows:

[0036] L=L recon +λ1L dcl +λ2L sub-1 +λ3L sub-2

[0037] Among them, λ1, λ2, λ3 are hyperparameters that control the balance between different loss terms;

[0038] The sample set is used as the training set, and the Adam optimizer is used to train M′ using L. The training is stopped when L reaches the minimum value or the training reaches the maximum number of iterations, and the trained clustering model M″ is obtained;

[0039] S800: Select a data set Y to be clustered, input Y into M″, and obtain the clustering results of all data in Y.

[0040] Preferably, the data enhancement method in S200 is random cropping and color perturbation.

[0041] As a preference, the step S300 obtains The corresponding latent feature embedding and The corresponding latent feature embedding The calculation expression is as follows:

[0042]

[0043] in, and Represents the corresponding two encoder networks, and Denotes the corresponding encoder weights.

[0044] As a preference, the step S300 obtains The corresponding restored image and The corresponding restored image The calculation expression is as follows:

[0045]

[0046] in, and represents two decoder networks, and Denotes the corresponding decoder weight.

[0047] As a preference, the S400 utilizes and The linear combination is calculated to get z i The calculation expression is as follows:

[0048]

[0049] Among them, z i represents the embedding representation of the i-th sample.

[0050] As an advantage, the double debiasing contrast loss function L is constructed in S600. dcl Also included:

[0051] In the obtained and In is regarded as an anchor sample z, and its positive sample set is defined as: The negative sample set is defined as: Then the joint probability distribution of the data is defined as: p (z,k) (z,k)=p(z|k)ρ(k), where ρ(k)=θ + represents the prior probability of the kth class;

[0052] Select negative samples z that are similar to the anchor sample z - , define negative sample z - The distribution of is:

[0053] p neg (z - )∝p(z - )exp(βs(z,z - ))

[0054] Among them, β≥0 represents the concentration parameter, which is used to control the similarity between negative samples and anchor samples, s(z,z - ) represents the cosine similarity between the anchor sample and the negative sample;

[0055] The negative sample distribution p neg (z - ) decomposes into:

[0056]

[0057] in, Indicates the true negative sample distribution, that is, the negative samples that truly belong to other categories. represents the distribution of false negative samples, i.e., samples that actually belong to the anchor class but are misclassified as negative samples, θ + represents the prior probability of the class, θ - Indicates the division of θ + The prior probability of other classes.

[0058] Compared with the prior art, the present invention has at least the following advantages:

[0059] 1. Introduce a double debiased contrastive learning mechanism to alleviate the problem of bias in the construction of positive and negative samples. The present invention introduces a probability adjustment mechanism in negative sampling and positive sampling respectively by designing a double debiased contrastive loss function, which effectively solves the problem of false negative samples and false positive samples caused by random sampling in existing contrastive learning methods. Specifically, this method introduces a distribution adjustment strategy based on the weighted similarity of anchor samples for negative sample sampling, which excludes "false negative samples" with high similarity to anchor samples; in terms of positive sampling, suspected false negative samples are further reclassified as positive samples and included in the positive pair set for optimization. This strategy effectively alleviates the problem of training instability caused by sampling bias in contrastive learning, thereby improving the discriminative representation ability of samples in the deep embedding space and providing a clearer feature structure for downstream clustering tasks.

[0060] 2. Combine subspace modeling and contrast discrimination to improve the accuracy and structure of high-dimensional complex data clustering. The present invention designs a deep clustering framework based on encoder-subspace-decoder, which performs triple constraints on the feature representation of input samples through joint reconstruction loss, subspace projection loss and double debiasing contrast loss. Among them, the subspace module learns the representation coefficient of each sample in the low-dimensional subspace, effectively capturing the local structural relationship of high-dimensional data, and the contrast loss further strengthens the intra-class compactness and inter-class separability through the similarity between samples. Compared with the traditional deep subspace clustering method based on reconstruction error, the method of the present invention not only models the geometric structure of the feature space, but also significantly enhances the discriminability of the representation, and finally obtains an embedding space that is more conducive to the formation of a clear cluster structure, thereby improving the clustering accuracy and cluster interpretability.

[0061] 3. Jointly optimize the discriminability of features and the structural clustering objective. This method proposes a joint optimization objective that explicitly integrates structural subspace constraints and semantic contrast supervision. This objective simultaneously strengthens cluster structure generation and inter-class boundary reinforcement during training, avoiding the dilemma of traditional methods: "reconstruction alone but weak discriminability" or "contrast enhancement but blurred cluster structure." Experiments demonstrate that the joint loss function can simultaneously optimize the global cluster distribution structure and the relationship between local similar samples, improving the model's convergence stability and cluster consistency under different initial conditions.

[0062] 4. High-performance unsupervised clustering can be achieved without labels, with good task generalization. This invention adopts a completely unsupervised learning process, does not rely on any label information, and performs self-supervised modeling only through comparative learning mechanisms and structural priors. Therefore, it is suitable for various practical scenarios that lack labeled data (such as medical images, social network data, financial transaction behaviors, etc.); at the same time, this method can be directly transferred and applied to different modal data (images, text, time series, etc.) and various data structures (sparse, high-dimensional, nonlinear distribution, etc.), showing good task generalization capabilities.

[0063] 5. It has good data privacy protection capabilities, avoiding the risk of label leakage and exposure of original samples. The comparative learning process of the present invention operates at the feature level, using the embedded representation z of the sample rather than the original input x, and does not rely on manual annotation or explicit category information. Therefore, the true label or original data content will not be leaked during training and deployment. This is especially important in fields such as medicine and finance that require high data sensitivity. It can effectively protect data privacy and security and comply with the technical requirements of relevant privacy protection regulations such as GDPR.

[0064] 6. Applicable to various deep clustering frameworks, with good scalability and compatibility. The proposed dual debiasing contrastive learning mechanism can be flexibly integrated into various existing deep clustering architectures (such as DEC, IDEC, DSC, DCC, etc.) as a modular loss component. It can also be extended to other unsupervised learning scenarios such as graph neural networks, self-supervised learning, and self-representation learning, showing high versatility and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a simplified flow chart of the method of the present invention;

[0066] Figure 2 t-SNE visualization results of the embedding representation of the proposed method and the existing state-of-the-art methods (EDESC, DMICC) on the STL-10 dataset;

[0067] Figure 3 t-SNE visualization of the evolution of the embedded representation of the proposed method when trained on the STL-10 dataset. DETAILED DESCRIPTION

[0068] The present invention is described in further detail below.

[0069] See also Figure 1-Figure 3 , a deep embedding subspace clustering method based on double debiased contrastive learning, comprising the following steps:

[0070] S100: Select a public image dataset as a sample set, where each image in the sample set is a sample, and the sample set contains n samples. Construct a clustering model M, where M includes an autoencoder network module and a subspace basis matrix module. The autoencoder network module includes two encoders and two decoders. The encoder and decoder form an autoencoder network, and both encoder and decoder are derived from convolutional neural networks, which are existing technologies.

[0071] S200: Initialize the autoencoder network module and select the i-th sample x from the sample set i , and then use two independent data augmentation methods to i Processing to obtain enhanced data and The expression is as follows:

[0072]

[0073] Among them, T a and T b Represents two different data enhancement methods, and Represents the enhanced samples obtained by the corresponding data enhancement method;

[0074] The data enhancement methods in S200 include random cropping and color perturbation. Random cropping involves randomly cropping the image into different regions and then scaling them. Color perturbation involves randomly adjusting the brightness, contrast, saturation, etc. of the image to generate different views.

[0075] S300: and As input to the autoencoder network module:

[0076] and Each is encoded by an encoder to obtain The corresponding latent feature embedding and The corresponding latent feature embedding Then and Then pass through two identical decoders to get The corresponding restored image and The corresponding restored image

[0077] in accordance with and Use mean square error MSE to construct the loss function L recon , the expression is as follows:

[0078]

[0079] Where n represents the total number of samples in the sample set; by optimizing the encoder, the key features of the input samples can be effectively captured; by optimizing the decoder, the reconstructed samples can restore a structure similar to the original input samples, retaining as much information as possible from the local structure and feature diversity of the data;

[0080] The enhanced data and restored images of all samples are used as training data, and L recon Pre-train the autoencoder network module. When the number of pre-training times reaches the maximum or L recon When the minimum value is reached, the pre-training is stopped and the pre-trained autoencoder network module is obtained;

[0081] The pre-trained autoencoder network module and subspace basis matrix module are taken as M', and then a joint optimization objective function L is constructed to train M':

[0082] The S300 is obtained The corresponding latent feature embedding and The corresponding latent feature embedding The calculation expression is as follows:

[0083]

[0084] in, and Represents the corresponding two encoder networks, and Denotes the corresponding encoder weights.

[0085] The S300 is obtained The corresponding restored image and The corresponding restored image The calculation expression is as follows:

[0086]

[0087] in, and represents two decoder networks, and Denotes the corresponding decoder weight.

[0088] S400: and Perform linear combination calculation to get the embedded representation z i ; Traverse all samples in the sample set and obtain the embedded representation set Z of all samples;

[0089] The K-means clustering method is used to cluster the embedded representation in Z to obtain K categories, and the corresponding initialization subspace basis matrix set B is obtained through the K categories, B=[B (1) ,B (2) ,...,B (K) ], represents the j-th subspace basis matrix and j∈[1,K], b represents the number of basis vectors of all subspaces, and p represents the dimension of each basis vector; the k-means clustering method is an existing technology;

[0090] Use B to define the subspace constraint loss function L sub-1 , L sub-1 The calculation expression is as follows:

[0091]

[0092] Among them, B T represents the transposed matrix of B, ⊙ represents element-by-element multiplication, I represents a diagonal matrix of size k×k, with diagonal elements set to 1 and all other elements set to 0; F represents the formula representation, and O represents the parameter for eliminating the self-influence of corner blocks; O is used to eliminate the influence of diagonal blocks on their own subspaces to highlight the differences between subspaces, and B is continuously optimized to ensure that the basis vectors of each subspace are orthogonal;

[0093] The S400 utilizes and The linear combination is calculated to get z i The calculation expression is as follows:

[0094]

[0095] Among them, z i represents the embedding representation of the i-th sample.

[0096] S500: Calculate the affinity a of the i-th sample to the j-th subspace basis ij , through a ij Then calculate the normalized distribution probability q of the i-th sample to the j-th subspace basis ij , the calculation expression is as follows:

[0097]

[0098] Among them, γ is the adjustment term to avoid the denominator being 0, and b is the number of subspace basis vectors;

[0099] Using q ij Construct KL divergence loss function L sub-2 ,

[0100]

[0101] Among them, Q represents the subspace allocation matrix, P represents the redefined allocation probability, and p ij represents the refined subspace affinity vector; p ij It's a pair of q ij The purpose of post-processing and refining the distribution is usually to emphasize the distribution that the model is most confident in, that is, to make the large ones larger and the small ones smaller, so that the categories they belong to are clearer, which can improve the discriminability of clustering;

[0102] By optimizing L sub-1 and L sub-2 These two loss functions gradually aggregate the embedded features into a subspace suitable for their characteristics, thereby improving the clustering effect;

[0103] S600: Constructing a double debiasing contrast loss function L dcl Used to ensure the stability of autoencoding network learning, the formula is as follows:

[0104]

[0105]

[0106] Among them, p pos represents the positive sample set, p neg represents the negative sample set, F d-neg Debiasing negative sample loss, F d-pos Debiasing positive sample loss, For all possible sample pairs (z,z + ) performs expected calculation, z represents the anchor point sample, z - Represents the distribution of negative samples p neg The negative samples obtained by sampling are samples of different categories from the anchor samples. We hope that the model can recognize the difference between them. + is from the positive sample distribution p pos The positive samples sampled from the embedding space belong to the same category as the anchor samples. We hope that the model can identify the similarity between them. s(·,·) represents the cosine similarity function, v represents the false negative samples, and W represents the weighting parameter. A contrastive loss function is used to promote positive images (different enhancements of the same image) to be closer in the embedding space, while negative images (different images) are farther away. To reduce the bias of negative samples, a double debiasing contrastive learning method is introduced to adjust the contribution weight of negative samples, making the model learning more stable.

[0107] The double debiasing contrast loss function L is constructed in S600 dcl Also included:

[0108] In the obtained and In is regarded as an anchor sample z, and its positive sample set is defined as: The negative sample set is defined as: Then the joint probability distribution of the data is defined as: p (z,k) (z,k)=p(z|k)ρ(k), where ρ(k)=θ + represents the prior probability of the kth class;

[0109] Select negative samples z that are similar to the anchor sample z - , define negative sample z - The distribution of is:

[0110] p neg (z - )∝p(z - )exp(βs(z,z - ))

[0111] Among them, β≥0 represents the concentration parameter, which is used to control the similarity between negative samples and anchor samples, s(z,z - ) represents the cosine similarity between the anchor sample and the negative sample;

[0112] The negative sample distribution p neg (z - ) decomposes into:

[0113]

[0114] in, Indicates the true negative sample distribution, that is, the negative samples that truly belong to other categories. represents the distribution of false negative samples, i.e., samples that actually belong to the anchor class but are misclassified as negative samples, θ + represents the prior probability of the class, θ - Indicates the division of θ + Prior probabilities of other classes outside the class; in the embedding space composed of the embedding representations of all samples, debiased positive sampling and debiased negative sampling are performed, that is, false negative samples are removed from the negative sample distribution and false negative samples are incorporated into the positive sample set;

[0115] S700: Joint L recon , L dcl , L sub-1 and L sub-2 As the loss function L of M', the calculation expression is as follows:

[0116] L=L recon +λ1L dcl +λ2L sub-1 +λ3L sub-2

[0117] Among them, λ1, λ2, λ3 are hyperparameters that control the balance between different loss terms;

[0118] The sample set is used as the training set, and the Adam optimizer is used to train M′ using L. The training is stopped when L reaches the minimum value or the training reaches the maximum number of iterations, and the trained clustering model M″ is obtained. The Adam optimizer is an existing technology.

[0119] S800: Select a data set Y to be clustered, input Y into M″, and obtain the clustering results of all data in Y.

[0120] The steps of obtaining the clustering results of all data in Y in S800 are as follows:

[0121] S810: Input all data in Y into M″, and obtain the embedding representation set Z corresponding to Y through the autoencoding network module in M″ Y ;

[0122] S820: Z is converted through the subspace basis matrix module in M″ Y Perform initial clustering to obtain K categories, and obtain the corresponding subspace basis matrix set B′ through the K categories;

[0123] S830: Calculate the affinity a of the i′th sample in Y to the j′th subspace basis i′j′ , through a i′j′ Then calculate the normalized distribution probability q of the i′th sample to the j′th subspace basis i′j′ ;

[0124] S840: For the i′th sample y in Y i′ The K normalized distribution probability values ​​are related to each other, and the category corresponding to the subspace basis corresponding to the maximum value is selected as y i′ The final cluster category c i′ , the calculation expression is as follows:

[0125]

[0126] Among them, argmax j′ (·) represents the maximum value among j′;

[0127] S850: Traverse Z Y All embedded representations in , repeat S830-S840 to obtain the clustering results of all data in Y.

[0128] Experimental results and analysis

[0129] The clustering performance of the proposed method is evaluated on five benchmark datasets: a grayscale image dataset (Fashion-MNIST), three challenging color image datasets (CIFAR-10, CIFAR-100, STL-10), and a text dataset (REUTERS-10K).

[0130] The present invention uses three widely used clustering indices to quantify clustering performance, namely, clustering accuracy (ACC), normalized mutual information (NMI), and adjusted Rand index (ARI). The higher the three indices, the better the clustering performance.

[0131] The method model proposed in the present invention is constructed by two encoders and decoders with the same network structure. The encoder network includes an input layer, three hidden layers, and an output layer, and the sizes of the three hidden layers are set to 500, 500, and 1000. The decoder is symmetrical with it. First, the proposed model is initialized by pre-training the above-mentioned autoencoder network. The Adam optimizer is used in this study. The learning rate is set to 0.0005, the training period is set to 400, and the batch size is reasonably adjusted according to the dataset. The number of cluster categories K is the same as the number of categories of the corresponding dataset. For the three color image datasets (CIFAR-10, CIFAR-100, STL-10), this paper extracts their features through Resnet50. For the setting of hyperparameters, this paper sets γ and d to the same value, d should be set near the number of cluster categories K, and the class prior probability θ + It can be estimated from the data. λ1 and λ2 are set to 0.001 and 0.5 respectively. For each dataset, λ1 and λ2 are fine-tuned accordingly and set to their optimal values.

[0132] 1. Comparison with the most advanced methods

[0133] The effectiveness of the proposed method was evaluated on five benchmark datasets and compared with 12 representative state-of-the-art clustering methods, including the classic clustering method kmeans, deep clustering methods (DEC, IDEC, JULE, DAC, DCCM, DSEC, PICA, EDESC), and comparative clustering methods (CC, CCR, DMICC). The results shown in Table 1 show that the proposed method performs at the forefront on all five benchmark datasets, significantly outperforming these state-of-the-art methods on four datasets. In particular, on STL-10, the proposed method achieves the highest performance in all three metrics, exceeding DMICC by 5.8%, 15.0%, and 15.2%, respectively, and the benchmark method EDESC by 11.3%, 15.2%, and 25.8%, respectively. The experimental results demonstrate the effectiveness and superiority of the proposed method. Furthermore, on the CIFAR-10 dataset, this method achieves lower NMI and ARI than DMICC. This may be because DMICC directly optimizes mutual information, while our method, like most deep clustering methods, is based on the Euclidean distance metric, resulting in a lower performance than DMICC on the CIFAR-10 dataset. Considering different metrics (such as the angular relationship between data) may help reveal a better clustering structure. The comparison is shown in Table 1.

[0134] Table 1 Comparison of clustering performance on five benchmark datasets

[0135]

[0136] 2. Qualitative Research

[0137] Visualization of embedded representation: To demonstrate the superiority of the proposed method, the present invention conducted a t-SNE visualization experiment on the STL-10 dataset to compare the results with those of EDESC and DMICC. Figure 1-Figure 3 As shown, it can be observed that the competitor method does not reflect the data structure well, and the classes are not completely separated. However, the method of the present invention is more able to correctly cluster the data, and the individual classes are more separated, resulting in better clustering effect. To observe the evolution of the cluster assignment of the proposed model, the present invention also performed t-SNE on the learned embedding representation on the STL-10 dataset in four epochs. In the initial stage of training, a small number of data points were correctly clustered together. As the training process progressed, the majority of data points were reasonably distributed.

[0138] 3. Ablation Experiment

[0139] We conducted three ablation studies to further demonstrate the importance of the proposed debiased contrastive loss and the superiority of the feature-level contrastive learning strategy used. In this experiment, we used the EDESC method as the "Baseline". Based on EDESC, we used hard negative sampling to filter negative samples in the embedding space and constructed a new model called "Baseline+HCL". This comparison model was trained by jointly optimizing the EDESC total loss and the hard negative sampling contrastive loss. We then conducted experiments on the CIFAR-10, CIFAR-100, and STL-10 datasets to compare the two models with our method. As shown in Table 2, our method outperformed both "Baseline" and "Baseline+HCL" in terms of clustering performance. Therefore, the dual debiased contrastive loss proposed in this paper is crucial for learning more effective embedded representations, significantly improving the clustering performance of the model. On the other hand, previous works often applied the cluster-level contrastive learning strategy (CCL) to improve clustering performance. Therefore, we combined CCL and EDESC to establish another comparison model, called "Baseline+CCL". However, research has shown that the quality of the feature representation of the learned data is crucial for achieving a good clustering assignment. Therefore, the proposed model employs a feature-level contrastive learning strategy (FCL). As shown in Table 2, FCL is more effective than CCL in learning the most effective embedding representation from the data, enabling the model to achieve a more reasonable cluster assignment.

[0140] Table 2: Ablation experiments on three datasets comparing the proposed method with other comparative learning strategies.

[0141]

[0142] To optimize the values ​​of the parameters λ2 and λ3, we conducted ablation experiments on the STL-10 dataset using different values ​​of λ2 and λ3. In this experiment, the range of λ2 was set to {0.001, 0.01, 0.1}, and the range of λ3 was set to {0.1, 0.5, 1}. The experimental results are shown in Table 3. As can be seen from the table, the best clustering performance is achieved when λ2 is 0.001 and λ3 is 0.5.

[0143] Table 3 Experimental results of different λ2 and λ3 on the STL-10 dataset

[0144]

[0145]

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A deep embedding subspace clustering method based on double debiased contrastive learning, characterized by: The steps include: S100: Select a public image dataset as a sample set, where each image in the sample set is a sample, and the sample set contains n samples; construct a clustering model M, where M includes an autoencoder network module and a subspace basis matrix module. The autoencoder network module includes two encoders and two decoders; S200: Initialize the autoencoder network module and select the i-th sample x from the sample set i , and then use two independent data augmentation methods to i Processing to obtain enhanced data and The expression is as follows: Among them, T a and T b Represents two different data enhancement methods, and Represents the enhanced samples obtained by the corresponding data enhancement method; S300: and As input to the autoencoder network module: and Each is encoded by an encoder to obtain The corresponding latent feature embedding and The corresponding latent feature embedding Then and Then pass through two identical decoders to get The corresponding restored image and The corresponding restored image in accordance with and Use mean square error MSE to construct the loss function L recon , the expression is as follows: Where n represents the total number of samples in the sample set; The enhanced data and restored images of all samples are used as training data, and L recon Pre-train the autoencoder network module. When the number of pre-training times reaches the maximum or L recon When the minimum value is reached, the pre-training is stopped and the pre-trained autoencoder network module is obtained; The pre-trained autoencoder network module and subspace basis matrix module are taken as M', and then a joint optimization objective function L is constructed to train M': S400: and Perform linear combination calculation to get the embedded representation z i ; Traverse all samples in the sample set and obtain the embedded representation set Z of all samples; The K-means clustering method is used to cluster the embedded representation in Z to obtain K categories, and the corresponding initialization subspace basis matrix set B is obtained through the K categories, B=[B (1) ,B (2) ,...,B (K) ], represents the j-th subspace basis matrix and j∈[1,K], b represents the number of basis vectors of all subspaces, and p represents the dimension of each basis vector; Use B to define the subspace constraint loss function L sub-1 , L sub-1 The calculation expression is as follows: Among them, B T represents the transposed matrix of B, ⊙ represents element-by-element multiplication, I represents a diagonal matrix of size k×k, with diagonal elements set to 1 and the remaining elements set to 0; F represents the formula representation, and O represents the parameter for eliminating the self-influence of corner blocks; S500: Calculate the affinity a of the i-th sample to the j-th subspace basis ij , through a ij Then calculate the normalized distribution probability q of the i-th sample to the j-th subspace basis ij , the calculation expression is as follows: Among them, γ is the adjustment term to avoid the denominator being 0, and b is the number of subspace basis vectors; Using q ij Construct KL divergence loss function L sub-2 , Among them, Q represents the subspace allocation matrix, P represents the redefined allocation probability, and p ij represents the refined subspace affinity vector; S600: Constructing a double debiasing contrast loss function L dcl Used to ensure the stability of autoencoding network learning, the formula is as follows: Among them, p pos represents the positive sample set, p neg represents the negative sample set, F d-neg Debiasing negative sample loss, F d-pos Debiasing positive sample loss, For all possible sample pairs (z,z + ) performs expected calculation, z represents the anchor point sample, z - Represents the distribution of negative samples p neg The negative samples obtained by sampling, z + is from the positive sample distribution p pos The positive samples are obtained by sampling, s(·,·) represents the cosine similarity function, v represents the false negative samples, and W represents the weighting parameter; S700: Joint L recon , L dcl , L sub-1 and L sub-2 As the loss function L of M', the calculation expression is as follows: L=L recon +λ1L dcl +λ2L sub-1 +λ3L sub-2 Among them, λ1, λ2, λ3 are hyperparameters that control the balance between different loss terms; The sample set is used as the training set, and the Adam optimizer is used to train M′ using L. The training is stopped when L reaches the minimum value or the training reaches the maximum number of iterations, and the trained clustering model M″ is obtained; S800: Select a data set Y to be clustered, input Y into M″, and obtain the clustering results of all data in Y.

2. The deep embedding subspace clustering method based on double debiased contrastive learning according to claim 1, characterized in that: The data enhancement method in S200 is random cropping and color perturbation.

3. The deep embedding subspace clustering method based on double debiased contrastive learning according to claim 2, characterized in that: The S300 is obtained The corresponding latent feature embedding and The corresponding latent feature embedding The calculation expression is as follows: in, and Represents the corresponding two encoder networks, and Denotes the corresponding encoder weights.

4. The deep embedding subspace clustering method based on double debiased contrastive learning according to claim 3, characterized in that: The S300 is obtained The corresponding restored image and The corresponding restored image The calculation expression is as follows: in, and represents two decoder networks, and Denotes the corresponding decoder weight.

5. The deep embedding subspace clustering method based on double debiased contrastive learning according to claim 4, characterized in that: The S400 utilizes and The linear combination is calculated to get z i The calculation expression is as follows: Among them, z i represents the embedding representation of the i-th sample.

6. The deep embedding subspace clustering method based on double debiased contrastive learning according to claim 5, characterized in that: The double debiasing contrast loss function L is constructed in S600 dcl Also included: In the obtained and In is regarded as an anchor sample z, and its positive sample set is defined as: The negative sample set is defined as: Then the joint probability distribution of the data is defined as: p (z,k) (z,k)=p(z|k)ρ(k), where ρ(k)=θ + represents the prior probability of the kth class; Select negative samples z that are similar to the anchor sample z - , define negative sample z - The distribution of is: p neg (z - )∝p(z - )exp(βs(z,z - )) Among them, β≥0 represents the concentration parameter, which is used to control the similarity between negative samples and anchor samples, s(z,z - ) represents the cosine similarity between the anchor sample and the negative sample; The negative sample distribution p neg (z - ) decomposes into: in, Indicates the true negative sample distribution, that is, the negative samples that truly belong to other categories. represents the distribution of false negative samples, i.e., samples that actually belong to the anchor class but are misclassified as negative samples, θ + represents the prior probability of the class, θ - Indicates the division of θ + The prior probability of other classes.

Images