A deep self-paced active image clustering method
By introducing active learning and self-stepping learning into deep image clustering, representative and high-confidence samples are selected and labeled, solving the problem of high resource consumption in existing image clustering techniques and achieving efficient image clustering results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI UNIV
- Filing Date
- 2023-11-07
- Publication Date
- 2026-05-12
AI Technical Summary
Existing image clustering methods are not effective for high-dimensional image data, large-scale image data, and image data with noise and outliers. Semi-supervised image clustering algorithms require a large amount of supervised information and ignore the selection of supervised information, making them difficult to implement.
Active learning and self-stepping learning are introduced. By selecting representative and high-confidence samples in deep image clustering, assigning real and pseudo labels, the amount of manual annotation is reduced and the number of labeled samples is increased. The neural network is trained using cross-entropy and center loss.
It improves clustering accuracy and reduces resource consumption with minimal manual annotation, achieving efficient image clustering results.
Smart Images
Figure CN117475185B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image clustering, and more particularly to a deep self-synchronizing active image clustering method. Background Technology
[0002] Image clustering is a fundamental problem in computer vision, a method for clustering images based on the similarity and correlation of their content. This technique extracts and represents image features, grouping similar images into the same category and dissimilar images into different categories. Image clustering algorithms can be based on various methods and techniques, such as traditional clustering algorithms (e.g., K-means), spectral clustering, self-organizing map networks, and deep learning. Traditional clustering methods typically utilize features like color, texture, and shape; spectral clustering methods perform spectral decomposition of the image similarity matrix to obtain the spectral distribution, thus enabling clustering; and self-organizing map network methods construct a two-dimensional self-organizing map, encoding neighboring nodes of similar images to achieve clustering. However, traditional clustering algorithms perform poorly on high-dimensional image data, large-scale image data, and image data with noise and outliers. In recent years, deep learning has been extensively researched. Deep neural networks can better extract features from image data and achieve promising performance in many tasks, thus attracting increasing attention to deep image clustering.
[0003] However, due to the lack of data labels, clustering often becomes an ill-posed problem. Even deep clustering frequently results in incorrect clustering in practical applications. To address this issue, semi-supervised deep clustering methods have been proposed. These methods apply supervisory information, such as partial data labels or pairwise constraints between data pairs, to guide the clustering. Because semi-supervised clustering uses supervisory information, it can sometimes achieve more reliable clustering results. However, it should be noted that in these semi-supervised clustering methods, the supervisory information is provided in advance. We know that the performance of semi-supervised learning largely depends on the quality of the supervisory information; low-quality supervisory information may be of no help to clustering. This means that general semi-supervised clustering methods need to obtain a large amount of supervised information in advance to ensure advanced clustering results. Unfortunately, all these semi-supervised methods only focus on how to use supervisory information while ignoring the selection of supervisory information. Therefore, there is an urgent need for a method to select appropriate supervised information for image clustering.
[0004] Based on the above analysis, existing clustering methods in the field of image clustering, particularly traditional clustering algorithms, perform poorly on high-dimensional image data, large-scale image data, and image data with noise and outliers. Existing semi-supervised image clustering algorithms require a large amount of supervised information to ensure efficient clustering results. These algorithms focus only on how to use supervised information while neglecting its selection. Furthermore, the implementation of these algorithms requires a significant amount of supervised information, which, in the real world, expends considerable resources to obtain a large amount of supervised information from unlabeled image datasets. Clearly, such semi-supervised image clustering algorithms are difficult to implement practically. Therefore, if the amount of labeling required for supervised clustering algorithms can be reduced, only limited and affordable resources will be needed for implementation, thus significantly saving manpower and time costs. Therefore, this invention proposes a deep self-stepping active image clustering method to address the problems existing in the prior art. Summary of the Invention
[0005] To address the aforementioned problems, the present invention aims to propose a deep self-stepping active image clustering method. This method introduces active learning and self-stepping learning on the basis of deep image clustering, which helps to identify samples in the dataset that are both representative and challenging, as well as samples with high confidence. These samples are then assigned real and pseudo labels, respectively. This increases the number of labeled samples with minimal manual annotation, enabling the algorithm to be implemented with limited resources.
[0006] To achieve the objectives of this invention, the invention is implemented through the following technical solution: a deep self-synchronizing active image clustering method, comprising the following steps:
[0007] Step 1: Obtain a labeled image dataset and combine the training set and the test set into a single training set;
[0008] Step 2: Pre-train the ConvNeXt neural network using contrastive loss;
[0009] Step 3: Use a pre-trained neural network to extract features from each image in the image dataset, and then use the K-means method to cluster the extracted features to obtain the clustering results.
[0010] Step 4: Based on the clustering results, obtain the auto-weight of each sample. Manually label the image samples that are both representative and have high auto-weights. Assign corresponding pseudo-labels to the high-auto-weight samples in the remaining image data. Separate the new labeled image data from the unlabeled image dataset and merge it with the labeled image dataset to obtain a new labeled image dataset and an unlabeled image dataset.
[0011] Step 5: Train the neural network using cross-entropy loss and center loss for labeled image datasets, and train the neural network using only center loss for unlabeled image datasets.
[0012] Repeat steps three through five until the budget is exhausted;
[0013] Step 6: Use a neural network to obtain all data features, use a softmax layer to obtain the final clustering results, and evaluate the model using the clustering result evaluation metrics.
[0014] A further improvement is made in step two, where the comparison loss is: ;
[0015] in This represents the loss between the i-th sample and the j-th sample. Represents the features of the i-th sample. It is an indicator function. The cosine loss is expressed as , For temperature parameters, This represents the number of samples. Each sample is augmented with data to create an additional sample, so each sample will have one similar sample and... The data augmentation methods include random cropping, color distortion, and Gaussian blur. Each sample will be randomly selected from the three data augmentation methods for data augmentation.
[0016] Further improvements are made in step three, where the pre-trained neural network from step two is used to extract features from all data in the training set, and the K-means method is used to cluster the obtained features to obtain the clustering results.
[0017] A further improvement lies in step four, where, based on the clustering results obtained in step three, the self-step weight, i.e., the confidence level, is calculated for each sample. The calculation is as follows: ;
[0018] in, The hyperparameters represent self-synchronous learning. Representing data After neural network Extracted features and data The center of the cluster The L2 norm between Represents the parameters of the neural network.
[0019] data Compactness of representative use data The formula for its calculation is as follows: ;
[0020] in, Indicates distance from sample Recent A set of samples.
[0021] The further improvement lies in: in step five, the cross-entropy loss... for: ;
[0022] in, This indicates a supervised dataset. The total number of samples in the middle, It is a fully connected network that maps data from 200 dimensions to the number of categories in the dataset. Representing data The tag, It is the standard cross-entropy function, expressed as: ;
[0023] in This represents the unique hot tag corresponding to the real tag. Represents the neural network prediction vector. Indicates the unique hot tag dimension. and Representing the one-hot label and the prediction vector respectively Dimension value,
[0024] The central loss described in step five for: ;
[0025] in Indicates sample In labeled image datasets The center of the cluster to which it belongs,
[0026] In labeled image datasets Total loss for: ;
[0027] In unlabeled image datasets Total loss for: .
[0028] The beneficial effects of this invention are as follows: By introducing active learning and self-stepping learning on the basis of deep image clustering, this invention helps to find samples in the dataset that are both representative and challenging, as well as samples with high confidence, and assigns them real labels and pseudo labels respectively. This increases the number of labeled samples with very little manual annotation, enabling the algorithm to be implemented with limited resources. Attached Figure Description
[0029] Figure 1 This is a schematic diagram of the DSAC model framework of the present invention;
[0030] Figure 2 This is a flowchart of the steps of the present invention. Detailed Implementation
[0031] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.
[0032] according to Figure 1 , Figure 2 As shown, this embodiment provides a deep self-synchronizing active image clustering method, which involves five rounds of annotation, with 20 samples labeled in each round, including the following steps:
[0033] Step 1: Obtain a labeled image dataset. Combine the training set and the test set into a single training set. The training dataset contains 8000 images, and the test dataset contains 5000 images. Each image in the dataset is 96 pixels × 96 pixels in size.
[0034] Step 2: Pre-train the ConvNeXt neural network using contrastive loss.
[0035] The contrast loss used is: ;
[0036] in This represents the loss between the i-th sample and the j-th sample. Represents the features of the i-th sample. It is an indicator function. The cosine loss is expressed as , For temperature parameters, This represents the number of samples. Each sample is augmented with data to create an additional sample, so each sample will have one similar sample and... The data augmentation methods include random cropping, color distortion, and Gaussian blur. Each sample will be augmented using one of the three data augmentation methods.
[0037] The neural network used is ConvNeXt, with an output dimension of 200. The parameter settings are as follows: batch size 256, number of training epochs 20, and temperature parameter... The value is 0.5, the optimizer uses AdamW, and the learning rate is 1×10⁻⁵. , =0.999, the model was trained on the PyTorch framework and the NVIDIA RTX 3090 platform.
[0038] Step 3: Use the pre-trained neural network from Step 2 to extract features from each image sample in the image dataset. Then, use the K-means method to cluster the extracted features to obtain the clustering results.
[0039] Step 4: Obtain the autostep weight of each sample based on the clustering results. Manually label the image samples that are both representative and have high autostep weights. Assign corresponding pseudo-labels to the high autostep weight samples in the remaining image data. Separate the new labeled image data from the unlabeled image dataset and merge it with the labeled image dataset to obtain a new labeled image dataset and an unlabeled image dataset.
[0040] Calculate the self-stepping weights, or confidence scores, for each sample. The calculation is as follows: ;
[0041] in, The hyperparameters representing self-synchronous learning are set to the largest among all clusters. , Representing data After neural network Extracted features and data The center of the cluster The L2 norm between This represents the parameters of the neural network.
[0042] data Representativeness can be achieved using the compactness of the data. The formula for its calculation is as follows: ;
[0043] in, Indicates distance from sample Recent A sample set, Set to 6;
[0044] Remove labeled image data and select from each cluster. The smallest 200 samples are selected, and then one sample with the highest value is chosen from these 200 samples. The samples were manually labeled; the labeled samples were removed, and samples from each cluster were selected. The 200 largest samples are selected, and then one sample with the highest value is chosen from these 200 samples. The samples were manually labeled; the labeled samples were removed, and the top 50 samples from each cluster were selected. The samples are assigned labels to the clusters, thus dividing the dataset into labeled image datasets and unlabeled image datasets.
[0045] Step 5: Train the neural network using cross-entropy loss and center loss for the labeled image dataset, and train the neural network using only center loss for the unlabeled image dataset.
[0046] The aforementioned cross-entropy loss for: ;
[0047] in, This indicates a supervised dataset. The total number of samples in the middle, It is a fully connected network that maps data from 200 dimensions to the number of categories in the dataset. Representing data The tag, It is the standard cross-entropy function, expressed as: ;
[0048] in This represents the unique hot tag corresponding to the real tag. Represents the neural network prediction vector. Indicates the unique hot tag dimension. and Representing the one-hot label and the prediction vector respectively Dimension value,
[0049] The central loss described in step five for: ;
[0050] in Indicates sample In labeled image datasets The center of the cluster to which it belongs,
[0051] In labeled image datasets Total loss for: ;
[0052] In unlabeled image datasets Total loss for: ;
[0053] The parameter settings are as follows: batch size is set to 256, number of training epochs is 100, and the coefficient of the loss function is... The optimizer used is AdamW with a learning rate of 6×10⁻⁶. -6 , , =0.999, the model was trained on the PyTorch framework and the NVIDIA RTX 3090 platform.
[0054] Repeat steps three through five until the budget is used up.
[0055] Step 6: Use a neural network to acquire all data features, pass through a softmax layer, and obtain the final clustering result. Combine the clustering result evaluation indicators to evaluate the model. Clustering accuracy (ACC), normalized mutual information (NMI), and adjusted Land coefficient (ARI) are used as clustering evaluation indicators to evaluate the clustering results from an objective perspective. This method is compared with other active learning methods (this method labels 100 samples, while the other methods label 1000 samples). The experimental results are shown in Table 1.
[0056] Table 1. Comparison of clustering results:
[0057] ;
[0058] It can be seen that the DSAC proposed in this invention is significantly better than other algorithms in all three metrics.
[0059] This deep auto-stepping active image clustering method introduces auto-stepping learning and active learning on the basis of deep image clustering, allowing the algorithm to selectively use supervised information. Simultaneously, based on the weights of each data point obtained through auto-stepping learning, it selects data with high confidence and assigns pseudo-labels to the corresponding clusters. This results in highly accurate labeled image data with relatively little manual annotation. The neural network is trained using cross-entropy and centroid loss on the labeled image dataset, and centroid loss is used to train the neural network on the unlabeled image dataset. Finally, the trained neural network is used to extract data features, which are then passed through a softmax layer to obtain the final clustering result.
[0060] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.
Claims
1. A deep autosynchronous active image clustering method, characterized in that: The deep self-synchronizing active image clustering method includes the following steps: Step 1: Obtain a labeled image dataset and combine the training set and the test set into a single training set; Step 2: Pre-train the ConvNeXt neural network using contrastive loss; Step 3: Use a pre-trained neural network to extract features from each image in the image dataset, and then use the K-means method to cluster the extracted features to obtain the clustering results. Step 4: Obtain the autostep weight of each sample based on the clustering results. Manually label the image samples that are both representative and have high autostep weights. Assign corresponding pseudo-labels to the high autostep weight samples in the remaining image data. Separate the new labeled image data from the unlabeled image dataset and merge it with the labeled image dataset to obtain a new labeled image dataset and an unlabeled image dataset. Based on the clustering results obtained in step three, calculate the self-step weight, i.e., the confidence level, for each sample. The calculation is as follows: ; in, The hyperparameters represent self-synchronous learning. Representing data After neural network Extracted features and data The center of the cluster The L2 norm between Represents the parameters of the neural network. data Compactness of representative use data The formula for its calculation is as follows: ； in, Indicates distance from sample Recent A sample set; Step 5: Train the neural network using cross-entropy loss and center loss for labeled image datasets, and train the neural network using only center loss for unlabeled image datasets. Repeat steps three through five until the budget is exhausted; Step 6: Use a neural network to obtain all data features, pass them through a softmax layer to obtain the final clustering results, and evaluate the model using the clustering result evaluation metrics.
2. The deep self-synchronizing active image clustering method as described in claim 1, characterized in that: In step two, the contrast loss is: ; in This represents the loss between the i-th sample and the j-th sample. Represents the features of the i-th sample. It is an indicator function. The cosine loss is expressed as , For temperature parameters, This represents the number of samples. Each sample is augmented with data to create an additional sample, so each sample will have one similar sample and... The data augmentation methods include random cropping, color distortion, and Gaussian blur. Each sample will be randomly selected from the three data augmentation methods for data augmentation.
3. The deep self-synchronizing active image clustering method as described in claim 1, characterized in that: In step three, the pre-trained neural network from step two is used to extract features from all data in the training set, and the K-means method is used to cluster the obtained features to obtain the clustering results.
4. The deep self-synchronizing active image clustering method as described in claim 1, characterized in that: In step five, the cross-entropy loss... for: ; in, This indicates a supervised dataset. The total number of samples in the middle, It is a fully connected network that maps data from 200 dimensions to the number of categories in the dataset. Representing data The tag, It is the standard cross-entropy function, expressed as: ； in This represents the unique hot tag corresponding to the real tag. Represents the neural network prediction vector. Indicates the unique hot tag dimension. and Representing the one-hot label and the prediction vector respectively Dimension value, The central loss described in step five for: ; in Indicates sample In labeled image datasets The center of the cluster to which it belongs, In labeled image datasets Total loss for: ； In unlabeled image datasets Total loss for: 。