An Adaptive Weight-based Complex Image Clustering Method

Abstract

The present invention relates to the fields of deep learning and image clustering, and specifically to a complex image clustering method based on adaptive weights, which includes the following steps: First, initialize the clustering network using an existing classification network and a traditional clustering algorithm; Second, perform clustering on the images and calculate the network objective for this iteration in the direction of entropy reduction and update the network; Third, calculate the weights of each sample in the next iteration using the sample entropy value; Finally, when the clustering loss is less than the stop iteration threshold, output the clustering result. It solves the problem in the existing image clustering model that it is difficult to use the quality of graphical samples to determine the weight of a sample and use adaptive weights for model training.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and image clustering, and specifically refers to a complex picture clustering method based on adaptive weights. Background Art

[0002] With the popularization of portable media devices, images are generated more and more quickly. Due to their intuitive and rich content display methods, pictures have become one of the most important resources in the big data era; for image-based work including object recognition, product recommendation and other applications, image clustering is the underlying work; for the clustering of traditional and relatively simple images such as license plate numbers, there has been good development, while for more and more complex pictures, existing image clustering algorithms or even deep image clustering algorithms cannot adapt to it well; complex image classification networks have had good development, such as ResNet, GoogleNet, etc., but it is an impossible task to accurately label a large number of pictures, so the clustering of complex unlabeled images is particularly important.

[0003] Compared with simple images, the reasons why unlabeled complex images are difficult to cluster are mainly manifested in two aspects: one is that it is difficult to extract complex image features, for which we can borrow the idea of transfer learning and use existing image feature extraction networks for network initialization; the other is that it is difficult to measure the quality of complex images. Images with clear target features play an important role in network training, while images with high noise and complex content will damage the performance of the network; for this, we will propose an image clustering network for complex images based on the self-paced idea and using adaptive sample weights. Summary of the Invention

[0004] Based on the above problems, the present invention provides a complex picture clustering method based on adaptive weights, which solves the problem in the existing image clustering model that the quality of graphic samples cannot be used to determine the weight of a sample and the model is trained using adaptive weights.

[0005] To solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0006] A complex picture clustering method based on adaptive weights includes the following steps:

[0007] Step 1: Construct an image data set, divide the image data set into a training set and a validation set, and preprocess the images;

[0008] Step 2: Construct a feature extraction network, where the feature extraction network includes a feature extraction part, and two fully connected layers and a clustering layer connected after the feature extraction part;

[0009] Step 3: Input the training set into the constructed feature extraction network for training and output the probability distribution matrix P of each sample belonging to each class;

[0010] Step 4: Calculate the target probability distribution matrix Q in the clustering layer according to the probability distribution matrix P;

[0011] Step 5: Calculate the cross-entropy loss between the probability distribution matrix P and the target probability distribution matrix Q. If the cross-entropy loss is less than the threshold, stop the network training, save the model, and enter Step 6. If the change rate of the exponential loss is greater than the threshold, backpropagate to update the weights of the samples with the entropy of the samples, start a new round of network training, and enter Step 3;

[0012] Step 6: Input the validation set into the model saved in Step 5 to verify the model.

[0013] Furthermore, in Step 1, the image dataset is OFFICE-31, and all images in this image dataset are divided into a training set and a validation set according to a ratio of 9:1.

[0014] Furthermore, in Step 1, the image preprocessing process includes upsampling and downsampling operations on the images. Among them, cubic interpolation is used for upsampling, and sampling is performed row by row and column by column according to the target size and the original size for downsampling. After the images are processed by upsampling and downsampling, the size is unified to 299*299*3.

[0015] Furthermore, in Step 2, the first layer to the penultimate layer of InceptionV3 are used as the feature extraction part of the feature extraction network to extract the feature representation of the images, and the initialization parameters are selected as AlexNet parameters without freezing the parameters.

[0016] Furthermore, the number of neurons in the first fully connected layer is 256, and the output unit of the second fully connected layer is the number of pre-clustering clusters, and its number of neurons is 5.

[0017] Furthermore, the DBSCAN algorithm is used in the clustering layer to cluster the extracted feature representations, and the obtained class centers are used to initialize the clustering layer. Among them, when using the DBSCAN algorithm for clustering, the radius is calculated using the Euclidean norm of two feature vectors, the neighborhood radius is set to 0.17, and the minimum number of points in the neighborhood of the core object is set to 10.

[0018] Furthermore, in Step 4, the calculation formula for the target probability distribution matrix Q is:

[0019]

[0020] where q ij represents the probability that sample i belongs to class j, and p ij is qij The target distribution, which is used to calculate the updated values of the neural network parameters. There are n samples and m classes in total. l represents the step size for optimizing the distribution towards low entropy, and l = 3.

[0021] Furthermore, in step five, the threshold is set to 0.001.

[0022] Furthermore, in step five, the method for updating the sample weights is as follows:

[0023] In the first iteration, the weight of each sample is set to 1. In subsequent iterations, the update formula for the sample weights is:

[0024]

[0025] where w i is the weight of the i-th sample, and H(p i ) is the entropy of the i-th sample.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows: By optimizing the sample label probability distribution and updating the sample weights according to the sample label distribution to train the network, images with higher label credibility have higher weights during the network training process, while ignoring the influence of low-credibility images on the network. Subsequently, by judging the change of the cross-loss with respect to the clustering labels, the network training is stopped when the change is less than a given threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is the flowchart of Embodiment 1. DETAILED DESCRIPTION OF THE INVENTION

[0028] The present invention will be further described below with reference to the accompanying drawings. The embodiments of the present invention include but are not limited to the following examples.

[0029] As Figure 1 shown, a complex picture clustering method based on adaptive weights includes the following steps:

[0030] Step 1: Construct an image data set, divide the image data set into a training set and a validation set, and preprocess the images.

[0031] In this step, the image data set is divided into a training set and a validation set according to 90% for the training set and 10% for the validation set. At the same time, in this embodiment, the image data set is OFFICE-31. There are 5 classes in total in this image data set, each class contains 100 samples, and the sample sizes are not uniform.

[0032] In this step, image preprocessing includes upsampling and downsampling operations on the image. Among them, upsampling uses the cubic interpolation method, and downsampling samples every other row and column according to the target size and the original size. After the image is processed by upsampling and downsampling, the size is unified to 299*299*3 to adapt to the input standard of the InceptionV3 network.

[0033] Step 2: Construct a feature extraction network.

[0034] In this step, the feature extraction network includes a feature extraction part, and two fully connected layers and one clustering layer connected after the feature extraction part.

[0035] In this step, the first layer to the penultimate layer of InceptionV3 are used as the feature extraction part of the feature extraction network to extract the feature representation of the image. The initial parameters are selected as AlexNet parameters and the parameters are not frozen.

[0036] In this step, the number of neurons in the first fully connected layer is 256, and the output unit of the second fully connected layer is the number of pre-clustering clusters, and the number of its neurons is 5.

[0037] In this step, the clustering layer uses the DBSCAN algorithm to cluster the extracted feature representation, and initializes the clustering layer with the obtained class centers. Among them, when using the DBSCAN algorithm for clustering, the radius is calculated using the Euclidean norm of two feature vectors, the neighborhood radius is set to 0.17, and the minimum number of points in the neighborhood of the core object is set to 10.

[0038] Step 3: Input the training set into the constructed feature extraction network for training and output the probability distribution matrix P of each sample belonging to each class.

[0039] Step 4: Calculate the target probability distribution matrix Q according to the probability distribution matrix P.

[0040] In this step, the calculation formula of the target probability distribution matrix Q is:

[0041]

[0042] where q ij represents the probability that sample i belongs to class j, and p ij is the target distribution of q ij , which is used to calculate the updated value of the neural network parameters. There are n samples and m classes in total, and l represents the step size for optimizing the distribution towards low entropy and l = 3.

[0043] Step 5: Calculate the cross-entropy loss between the probability distribution matrix P and the target probability distribution matrix Q. If the cross-entropy loss is less than the threshold, stop the network training, save the model, and proceed to Step 6. If the change rate of the exponential loss is greater than the threshold, perform backpropagation to update the weights of the samples with the entropy of the samples, start a new round of network training, and proceed to Step 3.

[0044] In this step, first calculate the cross-entropy loss loss between the probability distribution matrix P and the target probability distribution matrix Q, and return the value of the loss function. Optimize the parameters of the model according to the backpropagation rule. At the same time, calculate the change rate of the loss in this training and the loss in the previous round of training, so as to obtain the change rate of the cross-entropy loss between the probability distribution matrix P and the target probability distribution matrix Q. The formula is:

[0045]

[0046] where loss pre represents the cross-entropy loss in the previous round of training.

[0047] In this step, the threshold is set to 0.01, that is, if the change rate of the cross-entropy loss is less than 0.01, stop the network training, save the model, and proceed to Step 6. If the change rate of the exponential loss is greater than 0.01, perform backpropagation to update the weights of the samples with the entropy of the samples, start a new round of network training, and proceed to Step 3.

[0048] In this step, the method for updating the sample weights is as follows:

[0049] In the first iteration, set the weight of each sample to 1. In subsequent iterations, the update formula for the sample weights is:

[0050]

[0051] where w i is the weight of the i-th sample, and H(p i ) is the entropy of the i-th sample. It restarts a new round of training the network by optimizing the sample label probability distribution and updating the sample weights according to the sample label distribution, so that images with higher label credibility have higher weights during the network training process, while ignoring the influence of low-credibility images on the network.

[0052] Step 6: Input the validation set into the model saved in Step 5 to verify the model.

[0053] The above are the embodiments of the present invention. The above embodiments and the specific parameters in the embodiments are only for clearly expressing the inventor's invention verification process, and are not used to limit the patent protection scope of the present invention. The patent protection scope of the present invention still takes its claims as the criterion. Any equivalent structural changes made by using the content of the specification and drawings of the present invention should also be included in the protection scope of the present invention by the same token.

Claims

1. A complex image clustering method based on adaptive weights, characterized in that It includes the following steps: Step 1: Construct an image dataset, divide the image dataset into a training set and a validation set, and preprocess the images; Step 2: Construct a feature extraction network, which includes a feature extraction part, and two fully connected layers and one clustering layer connected after the feature extraction part; Step 3: Input the training set into the constructed feature extraction network for training and output the probability distribution matrix of each sample belonging to each class ; Step 4. According to the probability distribution matrix calculate the target probability distribution matrix at the clustering layer ; Step 5: Calculate the probability distribution matrix and the cross-entropy loss of the target probability distribution matrix If the cross-entropy loss is less than the threshold, stop the network training, save the model, and proceed to Step 6. If the change rate of the exponential loss is greater than the threshold, update the weights of the samples with the entropy of the samples through backpropagation, start a new round of network training, and proceed to Step 3; Step 6: Input the validation set into the model saved in Step 5 to verify the model; In the fourth step, the target probability distribution matrix is calculated as follows: ， Among them, represents the probability that a sample belongs to a category is the target distribution for calculating the updated value of the neural network parameters. There are samples and categories represents the step size for optimizing the distribution towards low entropy and = 3.

2. The complex picture clustering method based on adaptive weights according to claim 1, wherein In Step 1, the image dataset is OFFICE-31, and all images in this image dataset are divided into a training set and a validation set according to a ratio of 9:

1.

3. A complex picture clustering method based on adaptive weights according to claim 2, characterized in that In Step 1, the image preprocessing process includes upsampling and downsampling operations on the images. Among them, cubic interpolation is used for upsampling, and sampling is performed row by row and column by column according to the target size and the original size for downsampling. After the images are processed by upsampling and downsampling, the size is unified to 299*299*3.

4. A complex picture clustering method based on adaptive weights according to claim 1, characterized in that In Step 2, the first layer to the penultimate layer of InceptionV3 are used as the feature extraction part of the feature extraction network to extract the feature representation of the images. The initial parameters are selected as AlexNet parameters, and the parameters are not frozen.

5. A complex picture clustering method based on adaptive weights according to claim 4, characterized in that, The number of neurons in the first fully connected layer of the feature extraction network is 256, and the output unit of the second fully connected layer of the feature extraction network is the number of pre-clustering clusters, and the number of its neurons is 5.

6. A method for clustering complex pictures based on adaptive weights according to claim 4, characterized in that The clustering layer uses the DBSCAN algorithm to cluster the extracted feature representations, and initializes the clustering layer with the obtained cluster centers. Among them, when using the DBSCAN algorithm for clustering, the radius is calculated using the Euclidean norm of two feature vectors, the neighborhood radius is set to 0.17, and the minimum number of points in the neighborhood of the core object is set to 10.

7. A complex picture clustering method based on adaptive weights according to claim 1, characterized in that In Step 5, the threshold is set to 0.

001.

8. A method for clustering complex pictures based on adaptive weights according to claim 1, characterized in that In Step 5, the method for updating the sample weights is as follows: The weight of each sample is set to 1 in the first iteration. In subsequent iterations, the update formula for the sample weights is: ， Among them, is the weight of the th sample, is the entropy of the th sample.

Images