Clustering apparatus, method, and program

Abstract

To provide a clustering apparatus, method and program for improving clustering performance or visibility of clustering results.SOLUTION: A clustering apparatus includes a data acquisition unit, a first feature calculation unit, a second feature calculation unit, a second cluster division unit, and an integration unit. The data acquisition unit acquires target data, a first trained model configured to receive an input of the target data and output a first feature vector, and a second trained model configured to receive an input of the target data and output a second feature vector. The first feature calculation unit calculates the first feature vector using the first trained model and the target data. The second feature calculation unit calculates the second feature vector using the second trained model and target data. The second cluster division unit divides the second feature vectors to calculate second clusters. The integration unit integrates the first feature vectors with the second clusters.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] SUMMARY OF THE PRESENT EMBODIMENT An embodiment of the present invention relates to a clustering device, a clustering method, and a program. [Background technology]

[0002] A clustering device is a device that classifies target data, which is a group of vector data, into classes (also called cluster division or clustering) using a learning model. For example, the clustering device compresses the dimensionality of the group of vector data using a method such as t-SNE, estimates the number of clusters, and classifies the group of dimensionally compressed vector data using a learning model. The learning model identifies cluster labels (cluster IDs) using, for example, an unsupervised learning method. The cluster labels (cluster IDs) identified by the learning model are used, for example, as supervised data in supervised learning of another learning model.

[0003] In general, data groups to be clustered may contain data that is harmful to clustering (hereinafter, referred to as harmful data). Harmful data for clustering may be, for example, indistinguishable data or noisy data that has been mixed in by mistake. Indistinguishable data may be data that 90% or more of humans cannot correctly distinguish. For example, when classifying a group of photos that show animals into two clusters, "dogs" and "cats," the group of photos may contain, as indistinguishable data, images in which indistinguishable parts are enlarged or images that show both dogs and cats. In addition, as noisy data, images of "horses" or "vehicles" that have been mixed in by mistake may be included in the group of photos of the target data.

[0004] These harmful data can prevent correct clustering and can be a factor in reducing clustering performance. In addition, these harmful data can be a factor in reducing the visibility of clustering results by making the boundaries of clusters unclear when displaying feature vectors output from a learning model or by forming spurious clusters that are not essential. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6846369 Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to provide a clustering device, method, and program that can improve the visibility of clustering results or clustering performance. [Means for solving the problem]

[0007] In order to solve such problems, a clustering device according to an embodiment includes a data acquisition unit, a first feature calculation unit, a second feature calculation unit, a second cluster division unit, and an integration unit. The data acquisition unit acquires target data, a first learning model that receives an input of the target data and outputs a first feature vector, and a second learning model that receives an input of the target data and outputs a second feature vector. The first feature calculation unit calculates the first feature vector using the first learning model and the target data. The second feature calculation unit calculates the second feature vector using the second learning model and the target data. The second cluster division unit divides the second feature vector to calculate a second cluster. The integration unit integrates the first feature vector and the second cluster. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a clustering device according to a first embodiment; [Diagram 2] 11 is a flowchart illustrating a processing procedure of a display process performed by the clustering device according to the first embodiment; [Diagram 3] Scatter plot of the first feature vector according to the first cluster [Figure 4] Scatter plot of the second feature vector according to the second cluster [Diagram 5] Scatter plot of the first feature vector according to the second cluster [Figure 6] FIG. 13 is a diagram showing an example of the configuration of a clustering device according to a second embodiment; [Figure 7] 11 is a flowchart illustrating a processing procedure of a display process performed by a clustering device according to a second embodiment; [Figure 8] FIG. 1 is a diagram illustrating an example of a configuration of a clustering device according to an application example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of a clustering device, method, and program will be described in detail with reference to the drawings. In the following description, components having substantially the same functions and configurations are denoted by the same reference numerals, and repeated description will be given only when necessary.

[0010] (First embodiment) 1 is a diagram showing a configuration of a clustering device 100 according to a first embodiment. The clustering device 100 is a device that displays an output from a first learning model in order to check the clustering performance of class classification (also called cluster division or clustering) using the first learning model.

[0011] Clustering device 100 includes a data acquisition unit 101, a first feature calculation unit 102, a second feature calculation unit 103, a second cluster division unit 104, an integration unit 105, and a display unit .

[0012] The data acquiring unit 101 acquires target data, a first learning model, and a second learning model.

[0013] The target data is data that is the target of clustering. The target data is, for example, image data. For example, when dividing (clustering) a group of images that include animals into groups according to the type of animal, the target data is the images that include the animals. The target data is not limited to image data, and may be data in other formats, as long as it is data that is the object of clustering.

[0014] In this embodiment, the purpose is to improve the clustering performance of the first learning model and the visibility of the clustering result in order to support clustering taking harmful data into consideration. The second learning model is a machine learning model used as an aid to improve the performance or visibility of the first learning model. As the first learning model and the second learning model, for example, a general machine learning model used for clustering a data group, such as a DNN (Deep Neural Network: deep learning model), can be used.

[0015] The first learning model accepts input of target data and outputs a first feature vector. That is, the first learning model is a machine learning model that converts the target data into a first feature vector. The second learning model accepts input of target data and outputs a second feature vector. That is, the second learning model is a machine learning model that converts the target data into a second feature vector. The second feature vector output from the second learning model is used auxiliary to improve the clustering performance of the first feature vector and the visibility of the clustering result.

[0016] The second learning model is a machine learning model different from the first learning model. Therefore, even if the same target data is input to each of the first learning model and the second learning model, the output results of the first learning model and the second learning model may be different. For example, when the same target data is input to each of the first learning model and the second learning model, the first feature vector output from the first learning model and the second feature vector output from the second learning model may be different vectors. As the second learning model, it is preferable to use a model that has a small clustering performance difference from the first learning model and has a different output tendency from the first learning model. A model with a different output tendency is, for example, a model that has a different way of capturing data. For example, the second learning model is a machine learning model that is different from the first learning model in at least one of the model structure, learning method, hyperparameters at the time of learning, and dataset at the time of learning.

[0017] The model structure is, for example, structural parameters such as the total number of DNNs and the number of channels, and the type of model structure such as ResNet / MobileNet / EfficientNet. The learning method is, for example, ID, IDFD, simCLR, MOCO, BYOL, BallowTwins, tSNE, UMAP, AE, VAE, etc. The hyperparameters during learning are, for example, an optimizer, a learning rate, a learning rate schedule, the number of updates, a batch size, a loss function, a random number seed value, etc. The loss function is, for example, a regularization strength (FD, WeightDecay, etc.), a temperature parameter, a momentum coefficient, etc. The dataset during learning is, for example, CIFAR-10, ImageNet, etc. The model structure may be called a learning method.

[0018] In this embodiment, an example will be described in which an unsupervised learning model (DNN) using IDFD is used as the first learning model, and a DNN learned by changing the random number seed value of the first learning model is used as the second learning model. However, the first learning model and the second learning model may be different in at least one of the above-mentioned elements.

[0019] The first feature calculation unit 102 calculates a first feature vector using the first learning model and target data. At this time, the first feature calculation unit 102 inputs the target data to the first learning model and obtains the first feature vector output from the first learning model.

[0020] The second feature calculation unit 103 calculates a second feature vector using the second learning model and the target data. At this time, the second feature calculation unit 103 inputs the target data to the second learning model and obtains the second feature vector output from the second learning model.

[0021] The second cluster division unit 104 performs clustering by dividing the second feature vector into clusters and calculating second clusters. A general clustering method such as k-means can be used for the clustering. Various methods such as the center of gravity method and kernel density estimation can be applied to the clustering.

[0022] The number of clusters divided by clustering (hereinafter referred to as the number of clusters) may be, for example, a numerical value designated by the user of the clustering device 100. When referring to a scatter plot of the second feature vector in a space dimensionally compressed using a method such as PCA, tSNE, or UMAP, or when there is a number of clusters that can be estimated in advance based on the characteristics of the data, that numerical value may be used as the number of clusters.

[0023] The integration unit 105 integrates the first feature vector and the second cluster. In this embodiment, the integration unit 105 integrates the first feature vector and the second cluster by associating the first feature vector and the second cluster obtained from the same target data.

[0024] The display unit 106 visualizes and displays the output of the first learning model in order to confirm the performance of the first learning model. The display unit 106 displays a scatter diagram in which the first feature vector output from the first learning model is mapped. At this time, the display unit 106 displays the first feature vector output from the first learning model based on the integration result by the integration unit 105. Furthermore, the display unit 106 displays the first feature vector based on the index of the corresponding second cluster in the scatter diagram in which the first feature vector is mapped. For example, the display unit 106 changes the color, size, shape, transparency, etc. of the first feature vector according to the second cluster.

[0025] Next, the operation of the display process executed by the clustering device 100 of this embodiment will be described. Fig. 2 is a flowchart showing an example of the procedure of the display process. The display process is a process of displaying a scatter diagram in which the first feature vector output from the first learning model is mapped in order to confirm the performance of the first learning model. Here, as an example, a case will be described in which images in which any of a plurality of types of animals is shown are used as target data, and each image is clustered by animal type without using teacher label information.

[0026] Note that the process steps in each process described below are merely examples, and each process can be modified as appropriate as possible. In addition, steps in the process steps described below can be omitted, replaced, or added as appropriate depending on the embodiment.

[0027] (Display processing) (Step S1-1) First, the data acquiring unit 101 acquires a target data group including a plurality of target data, a first learning model, and a second learning model.

[0028] The target data group is a group of images input as images of animals. Each image is a color image with an image size of 32 x 32 [pixels]. In this case, each target data is image data of a 3072 (= 32 x 32 x 3) dimensional vector, and the target data group is a vector data group of a 3072 (= 32 x 32 x 3) dimensional vector.

[0029] (Step S1-2) Next, the first feature calculation unit 102 inputs each target data to the first learning model and obtains the first feature vector output from the first learning model to calculate the first feature vector. The number of dimensions of the first feature vector is assumed to be 128.

[0030] (Step S1-3) Next, the second feature calculation unit 103 inputs each target data to a second learning model and obtains a second feature vector output from the second learning model to calculate a second feature vector. The number of dimensions of the second feature vector is assumed to be 128.

[0031] (Step S1-4) Next, the second cluster division unit 104 calculates second clusters by clustering the second feature vectors calculated by the second feature calculation unit 103. At this time, the second cluster division unit 104 divides each second feature vector into 30 categories by using an unsupervised clustering method such as k-means.

[0032] (Step S1-5) The integration unit 105 integrates the first feature vector calculated in the process of step S1-2 and the second cluster extracted in the process of step S1-4. At this time, for each first feature vector, the integration unit 105 extracts a second feature vector calculated from the same target data and a second cluster into which the second feature vector is classified, and associates the first feature vector with the extracted second cluster. After that, the display unit 106 displays a scatter diagram in which the first feature vector is mapped based on the integration result. At this time, the display unit 106 displays each first feature vector in a manner that allows the associated second cluster to be identified. For example, the display unit 106 changes the color, shape, size, transparency, etc. of each point indicating the first feature vector according to the second cluster and displays it. The user can check the clustering performance using the first learning model by checking the displayed scatter diagram.

[0033] (Effects of the First Embodiment) The effects of the clustering device 100 according to this embodiment will be described below.

[0034] The clustering device 100 according to the present embodiment includes a data acquisition unit 101, a first feature calculation unit 102, a second feature calculation unit 103, a second cluster division unit 104, and an integration unit 105. The data acquisition unit 101 acquires target data, a first learning model that receives the input of the target data and outputs a first feature vector, and a second learning model that receives the input of the target data and outputs a second feature vector. For example, the first learning model and the second learning model differ in model structure, learning method, hyperparameter at the time of learning, or data set at the time of learning. The first feature calculation unit 102 calculates the first feature vector using the first learning model and the target data. The second feature calculation unit 103 calculates the second feature vector using the second learning model and the target data. The second cluster division unit 104 divides the second feature vector to calculate a second cluster. The integration unit 105 integrates the first feature vector and the second cluster. Moreover, the clustering device 100 according to this embodiment further includes a display unit 106 that displays the first feature vector based on the index of the corresponding second cluster. The display unit 106 displays a scatter diagram in which the first feature vector output from the first learning model is mapped. The displayed scatter diagram visualizes the output by the first learning model. By checking the scatter diagram of the first feature vector to which the information of the second cluster is added, the user can check whether the input target data has been correctly identified and accurately check the performance of the first learning model.

[0035] Here, a method for confirming the performance of a learning model using the display according to this embodiment will be specifically described with reference to the examples of FIGS. FIG. 3 is an example of a scatter diagram in which the first feature vector is mapped. In FIG. 3, the 128-dimensional first feature vector is compressed to two dimensions for visualization and displayed. As a compression method, a method such as PCA, tSNE, or UMAP can be used. In FIG. 3, for the purpose of explanation, the first feature vector is divided into a plurality of first clusters using a known clustering method, and the display method of the first feature vector is changed according to the first cluster, but in the display process of this embodiment, the calculation of the first cluster may not be performed. In FIG. 3, the pixel value (color intensity) of each point indicating the first feature vector is changed according to the first cluster and displayed.

[0036] Fig. 4 is an example of a scatter plot in which the second feature vector is mapped. In Fig. 4, the 128-dimensional second feature vector is compressed to two dimensions for visualization. Methods such as PCA, tSNE, and UMAP can be used as a compression method. In Fig. 4, the pixel value (color intensity) of each point indicating the second feature vector is displayed changed according to the second cluster.

[0037] FIG. 5 is a scatter diagram in which the first feature vector output from the first learning model displayed on the display unit 106 is mapped. FIG. 5 is generated based on the integration result of the first feature vector shown in FIG. 3 and the second cluster shown in FIG. 4. In FIG. 5, for visualization, the 128-dimensional first feature vector is compressed to two dimensions and displayed. As a compression method, a method such as PCA, tSNE, or UMAP can be used. In FIG. 5, each first feature vector mapped in the same manner as the scatter diagram in FIG. 3 is displayed in a manner that allows identification of the second cluster of the second feature vector generated from the same target data. In FIG. 5, the pixel value (color intensity) of each point indicating the first feature vector is displayed by changing it according to the corresponding second cluster. Note that the color, shape, size, transparency, etc. of each point indicating the first feature vector may be changed according to the second cluster.

[0038] When multiple target data belonging to one cluster belong to the same cluster even when clustered using another learning model, those target data are often data that are easy to cluster. For example, when clustering animal species captured in images, data that is easy to cluster is images in which 90% or more of people can correctly classify the animal species.

[0039] On the other hand, when multiple target data belonging to one cluster belong to different clusters in cluster division using another learning model, the target data are often harmful data that are not suitable for clustering. For example, multiple target data belonging to a single cluster generated using the first learning model may belong to different clusters in the clustering results using the second learning model. In this case, since the cluster ID of the second cluster is not consistent, a single cluster with multiple cluster IDs is generated in the scatter plot that integrates the first feature vector and the second cluster. Such a cluster is highly likely to be a pseudo cluster formed by harmful data that is not suitable for clustering, since the reproducibility of the clustering results is low.

[0040] For example, it can be seen that the first feature vectors contained in region "A" belong to the same first cluster, as shown in Fig. 3. Also, it can be seen that the first feature vectors contained in region "A" belong to the same second cluster, as shown in Fig. 5. By checking the display in Fig. 5, the user can consider the first feature vectors contained in region "A" to be data that is easy to cluster.

[0041] On the other hand, it can be seen that the first feature vectors contained in region "B" belong to the same first cluster, as shown in Figure 3. However, it can be seen that the first feature vectors contained in region "B" belong to multiple second clusters, as shown in Figure 5. By checking the display in Figure 5, the user can determine that the first feature vectors contained in region "B" are highly likely to be harmful data, and that the cluster formed in region "B" is a pseudo cluster made up of harmful data.

[0042] In this way, the user can confirm the reproducibility of the clustering results using multiple learning models using the display method of the integrated results as shown in FIG. 5, thereby grasping the harmful data contained in the target data, and can analyze and analyze the performance of the first learning model taking the harmful data into consideration. For example, when multiple cluster IDs exist in a single cluster, the user can consider the cluster as a pseudo cluster formed by harmful data, and can consider and analyze the clustering results without considering the pseudo cluster. Therefore, by auxiliary use of the output of the second learning model different from the first learning model, the visibility of the clustering results of the first learning model can be improved. That is, in the clustering device 100 using the first learning model, by adding information on the clustering results of another second learning model, the visibility of the clustering results can be improved, and the clustering performance of the first learning model can be easily confirmed. The user can efficiently consider and analyze the clustering results by checking, for example, a scatter diagram of the first feature vectors displayed in a identifiable manner according to the second cluster. In addition, the user can visually grasp the pattern of each cluster formed on the scatter diagram of the first feature vectors. Therefore, even if the amount of target data increases, the user can easily and efficiently investigate what kind of image pattern each cluster formed on the scatter plot represents.

[0043] As the second learning model, it is preferable to use a machine learning model that has clustering performance close to that of the first learning model but has a different output tendency.

[0044] In this embodiment, a case has been described in which a DNN trained by changing the random number seed value of the first learning model is used as the second learning model. This method is the method in which the difference in learning conditions between the second learning model and the first learning model is the smallest. Target data in which the clustering result changes simply by changing the random number seed value is highly likely to be harmful data. Therefore, by using such a second learning model, it is possible to reduce the influence of obvious harmful data in which the clustering tendency changes depending on the initial value of the DNN weight or the order in which data is learned.

[0045] Also, as the second learning model, a DNN with structural parameters such as the total number of DNNs and the number of channels of the first learning model may be used. In this case, since the expressive capabilities and reference ranges for inputs of the first learning model and the second learning model are different, a model with a different way of grasping the resolution and complexity of data from the first learning model can be used as the second learning model. The user can grasp whether each target data is suitable for clustering, in which the clustering results do not change even when multiple models that extract features from different perspectives are used, or harmful data, in which the clustering results change.

[0046] In addition, a DNN generated by a learning method different from that of the first learning model may be used as the second learning model. For example, in learning methods such as BYOL and BallowTwins, learning is performed with PositivePair, and even if the data itself is transformed, it is treated as a positive example (same class). Therefore, a model trained with a learning method such as BYOL and BallowTwins is a model that only considers PositivePair. On the other hand, in learning methods such as ID, IDFD, and simCLR, learning is performed with NegativePair, and even if the data itself is transformed, data other than the data itself is treated as a negative example (different class). Therefore, a model trained with a learning method such as ID, IDFD, and simCLR is a model that also considers NegativePair. For example, by making one of the first learning model and the second learning model a model trained with a learning method such as BYOL or BallowTwins, and the other a model trained with a learning method such as ID, IDFD, or simCLR, a model that only considers PositivePair and a model that also considers NegativePair can be used. Even in this case, the user can understand for each target data whether it is data suitable for clustering, in which the clustering results will not change even when multiple models that extract features from different perspectives are used, or whether it is harmful data, in which the clustering results will change.

[0047] In addition, dimensionality reduction methods such as tSNE and UMAP are methods that take into account the distance between data in the input space and the compressed space. On the other hand, dimensionality reduction methods such as AE and VAE are methods that take into account the reconstruction ability from the compressed space to the input space. For this reason, by making one of the first learning model and the second learning model a model using a dimensionality reduction method such as tSNE or UMAP, and making the other a model trained by a learning method such as a dimensionality reduction method such as AE or VAE, it is possible to use a model that takes into account the distance between data in the input space and the compressed space and a model that takes into account the reconstruction ability from the compressed space to the input space. Even in this case, the user can grasp whether each target data is suitable for clustering, in which the clustering result does not change even when multiple models that perform feature extraction from different perspectives are used, or harmful data, in which the clustering result changes.

[0048] Second embodiment A second embodiment will be described. This embodiment is a modification of the configuration of the first embodiment as follows. Descriptions of the same configuration, operation, and effects as those of the first embodiment will be omitted. The clustering device 100 of this embodiment calculates first clusters obtained by cluster division using a first learning model, calculates the degree of mixture of second clusters for each first cluster, and highlights the first feature vector according to the degree of mixture.

[0049] 6 is a diagram showing a configuration of a clustering device 100 according to this embodiment. The clustering device 100 further includes a first cluster division unit 107. The first cluster division unit 107 performs clustering by dividing the first feature vector calculated by the first feature calculation unit 102 to calculate first clusters. A general clustering method such as k-means can be used for the clustering. For example, various methods such as a centroid method and kernel density estimation can be applied to the clustering.

[0050] The number of clusters in the first cluster is preferably the same as the number of clusters in the second cluster, but may be different from the number of clusters in the first cluster. For example, the number of clusters in the first cluster may be a value specified by the user.

[0051] The integration unit 105 integrates the first feature vector and the second cluster by calculating the degree of mixing of the second cluster in a single cluster for the first cluster. The degree of mixing is an index representing the degree of mixing of the cluster IDs of the second cluster in a single first cluster. The greater the number of second clusters to which the target data in a single first cluster belongs (the greater the types of second clusters), the greater the degree of mixing. As the degree of mixing, for example, the inverse of the maximum ratio of the second cluster IDs included in the first cluster, the entropy of the distribution of the second clusters included in the first cluster, or the total number of second cluster IDs included in the first cluster and having a content rate of 3% or more can be used.

[0052] The value of the mixture degree is small when multiple target data belonging to the same first cluster are classified into the same second cluster, and is large when target data belonging to the same first cluster are classified into different second clusters. Therefore, the mixture degree can also be called an index that represents the consistency and reproducibility of the clustering results when clustering is performed using multiple learning models.

[0053] The display unit 106 displays the clustering result using the first learning model. At this time, the display unit 106 displays the clustering result using the first learning model based on the integration result by the integration unit 105. In this embodiment, the display unit 106 displays a scatter diagram in which the first feature vector is mapped according to the mixture degree of the first cluster to which the first feature vector belongs. At this time, the display unit 106 highlights the first feature vector so that the difference in the calculated mixture degree can be visually recognized. As a display method, for example, the first feature vector may be displayed in color using a color bar according to the mixture degree, or the size, color, shape, transparency, etc. of a point indicating the first feature vector may be changed according to the mixture degree.

[0054] Next, the operation of the display process executed by the clustering device 100 of this embodiment will be described. Fig. 7 is a flowchart showing an example of the procedure of the display process. Here, as an example, a case will be described in which a model for clustering images in which any of a plurality of types of animals appears as target data by animal type is used as a learning model to be learned.

[0055] The processes in steps S2-1 to S2-3 and S2-5 are the same as those in steps S1 to S1-4 in FIG. 2, and therefore will not be described.

[0056] (Step S2-4) The first cluster division unit 107 calculates first clusters by clustering the first feature vectors calculated by the first feature calculation unit 102. At this time, the first cluster division unit 107 divides each of the first feature vectors into 30 categories using an unsupervised clustering method such as k-means.

[0057] (Step S2-6) The integration unit 105 integrates the first feature vector and the second cluster. At this time, the integration unit 105 first acquires the first cluster extracted in the process of step S2-4 and the second cluster extracted in the process of step S2-5. Next, for each first cluster, the integration unit 105 extracts a second cluster to which the target data included in the first cluster belongs, and calculates the degree of mixing of the second cluster in the first cluster based on the number of extracted second clusters.

[0058] Next, the display unit 106 displays a scatter diagram in which the first feature vectors are mapped based on the integration result. At this time, the display unit 106 displays each first feature vector in a manner according to the mixture degree of the first cluster to which the first feature vector belongs. For example, the shape, color, and size of the point indicating each first feature vector are changed according to the mixture degree. This scatter diagram is used to check the clustering performance of the first learning model.

[0059] (Effects of the second embodiment) The effects of the clustering device 100 according to this embodiment will be described below. The clustering device 100 according to this embodiment includes a data acquisition unit 101, a first feature calculation unit 102, a second feature calculation unit 103, a second cluster division unit 104, an integration unit 105, and a display unit 106, as well as a first cluster division unit 107.

[0060] The first cluster division unit 107 divides the first feature vector to calculate the first cluster. For example, the number of clusters in the first cluster and the number of clusters in the second cluster are set to the same number. The integration unit 105 calculates the mixing degree of the second cluster in a single cluster for the first cluster. The display unit 106 displays the first feature vector based on the mixing degree of the corresponding first cluster.

[0061] With the above configuration, the clustering device 100 of the present embodiment can improve the visibility of the clustering results by integrating the first feature vector calculated using the first learning model and the second cluster calculated using the second learning model, calculating the mixture degree of the second cluster for each first cluster, and displaying the first feature vector according to the mixture degree. For example, a user can check a scatter diagram of the first feature vectors color-coded according to the mixture degree to improve the efficiency of examining and analyzing the clustering results.

[0062] Harmful data that is not suitable for clustering has low consistency and reproducibility when clustered using a different learning model. Therefore, in a first cluster that contains a lot of harmful data that is not suitable for clustering, the number of second clusters is likely to be large. Therefore, images that belong to a first cluster with a low degree of mixture are often image patterns that are easy to cluster. On the other hand, images that belong to a first cluster with a high degree of mixture are often harmful data that is not suitable for clustering. Therefore, the degree of mixture of the cluster ID of the second cluster in each cluster of the first cluster can be an index for measuring whether the image pattern is easy to cluster. For example, a user can consider a first cluster with a high degree of mixture as a pseudo cluster formed by harmful data and consider and analyze the clustering result without considering the pseudo cluster. In this way, the visibility of the clustering result of the first learning model can be improved by auxiliary use of the output of the second learning model that is learned under conditions different from the first learning model. That is, in the clustering device 100 using the first learning model, by adding information on the clustering result of another second learning model, the clustering performance of the first learning model can be easily confirmed.

[0063] Furthermore, by making the number of clusters in the first cluster and the number of clusters in the second cluster the same, the formula for calculating the degree of mixture can be simplified, and the time required to calculate the degree of mixture can be reduced.

[0064] As described above, in this embodiment, by displaying a scatter plot of the first feature vectors for each first cluster according to the degree of mixing with the second cluster, it is possible to improve the visibility of the clustering results and increase the efficiency of considering and analyzing the clustering results.

[0065] (Modification of the second embodiment) Among the target data, target data having a certain degree of mixing or more may be presented to the user. For example, the display unit 106 extracts images having a certain degree of mixing or more, and displays the extracted images as thumbnails. The user can efficiently check harmful data by checking only images having a high degree of mixing.

[0066] Also, the target data to be assigned to the first cluster may be selected according to the degree of mixture. For example, the display unit 106 extracts target data whose degree of mixture is equal to or less than a certain level, and displays only the first feature vectors of the extracted target data in a scatter diagram. By not displaying target data whose degree of mixture is high in the scatter diagram of the first feature vectors, it is possible to display a scatter diagram from which data that is likely to be harmful has been removed in advance.

[0067] Also, the first cluster may be recalculated using the degree of mixture. For example, the first cluster division unit 107 adjusts the weight of the calculation for obtaining the center of gravity of the k-means method for each first feature vector to a value inversely proportional to the degree of mixture. Since target data with a high degree of mixture is highly likely to be harmful data, the influence of harmful data can be reduced by reducing the weight of target data with a high degree of mixture when calculating the first cluster.

[0068] In addition, the mixture degree calculated using the first learning model and the second learning model may be used to train another learning model. For example, a third learning model may be trained using the target data and the mixture degree of the target data. The third learning model is a machine learning model that receives the input of the target data and outputs a third feature vector. Since the target data with a high mixture degree is highly likely to be harmful data, by performing training in consideration of the mixture degree, the influence of harmful target data with a high mixture degree can be reduced, and a learning model with high clustering performance can be generated.

[0069] The clustering device 100 of the present embodiment may also have a search function for searching for data similar to predetermined search data from among the target data. In this case, the clustering device 100 further includes a search unit for calculating the similarity between each piece of target data and the search data, and searching for target data similar to the search data (hereinafter, referred to as similar data) based on the similarity of each piece of target data. The search unit inputs predetermined data into the first learning model or the second learning model, thereby acquiring the first feature vector or the second feature vector of the search data as a search feature vector, and calculating the similarity between the acquired search feature vector and the first feature vector or the second feature vector of each piece of target data. Thereafter, the search unit extracts target data having a similarity equal to or greater than a threshold value as similar data, and presents the extracted similar data to a user.

[0070] In addition, in the search for similar data, the search results may be adjusted according to the degree of mixture of each target data calculated by the integration unit. In this case, the search unit reduces the similarity of target data with a high degree of mixture, making it difficult for target data with a high degree of mixture to be extracted as similar data. Since target data with a high degree of mixture is highly likely to be harmful data, the influence of harmful data when searching for similar data can be reduced, and search performance can be improved.

[0071] (Modification of the first or second embodiment) In the above embodiment, the case where image data is used as the target data has been described, but other types of data that can be clustered may be used as the target data. For example, the target data may be voice data, table data, sensor data such as acceleration and voltage, etc.

[0072] In the above embodiment, the case where the DNN is used as the learning model is described, but other machine learning models that can be used for clustering may be used. For example, a multiple regression analysis model, an SVM, a decision tree model, etc. may be used as the learning model.

[0073] In the above embodiment, the case where one second learning model is used has been described, but multiple second learning models may be used. For example, multiple second clusters acquired from multiple second learning models may be integrated, and the first feature vector in the scatter diagram may be displayed according to the integrated second cluster. Also, the degree of mixture of the first feature vector may be calculated using the integrated second cluster, and the first feature vector in the scatter diagram may be displayed according to the degree of mixture. When multiple second clusters are integrated, the difference in the learning conditions between each second learning model and the first learning model may be quantified, and the integration ratio of each second cluster may be adjusted according to the value. As the difference in the learning conditions with the first learning model, for example, the difference in the hyperparameters during learning or the difference in the number of parameters related to the model structure may be used.

[0074] (Examples) Fig. 8 is a diagram showing a hardware configuration of the clustering device 100 of Fig. 1 and Fig. 6. As shown in Fig. 8, the clustering device 100 is a computer having a processor 1, a ROM (Read Only Memory) 2, a RAM (Random Access Memory) 3, an auxiliary storage device 4, an input device 5, a display device 6, and a communication device 7. Transmission and reception of data and various signals between the processor 1, ROM 2, RAM 3, the auxiliary storage device 4, the input device 5, the display device 6, and the communication device 7 is performed via a bus.

[0075] The processor 1 is an integrated circuit that controls the overall operation of the clustering device 100. For example, the processor 1 has a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), and / or a floating-point unit (FPU). The processor 1 may also have an internal memory and an I / O interface. The processor 1 executes various processes by interpreting and calculating programs stored in advance in the ROM 2 or the auxiliary storage device 4. The processor 1 may be realized in part or in whole by hardware such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0076] The ROM 2 is a non-volatile memory that stores various data. For example, the ROM 2 stores data and setting values ​​used when the processor 1 executes various processes. The ROM 2 may include a non-transitory computer-readable storage medium that stores a program executed by the processor 1.

[0077] The RAM 3 is a volatile memory used for reading and writing data. The RAM 3 temporarily stores data used when the processor 1 executes various processes. The RAM 3 provides a work area for the processor 1.

[0078] The auxiliary storage device 4 is a non-volatile memory that stores various data. For example, the auxiliary storage device 4 stores data and setting values ​​used when the processor 1 executes various processes, data generated by various processes in the processor 1, etc. The auxiliary storage device 4 is configured with an HDD (Hard Disk Drive), an SSD (Solid State Drive), an integrated circuit storage device, etc. The auxiliary storage device 4 may have a non-transitory computer-readable storage medium that stores a program executed by the processor 1.

[0079] The input device 5 accepts various operation inputs from the user. Examples of the input device 5 that can be used include a keyboard, a mouse, various switches, a touch pad, and a touch panel display. An electrical signal corresponding to the accepted operation input (hereinafter, an operation signal) is supplied to the processor 1.

[0080] The display device 6 displays various data under the control of the processor 1. A CRT (Cathode-Ray Tube) display, a liquid crystal display, an organic EL (Electro Luminescence) display, an LED (Light-Emitting Diode) display, a plasma display, or any other display may be appropriately used as the display device 6. The display device 6 may be a projector.

[0081] The communication device 7 includes a communication interface such as a network interface card (NIC) for performing data communication with various devices connected to the clustering device 100 via a network. An operation signal may be supplied from a computer connected via the communication device 7 or an input device included in the computer, and various data may be displayed on a display device or the like included in the computer connected via the communication device 7. However, for the sake of simplicity of the following description, unless otherwise specified, it is assumed that the source of the operation signal is the input device 5 and the display destination of the various data is the display device 6. The input device 5 can be replaced by a computer connected via the communication device 7 or an input device included in the computer, and the display device 6 can be replaced by a display device or the like included in the computer connected via the communication device 7.

[0082] The clustering device 100 does not need to include all of the processor 1, ROM 2, RAM 3, auxiliary storage device 4, input device 5, display device 6, and communication device 7. If necessary, some of the processor 1, ROM 2, RAM 3, auxiliary storage device 4, input device 5, display device 6, and communication device 7 may not be provided. The clustering device 100 may be provided with any additional hardware device useful for executing the process according to this embodiment. The clustering device 100 does not need to be physically composed of one computer, and may be composed of a computer system having multiple computers connected to each other so as to be able to communicate via wires or a network line. The allocation of the series of processes according to this embodiment to the multiple processors 1 implemented in each of the multiple computers can be set arbitrarily. All the processors 1 may execute all the processes in parallel, or a specific process may be assigned to one or some of the processors 1, and the series of processes according to this embodiment may be executed by the entire computer system.

[0083] Thus, according to any of the above-described embodiments, it is possible to provide a clustering device, method, and program that can improve the visibility of the clustering results or the clustering performance.

[0084] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0085] 100...clustering device, 101...data acquisition unit, 102...first feature calculation unit, 103...second feature calculation unit, 104...second cluster division unit, 105...integration unit, 106...display unit, 107...first cluster division unit, 1...processor, 2...ROM, 3...RAM, 4...auxiliary storage device, 5...input device, 6...display device, 7...communication device.

Claims

1. a data acquisition unit that acquires target data, a first learning model that receives an input of the target data and outputs a first feature vector, and a second learning model that receives an input of the target data and outputs a second feature vector; a first feature calculation unit that calculates the first feature vector by using the first learning model and the target data; a second feature calculation unit that calculates the second feature vector by using the second learning model and the target data; a second cluster division unit that divides the second feature vector to calculate a second cluster; an integration unit that integrates the first feature vector and the second cluster; A clustering device comprising:

2. a display unit that displays the first feature vector based on an index of the corresponding second cluster, The clustering device according to claim 1 .

3. The first learning model and the second learning model have different model structures, learning methods, hyperparameters at the time of learning, or data sets at the time of learning. The clustering device according to claim 1 .

4. A first cluster division unit that divides the first feature vector to calculate a first cluster, The integration unit calculates a degree of mixing of the second cluster within a single cluster for the first cluster. The clustering device according to claim 1 .

5. a display unit that displays the first feature vector based on a mixing degree of the corresponding first cluster, The clustering device according to claim 4.

6. the number of clusters in the first cluster is equal to the number of clusters in the second cluster; The clustering device according to claim 4.

7. The integration unit selects the target data to be assigned to the first cluster according to the degree of mixture. The clustering device according to claim 4.

8. Further comprising a learning unit configured to learn a third learning model based on the target data and the mixture degree. The clustering device according to claim 4.

9. a search unit that calculates a similarity between a search feature vector obtained by inputting search data into the first learning model or the second learning model and the first feature vector or the second feature vector for each of the plurality of target data, and searches for similar data similar to the search data from among the plurality of target data according to the similarity; The search unit adjusts the search results according to the degree of mixture of the target data. The clustering device according to claim 4.

10. The search unit reduces the similarity of the target data having a large degree of mixture. The clustering apparatus according to claim 9.

11. the first cluster division unit recalculates the first clusters using the mixture degree; The clustering device according to claim 4.

12. acquiring target data, a first learning model that receives an input of the target data and outputs a first feature vector, and a second learning model that receives an input of the target data and outputs a second feature vector; Calculating the first feature vector using the first learning model and the target data; Calculating the second feature vector using the second learning model and the target data; Dividing the second feature vector to calculate a second cluster; Integrating the first feature vector and the second cluster; A method for providing the above.

13. On the computer, A function of acquiring target data, a first learning model that receives the target data and outputs a first feature vector, and a second learning model that receives the target data and outputs a second feature vector; A function of calculating the first feature vector using the first learning model and the target data; A function of calculating the second feature vector using the second learning model and the target data; A function of dividing the second feature vector to calculate a second cluster; a function of integrating the first feature vector and the second cluster; A program to achieve this.

Images