Data analysis apparatus, method, and program

Abstract

To estimate the impact of data augmentation on clustering. [Solution] A data analysis device according to one embodiment acquires multiple target data, learns a first learning model by unsupervising learning the multiple target data using a first data augmentation condition which is a condition relating to a data augmentation transformation method, generates multiple first feature vectors corresponding to the multiple target data, generates a first clustering result by clustering the multiple first feature vectors, learns a second learning model by unsupervising learning the multiple target data using a second data augmentation condition which has different conditions relating to a transformation method from the first data augmentation condition, generates multiple second feature vectors corresponding to the multiple target data, and generates a comparison result by comparing the multiple first feature vectors and the multiple second feature vectors for each of the multiple clusters based on the first clustering result.

Description

Technical Field

[0001] Embodiments of the present invention relate to a data analysis apparatus, method, and program.

Background Art

[0002] Conventionally, in the learning method of machine learning, unsupervised learning is known in which a machine learning model learns the features of target data without giving a classification label as correct answer data. In this unsupervised learning, since the classification label is unknown, it may be classified into the number of clusters that reflects the features of the target data. However, there are cases where target data with different tendency features are classified into the same cluster, and it may be difficult to interpret the clustering result.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide a data analysis apparatus, method, and program capable of estimating the influence of data augmentation on clustering.

Means for Solving the Problems

[0005] A data analysis device according to one embodiment comprises an acquisition unit, a first learning unit, a clustering unit, and a comparison unit. The acquisition unit acquires a plurality of target data. The first learning unit learns a first learning model by unsupervised learning of the plurality of target data using a first data augmentation condition, which is a condition relating to a data augmentation transformation method, and generates a plurality of first feature vectors corresponding to the plurality of target data. The clustering unit generates a first clustering result by clustering the plurality of first feature vectors. The second learning unit learns a second learning model by unsupervised learning of the plurality of target data using a second data augmentation condition, which has different conditions relating to a transformation method from the first data augmentation condition, and generates a plurality of second feature vectors corresponding to the plurality of target data. The comparison unit generates a comparison result by comparing the plurality of first feature vectors and the plurality of second feature vectors for each of the plurality of clusters based on the first clustering result. [Brief explanation of the drawing]

[0006] [Figure 1] A block diagram illustrating the configuration of a data analysis device according to this embodiment. [Figure 2] A block diagram illustrating the specific configuration of the first learning section in Figure 1. [Figure 3] A flowchart illustrating the operation of the data analysis device according to the embodiment. [Figure 4] A diagram showing a first specific example of the target data in the embodiment. [Figure 5] A diagram showing a second specific example of the target data in the embodiment. [Figure 6] A diagram showing a third specific example of the target data in the embodiment. [Figure 7] A figure showing a fourth specific example of the target data in the embodiment. [Figure 8] A flowchart showing a specific example of the first learning process in the flowchart of Figure 3. [Figure 9] A flowchart showing a specific example of the second learning process in the flowchart of Figure 3. [Figure 10]This figure shows a specific example of a display image that includes a scatter plot visualizing multiple first feature vectors in the embodiment. [Figure 11] A figure showing a first specific example of a display image including a scatter plot visualizing multiple second feature vectors in an embodiment. [Figure 12] A second specific example of a display image including a scatter plot visualizing multiple second feature vectors in the embodiment. [Figure 13] A figure showing a first specific example of the comparison results in the embodiment. [Figure 14] A figure showing a second specific example of the comparison results in the embodiment. [Figure 15] A figure showing another specific example of a display image including a scatter plot in the embodiment. [Figure 16] A block diagram illustrating the hardware configuration of a computer according to one embodiment. [Modes for carrying out the invention]

[0007] The following describes in detail an embodiment of the data analysis device with reference to the drawings.

[0008] (Embodiment) In this embodiment, image data including shapes will be used as the data to be analyzed (hereinafter referred to as the target data). The data analysis device will use a machine learning model that clusters these images according to the type of shape using unsupervised learning. For example, a deep neural network (DNN) will be used as the machine learning model. That is, the learning model in this embodiment is a DNN model.

[0009] Figure 1 shows an example of the configuration of a data analysis device according to an embodiment. The data analysis device 100 in Figure 1 comprises an acquisition unit 110, a first learning unit 120, a clustering unit 130, a cluster selection unit 140 (selection unit), a second learning unit 150, a comparison unit 160, and a display control unit 170.

[0010] The acquisition unit 110 acquires a plurality of target data. The acquisition unit 110 outputs the plurality of target data to the first learning unit 120 and the second learning unit 150.

[0011] The above-mentioned target data is, for example, image data including shapes such as circles, triangles, and quadrilaterals. In a specific example of the embodiment, the image data is, for example, a color image with an image size of 32×32 pixels. That is, the target data is a group of vector data of 3072-dimensional vectors of 32×32×3 (RGB values). Note that the target data may be called learning data.

[0012] Also, the acquisition unit 110 may acquire the first learning condition and the second learning condition. At this time, the acquisition unit 110 outputs the first learning condition to the first learning unit 120 and the second learning condition to the second learning unit 150. Hereinafter, an overview of the learning conditions common to the first learning condition and the second learning condition will be described.

[0013] The above-mentioned learning conditions include, for example, the model structure of DNN, structural parameters, loss functions, and optimization parameters. Examples of the DNN model structure include ResNet, MobileNet, and EfficientNet specialized for image classification. The structural parameters include, for example, the number of network layers, the number of nodes in each layer, the connection method between each layer, and the type of activation function used in each layer. Examples of the loss functions include SimCLR (a simple framework for contrastive learning of visual representations), BYOL (Bootstrap Your Own Latent), and Brlow Twins. The optimization parameters include, for example, the type of optimizer (Momentum SGD (Stochastic Gradient Descent), Adam (Adaptive moment estimation), etc.), the learning rate (or learning rate schedule), the number of updates (the number of iterative learning), the number of mini-batches (mini-batch size), and the strength of WeightDecay. Also, the learning conditions include data augmentation conditions described later.

[0014] The first learning unit 120 receives a plurality of target data from the acquisition unit 110. The first learning unit 120 uses the plurality of target data to learn (iteratively learn) the first machine learning model under the first learning conditions. The first learning conditions include the first data augmentation conditions described later. The first learning unit 120 outputs a plurality of first feature vectors by inputting the plurality of target data to the first learned model that is the first machine learning model for which learning has been completed. The first learning unit 120 outputs the plurality of first feature vectors to the clustering unit 130 and the comparison unit 160.

[0015] Also, when the acquisition unit 110 has acquired the first learning conditions, the first learning unit 120 may receive the first learning conditions from the acquisition unit 110. Also, the first learning unit 120 may set the first learning conditions for the learning of the first machine learning model. Hereinafter, the specific configuration of the first learning unit 120 will be described using FIG. 2.

[0016] FIG. 2 is a block diagram illustrating the specific configuration of the first learning unit in FIG. 1. The first learning unit 120 in FIG. 2 includes a feature vector calculation unit 210, a loss calculation unit 220, a model update unit 230, and a model storage unit 240. In the following sections, the processing of one of the plurality of target data will be described.

[0017] The feature vector calculation unit 210 calculates a first feature vector based on the target data. Specifically, the feature vector calculation unit 210 outputs (calculates) a first feature vector by inputting the target data to the first machine learning model stored in the model storage unit 240. The feature vector calculation unit 210 outputs the calculated first feature vector to the loss calculation unit 220. In this embodiment, the first feature vector is, for example, 128-dimensional vector data output from the output layer of the DNN.

[0018] Furthermore, when calculating the loss obtained by training the first machine learning model, the feature vector calculation unit 210 outputs the first feature vector output from the output layer of the DNN. On the other hand, after training the first machine learning model, the feature vector calculation unit 210 may output the output of an intermediate layer prior to the output layer (for example, several layers prior to the output layer) as the first feature vector.

[0019] The loss calculation unit 220 receives the first feature vector from the feature vector calculation unit 210. The loss calculation unit 220 calculates the loss using the first feature vector. The loss calculation unit 220 outputs the loss to the model update unit 230.

[0020] In this embodiment, data augmentation, which is used to improve the learning accuracy of unsupervised learning, is used in the calculation of loss. Examples of data augmentation transformation methods for image data used in this embodiment include scaling, image rotation, and black and white inversion. The first data augmentation condition described above is a condition relating to the data augmentation transformation method set in the first learning unit 120. The model structure and structural parameters of the DNN used in the first learning unit 120 are set by the first learning condition. A specific example of unsupervised learning using data augmentation will be described below.

[0021] The loss calculation unit 220 calculates the loss using, for example, SimCLR, which is one of the unsupervised learning methods. The loss L using SimCLR can be obtained, for example, by the following equations (1) and (2).

[0022]

number

[0023]

number

[0024] In equation (1), N represents the number of target data used in loss calculation (corresponding to the mini-batch size when performing probabilistic optimization), and i and j represent the sequential numbers of two types of samples augmented from the same target data. SimCLR uses two types of samples obtained from a single target data through data augmentation, so the total number of samples is 2N.

[0025] Also, indicator function 1 [k≠i] `x` represents a function that is 1 when k≠1 and 0 otherwise, and `sim(A,B)` represents a function (e.g., a cosine function) that outputs a larger number as the similarity between A and B increases. `z` represents the output vector (feature vector) of the DNN, the subscripts of `z` (e.g., i, j, and k) represent the sequential numbers of the target data, and `τ` represents the temperature parameter related to the loss. The temperature parameter `τ` can be used to adjust the sensitivity of the numerical value output by the `sim` function; a smaller value results in higher sensitivity, and a larger value results in lower sensitivity.

[0026] In other words, the loss calculation unit 220 calculates the loss using a method (for example, SimCLR) in which the smaller the error between two different feature vectors obtained from the same target data, and the larger the error between two different feature vectors obtained from different target data, the smaller the loss.

[0027] The model update unit 230 receives the loss from the loss calculation unit 220. The model update unit 230 updates the parameters of the machine learning model using the loss. The model update unit 230 outputs the updated machine learning model parameters to the model storage unit 240.

[0028] Specifically, the model update unit 230 updates the machine learning model's parameters by applying optimization parameters based on the loss to the machine learning model. The optimization parameters are set by the first learning condition.

[0029] The model storage unit 240 receives machine learning model parameters from the model update unit 230. The model storage unit 240 stores the machine learning model updated based on the parameters.

[0030] In summary, the first learning unit 120 iteratively learns a first machine learning model (first learning model) by unsupervising learning multiple target data using a first data augmentation condition, which is a condition relating to the transformation method of data augmentation, and generates multiple first feature vectors corresponding to the multiple target data.

[0031] The clustering unit 130 receives multiple first feature vectors from the first learning unit 120. The clustering unit 130 generates a first clustering result by clustering the multiple first feature vectors. The clustering unit 130 outputs the first clustering result to the cluster selection unit 140 and the comparison unit 160.

[0032] One example of a clustering method used is the K-Means method (K-Means clustering). The clustering unit 130 generates a first clustering result with an arbitrary number of clusters by clustering multiple first feature vectors, for example, using the K-Means method. The arbitrary number of clusters can be specified by the user, or it can be specified using a cluster number estimation method. Examples of cluster number estimation methods include the elbow method and silhouette analysis.

[0033] The first clustering result described above includes, for example, a cluster number, which is the ID of the cluster to which the first feature vector belongs. Specifically, the first clustering result includes, for example, data associating the first feature vector with the cluster number. Alternatively, for example, the first clustering result may also include data associating the first feature vector with the target data corresponding to the first feature vector and the cluster number.

[0034] Furthermore, the clustering unit 130 may assign cluster labels corresponding to cluster numbers. Cluster label assignment can be done manually by a user or automatically using machine learning. In manual assignment, the user examines the data (images) included in the clusters and assigns a cluster label to each cluster, for example, one that indicates the image features. In automatic assignment, the images included in the clusters are analyzed using machine learning, and cluster labels are automatically assigned. Therefore, the first clustering result may include data that associates the first feature vector with the cluster labels. Alternatively, the first clustering result may include data that associates the first feature vector with the target data corresponding to the first feature vector and the cluster labels.

[0035] The cluster selection unit 140 receives the first clustering result from the clustering unit 130. The cluster selection unit 140 selects one or more clusters from among the multiple clusters included in the first clustering result. The cluster selection unit 140 outputs information about the selected one or more clusters (selected cluster information) to the comparison unit 160.

[0036] The cluster selection unit 140 may also determine an upper limit on the number of clusters to be selected. For example, the cluster selection unit 140 may select one or more clusters from among the multiple clusters included in the first clustering result, but less than the number of clusters.

[0037] The second learning unit 150 receives multiple target data from the acquisition unit 110. The second learning unit 150 uses the multiple target data to train (iteratively train) the second machine learning model under the second learning conditions. The second learning conditions include the second data augmentation conditions, which will be described later. The second learning unit 150 outputs multiple second feature vectors by inputting multiple target data into the second trained model, which is the second machine learning model after training is complete. The second learning unit 150 outputs the multiple second feature vectors to the comparison unit 160.

[0038] Furthermore, the second learning unit 150 may receive the second learning conditions from the acquisition unit 110 if the acquisition unit 110 has acquired them. The second learning unit 150 may also set the second learning conditions for training the second machine learning model. Note that the specific configuration of the second learning unit 150 is substantially the same as that of the first learning unit 120 shown in Figure 2, so a detailed explanation is omitted.

[0039] The aforementioned second data augmentation condition is a condition relating to the data augmentation transformation method set in the second learning unit 150. Furthermore, the second data augmentation condition differs from the first data augmentation condition in that it relates to the transformation method.

[0040] As a concrete example of different conditions regarding transformation methods, the set of transformation methods constituting the second data augmentation condition is a subset of the set of transformation methods constituting the first data augmentation condition. In other words, the second data augmentation condition is a subset of the first data augmentation condition.

[0041] Another specific example is that the first and second data augmentation conditions are each composed of the same set of transformation methods, but with different parameters relating to the degree of transformation associated with one or more of the transformation methods in the set.

[0042] Furthermore, when focusing on clustering results resulting from differences in data augmentation conditions, it is effective to keep conditions other than the data augmentation conditions (e.g., model structure and structure parameters) the same in the first and second learning conditions. In other words, the first and second learning conditions should differ only in the data augmentation conditions.

[0043] In summary, the second learning unit 150 iteratively learns a second machine learning model (second learning model) by unsupervising learning multiple target data using a second data augmentation condition that differs from the first data augmentation condition in terms of the transformation method, thereby generating multiple second feature vectors corresponding to multiple target data.

[0044] The comparison unit 160 receives multiple first feature vectors from the first learning unit 120, the first clustering result from the clustering unit 130, the selected cluster information from the cluster selection unit 140, and multiple second feature vectors from the second learning unit 150. Based on the first clustering result, the comparison unit 160 generates a comparison result by comparing multiple first feature vectors and multiple second feature vectors for each of the multiple clusters. The comparison unit 160 outputs the comparison result to the display control unit 170.

[0045] Specifically, the comparison unit 160 calculates, for example, the dispersion of multiple first feature vectors included in the cluster to be compared (first dispersion) and the dispersion of multiple second feature vectors included in the cluster to be compared (second dispersion), and generates a comparison result that includes the first dispersion and the second dispersion. The cluster to be compared is, for example, a cluster included in the selected cluster information. Therefore, the comparison unit 160 may generate a comparison result for one or more selected clusters.

[0046] For example, the comparison unit 160 calculates the difference between the first dispersion and the second dispersion and generates a comparison result that includes the difference in dispersion. That is, the comparison result may include at least one of the first dispersion and the second dispersion for the selected cluster, as well as the difference in dispersion between the first dispersion and the second dispersion. The comparison result may also include at least one of the information for the selected cluster and the information for the data augmentation conditions.

[0047] The dispersion described above is calculated, for example, using the standard deviation, variance, sum of the differences between sample pairs of feature vectors in the comparison cluster, and the maximum range of the cluster's distribution. A low dispersion of a cluster indicates that the samples within the cluster are clustered together, while a high dispersion indicates that the samples within the cluster are dispersed.

[0048] Furthermore, the comparison unit 160 may generate a second clustering result for a plurality of second feature vectors based on the first clustering result and the plurality of second feature vectors. In this way, the comparison unit 160 may generate a comparison result by comparing the first clustering result and the second clustering result. The second clustering result includes, for example, the cluster number used in the first clustering result. Specifically, the second clustering result includes, for example, data associating the second feature vector with the cluster number. The second clustering result may also include data associating the second feature vector with the cluster label. Furthermore, the second clustering result may be further associated with target data corresponding to the second feature vector.

[0049] Furthermore, the comparison unit 160 may generate a scatter plot to visualize the clustering results. Specifically, the comparison unit 160 uses dimensionality reduction techniques such as PCA, t-SNE, and UMAP to represent the feature vector with multiple different components and generates a scatter plot in which each point of the feature vector is grouped into clusters based on the clustering results. If there are two different components, the comparison unit 160 generates a two-dimensional scatter plot. If there are three different components, the comparison unit 160 generates a three-dimensional scatter plot. Grouping means, for example, distinguishing each cluster. For example, the comparison unit 160 generates a scatter plot in which each cluster can be identified by displaying the coordinate points corresponding to the feature vector with different colors and shapes for each cluster.

[0050] The display control unit 170 receives the comparison result from the comparison unit 160. The display control unit 170, for example, causes the comparison result to be displayed on the display. The display control unit 170 may also, for example, display a display image that includes a scatter plot in which at least one of the plurality of first feature vectors and plurality of second feature vectors is represented by a plurality of different components, and each point of the feature vectors is grouped into clusters based on the first clustering result. The display image may include, for example, the scatter plot and display information related to the scatter plot. The display information includes, for example, the type of display data included in the scatter plot (e.g., the type of feature vector), the type of transformation method included in the data augmentation condition, and at least one of the representative images of each cluster. The display control unit 170 may also display the comparison result and the display image.

[0051] The data analysis device 100 may also include memory and a processor. The memory stores, for example, various programs related to the operation of the data analysis device 100 (for example, data analysis programs). The processor reads and executes the various programs stored in memory to realize the functions of the acquisition unit 110, the first learning unit 120, the clustering unit 130, the cluster selection unit 140, the second learning unit 150, the comparison unit 160, and the display control unit 170.

[0052] Furthermore, the data analysis device 100 does not need to consist of a single physical computer, but may be composed of a computer system (e.g., a data analysis system) having multiple computers that are connected to each other via wired or network lines. The assignment of the series of processes according to the embodiment to the multiple processors implemented in each of the multiple computers can be arbitrarily set. All processors may execute all processes in parallel, or specific processes may be assigned to one or some processors, and the series of processes according to the embodiment may be executed by the computer system as a whole. Typically, the first learning unit 120 and the second learning unit 150 in the embodiment may be performed by an external computer.

[0053] The configuration of the data analysis device 100 according to this embodiment has been described above. Next, the operation of the data analysis device 100 according to this embodiment will be explained using the flowchart in Figure 3.

[0054] Figure 3 is a flowchart illustrating the operation of a data analysis device according to an embodiment. The process in the flowchart of Figure 3 begins, for example, when a data analysis program is selected by the user and executed by the processor.

[0055] (Step ST101) The acquisition unit 110 acquires multiple target data. Hereafter, the target data will be assumed to be image data containing one of the following shapes: a circle, a triangle, or a square.Specific examples of target data will be explained below using Figures 4 to 7.

[0056] Figure 4 shows a first specific example of the target data in the embodiment. The first specific example is an image containing black circles. Figure 4 shows variations of the image containing black circles, namely image BC-1, image BC-2, image BC-3, ..., image BC-n1. Note that n1 is the total number of image data containing black circles.

[0057] Figure 5 shows a second specific example of the target data in the embodiment. The second specific example is an image containing black triangles. Figure 5 shows variations of the image containing black triangles, namely image BT-1, image BT-2, image BT-3, ..., image BT-n2. Note that n2 is the total number of image data containing black triangles.

[0058] Figure 6 shows a third specific example of the target data in the embodiment. The third specific example is an image containing white rectangles. Figure 6 shows variations of the image containing white rectangles, namely image WR-1, image WR-2, image WR-3, ..., image WR-n3. Note that n3 is the total number of image data containing white rectangles.

[0059] Figure 7 shows a fourth specific example of the target data in the embodiment. The fourth specific example is an image containing white circles. Figure 7 shows variations of the image containing white circles, namely image WC-1, image WC-2, image WC-3, ..., image WC-n4. Note that n4 is the total number of image data containing white circles.

[0060] In the following explanation, the multiple target data will be assumed to be a mixture of the image data shown in the first to fourth specific examples above. Furthermore, the data analysis device 100 aims to classify this image data by type.

[0061] (Step ST102) After the acquisition unit 110 acquires multiple target data, the first learning unit 120 uses the multiple target data to train the first machine learning model under the first learning conditions. Hereafter, the process in step ST102 will be referred to as the "first learning process." A specific example of the first learning process will be explained below using the flowchart in Figure 8.

[0062] Figure 8 is a flowchart showing a specific example of the first learning process in the flowchart of Figure 3. The flowchart in Figure 8 transitions from step ST101 in the flowchart of Figure 3.

[0063] (Step ST201) After the acquisition unit 110 acquires multiple target data, the first learning unit 120 sets first learning conditions, which include first data augmentation conditions. As a specific example thereafter, the first data augmentation conditions include three transformation methods: "scaling," "image rotation," and "black and white inversion."

[0064] (Step ST202) After the first learning unit 120 sets the first learning conditions, the feature vector calculation unit 210 calculates the first feature vector based on the target data. Note that the target data here is the data transformed by the first data augmentation conditions.

[0065] (Step ST203) After the feature vector calculation unit 210 calculates the first feature vector, the loss calculation unit 220 calculates the loss using the first feature vector.

[0066] (Step ST204) After the loss calculation unit 220 calculates the loss, the model update unit 230 updates the first machine learning model using the loss.

[0067] Furthermore, it is preferable to perform "iterative learning" (probabilistic optimization) by repeating the processes from steps ST202 to ST204 above on a subset of data (mini-batch) randomly selected without duplication from multiple target data. Also, the completion of processing for all of the multiple target data is referred to as "1 epoch." For the sake of explanation, we will assume that processing for all of the multiple target data has been completed and proceed to step ST205.

[0068] (Step ST205) After processing all of the multiple target data has been completed, the first learning unit 120 determines whether or not to terminate the iterative learning. This determination may use, for example, a predetermined number of epochs as the termination condition. If it is determined that the iterative learning should not be terminated, the process returns to step ST202. If it is determined that the iterative learning should be terminated, the process proceeds to step ST103.

[0069] (Step ST103) After the first learning process is performed, the first learning unit 120 outputs multiple first feature vectors. Specifically, the first learning unit 120 outputs multiple first feature vectors by inputting multiple target data into the first trained model, which is the first machine learning model whose training has been completed by the first learning process.

[0070] (Step ST104) After the first learning unit 120 outputs multiple first feature vectors, the clustering unit 130 generates a first clustering result by clustering the multiple first feature vectors.

[0071] (Step ST105) After the clustering unit 130 generates the first clustering result, the cluster selection unit 140 selects one or more clusters from among the multiple clusters included in the first clustering result.

[0072] (Step ST106) After the cluster selection unit 140 selects one or more clusters, the second learning unit 150 trains a second machine learning model using multiple target data under second learning conditions. Hereafter, the process in step ST106 will be referred to as the "second learning process." A specific example of the second learning process will be explained below using the flowchart in Figure 9.

[0073] Figure 9 is a flowchart showing a specific example of the second learning process in the flowchart of Figure 3. The flowchart in Figure 9 transitions from step ST105 in the flowchart of Figure 3.

[0074] (Step ST301) After the cluster selection unit 140 selects one or more clusters, the second learning unit 150 sets a second learning condition that includes a second data augmentation condition. As a specific example thereafter, the second data augmentation condition includes "image rotation" and two transformation methods, "scaling" or "black and white inversion". More specifically, as variations of the second data augmentation condition, second data augmentation condition 1 and second data augmentation condition 2 are set. Second data augmentation condition 1 includes two transformation methods, "scaling" and "image rotation". Second data augmentation condition 2 includes two transformation methods, "image rotation" and "black and white inversion".

[0075] (Step ST302) After setting the second learning conditions, the second learning unit 150 calculates a second feature vector based on the target data.

[0076] (Step ST303) After calculating the second feature vector, the second learning unit 150 calculates the loss using the second feature vector.

[0077] (Step ST304) After calculating the loss, the second learning unit 150 updates the second machine learning model using the loss.

[0078] More precisely, "repetitive learning" is performed by repeating the processes from steps ST302 to ST304 above for all of the multiple target data. For the sake of explanation, we will assume that the processing for all of the multiple target data has been completed once, and then proceed to step ST305.

[0079] (Step ST305) After processing all of the multiple target data has been completed, the second learning unit 150 determines whether or not to perform iterative learning. This determination may use, for example, a predetermined number of epochs as a termination condition. If it is determined that iterative learning should not be terminated, the process returns to step ST302. If it is determined that iterative learning should be terminated, the process proceeds to step ST107.

[0080] (Step ST107) After the second learning process is performed, the second learning unit 150 outputs multiple second feature vectors. Specifically, the second learning unit 150 outputs multiple second feature vectors by inputting multiple target data into the second trained model, which is the second machine learning model whose training has been completed by the second learning process.

[0081] (Step ST108) After the second learning unit 150 outputs multiple second feature vectors, the comparison unit 160 generates a second clustering result for the multiple second feature vectors based on the first clustering result and the multiple second feature vectors.

[0082] (Step ST109) After generating the second clustering result, the comparison unit 160 generates a comparison result by comparing the first clustering result and the second clustering result.

[0083] (Step ST110) After the comparison unit 160 generates the comparison result, the display control unit 170 displays the comparison result. The display control unit 170 may also display a scatter plot or the like for at least one of the multiple first feature vectors and the multiple second feature vectors. After step ST110, the processing of the flowchart in Figure 3 is completed.

[0084] The flowcharts described above are just examples. The order of the steps in these flowcharts can be changed as much as possible, and other steps can be added.

[0085] (Specific examples of display) Figure 10 shows a specific example of a display image that includes a scatter plot visualizing multiple first feature vectors in the embodiment. The display image 1000 in Figure 10 includes a scatter plot 1010 and display information 1020.

[0086] Scatter plot 1010 shows multiple first feature vectors generated using the first data augmentation condition, represented by arbitrary first and second components. Scatter plot 1010 also includes the first cluster CL1, the second cluster CL2, and the third cluster CL3.

[0087] Display information 1020 contains information about the scatter plot 1010. Specifically, display information 1020 includes information about the display data (first feature vector), information about the data augmentation conditions (first data augmentation conditions ("scaling", "image rotation", and "black and white inversion")), and information about representative images (representative images for the first cluster, second cluster, and third cluster).

[0088] According to Figure 10, the first data augmentation condition classifies the image in Figure 5 (the image containing the black triangle) into the first cluster CL1, the image in Figure 6 (the image containing the white rectangle) into the second cluster CL2, and the image in Figure 4 (the image containing the black circle) and the image in Figure 7 (the image containing the white circle) into the third cluster CL3.

[0089] Figure 11 shows a first specific example of a display image including a scatter plot visualizing multiple second feature vectors in the embodiment. The display image 1100 in Figure 11 includes a scatter plot 1110 and display information 1120.

[0090] Scatter plot 1110 shows multiple second feature vectors 1 generated using the second data augmentation condition 1, represented by arbitrary first and second components. Scatter plot 1110 also includes the first cluster CL1, the second cluster CL2, and the third cluster CL3.

[0091] Display information 1120 contains information about the scatter plot 1110. Specifically, display information 1120 includes information about the display data (second feature vector 1), information about the data augmentation conditions (second data augmentation condition 1 ("scaling" and "image rotation")), and information about representative images (representative images for the first cluster, second cluster, and third cluster).

[0092] According to Figure 11, under the second data augmentation condition 1, the first cluster CL1 includes the image in Figure 5 (the image containing the black triangle), the second cluster CL2 includes the image in Figure 6 (the image containing the white square), and the third cluster CL3 includes the image in Figure 4 (the image containing the black circle) and the image in Figure 7 (the image containing the white circle). The reason why the types of images included in each cluster are the same as in Figure 10 is that the cluster IDs from the first clustering result are used for the visualization of multiple second feature vectors 1.

[0093] Figure 12 shows a second specific example of a display image including a scatter plot visualizing multiple second feature vectors in the embodiment. The display image 1200 in Figure 12 includes a scatter plot 1210 and display information 1220.

[0094] Scatter plot 1210 shows multiple second feature vectors 2 generated using the second data augmentation condition 2, represented by arbitrary first and second components. Scatter plot 1210 also includes the first cluster CL1, the second cluster CL2, and the third cluster CL3.

[0095] Display information 1220 contains information about the scatter plot 1210. Specifically, display information 1220 includes information about the display data (second feature vector 2), information about the data augmentation conditions (second data augmentation condition 2 ("image rotation" and "black and white inversion")), and information about representative images (representative images for the first cluster, second cluster, and third cluster).

[0096] According to Figure 12, under the second data augmentation condition 2, the first cluster CL1 includes the image in Figure 5 (the image containing the black triangle), the second cluster CL2 includes the image in Figure 6 (the image containing the white rectangle), and the third cluster CL3 includes the image in Figure 4 (the image containing the black circle) and the image in Figure 7 (the image containing the white circle). The reason why the types of images included in each cluster are the same as in Figure 10 is that, as in Figure 11, the cluster IDs from the first clustering result are used for the visualization of multiple second feature vectors 1.

[0097] Focusing on the third cluster CL3, we can see that in scatter plots 1010 in Figure 10 and 1210 in Figure 12, the samples within the cluster are clustered together (i.e., the dispersion is low), whereas in scatter plot 1110 in Figure 11, the samples within the cluster are dispersed (i.e., the dispersion is high). Furthermore, in scatter plot 1110 in Figure 11, it can be confirmed that the third cluster CL3 appears to be divided into two clusters. As can be inferred from the representative image of the third cluster CL3, the image in Figure 4 (the image containing black circles) and the image in Figure 7 (the image containing white circles), which are classified as the third cluster CL3, should ideally be clustered into separate clusters.

[0098] These differences stem from the different types of transformation methods included in the data augmentation conditions. Specifically, the second data augmentation condition 1 is the first data augmentation condition with the "black and white inversion" transformation method removed, and the second data augmentation condition 2 is the first data augmentation condition with the "scaling" transformation method removed. Scatter plots 1010 for the first data augmentation condition and 1210 for the second data augmentation condition show similar dispersion of samples in the third cluster CL3, while scatter plot 1110 for the second data augmentation condition shows a different dispersion of samples in the third cluster CL3 compared to the previous two. In other words, it can be inferred that the "black and white inversion" transformation method included in the data augmentation conditions is the reason why the image in Figure 4 (the image containing black circles) and the image in Figure 7 (the image containing white circles), which should have been clustered separately, were classified into the same cluster.

[0099] In summary, as shown in Figures 10 to 12, the data analysis device 100 can display scatter plots for each of the different data augmentation conditions, along with the types of transformation methods associated with the data augmentation conditions. This allows the user to see the cluster state due to differences in data augmentation conditions as the shape of the clusters.

[0100] Figure 13 shows a first specific example of the comparison results in the embodiment. The comparison result 1300 in Figure 13 includes information on the selected cluster, information on the data augmentation conditions to be compared, the dispersion of the selected cluster corresponding to the data augmentation conditions, and the difference in dispersion. Specifically, the comparison result 1300 shows the dispersion "disp10" for the first data augmentation condition, the dispersion "disp_21" for the second data augmentation condition 1, and the difference in dispersion "diff_1" for the selected third cluster.

[0101] Figure 14 shows a second specific example of the comparison results in the embodiment. The comparison result 1400 in Figure 14, like in Figure 13, includes information on the selected cluster, information on the data augmentation conditions to be compared, the dispersion of the selected cluster corresponding to the data augmentation conditions, and the difference in dispersion. Specifically, the comparison result 1400 shows the dispersion "disp10" for the first data augmentation condition, the dispersion "disp_22" for the second data augmentation condition 2, and the difference in dispersion "diff_2" for the selected third cluster.

[0102] In Figures 13 and 14, if the dispersion "disp_21" is greater than the dispersion "disp_22", the user can infer that, with respect to the shape of the third cluster, the samples are more dispersed under the second data augmentation condition 1 than under the second data augmentation condition 2. In other words, the user can infer that the transformation method present in the second data augmentation condition 1 but not in the second data augmentation condition 2 (in this case, "black and white inversion") has a greater influence on the phenomenon of multiple images being classified into the third cluster.

[0103] Furthermore, if the difference in dispersion "diff_1" is greater than the difference in dispersion "diff_2", the user can infer that the shape of the third cluster is different under the second data augmentation condition 1 than under the second data augmentation condition 2. In other words, the user can infer that the transformation method present in the first data augmentation condition but not in the second data augmentation condition 1 (in this case, "black and white inversion") has a significant impact.

[0104] In summary, as shown in Figures 13 and 14, the data analysis device 100 can display the dispersion of each of the different data augmentation conditions and the difference in dispersion between the different data augmentation conditions as comparison results for the selected cluster (third cluster), which is the cluster to be compared. This allows the user to numerically confirm the state of the cluster due to the differences in data augmentation conditions.

[0105] Figure 15 shows another specific example of a display image including a scatter plot in the embodiment. Display image 1500 in Figure 15 includes a scatter plot 1510.

[0106] Scatter plot 1510 shows multiple first feature vectors generated using the first data augmentation condition, represented by arbitrary first and second components. Scatter plot 1510 also includes the first cluster CL1, the second cluster CL2, and the third cluster CL3.

[0107] Furthermore, scatter plot 1510 shows representative images near each cluster. Specifically, scatter plot 1510 shows representative images I11 and I12, which correspond to the image in Figure 5, near the first cluster CL1; representative images I21 and I22, which correspond to the image in Figure 6, near the second cluster CL2; and representative image I31, which corresponds to the image in Figure 7, and representative image I32, which corresponds to the image in Figure 4, near the third cluster CL3.

[0108] In summary, as shown in Figure 15, the data analysis device 100 can include representative images of each cluster on the scatter plot when displaying an image that includes a scatter plot. This makes it easier for the user to visually identify the clusters shown on the scatter plot and their representative images.

[0109] As described above, the data analysis device according to the embodiment acquires multiple target data, learns a first learning model by unsupervising learning the multiple target data using a first data augmentation condition which is a condition relating to a data augmentation transformation method, generates multiple first feature vectors corresponding to the multiple target data, generates a first clustering result by clustering the multiple first feature vectors, learns a second learning model by unsupervising learning the multiple target data using a second data augmentation condition which has different conditions relating to a transformation method from the first data augmentation condition, generates multiple second feature vectors corresponding to the multiple target data, and generates a comparison result by comparing the multiple first feature vectors and the multiple second feature vectors for each of the multiple clusters based on the first clustering result.

[0110] Therefore, the data analysis device according to the embodiment can estimate the effect of data augmentation on clustering by comparing feature vectors generated from learning models with different data augmentation conditions.

[0111] (Variation 1) The data analysis device according to the above embodiment used image data as the target data, but is not limited to this. For example, any data such as audio data, table data, and sensor data such as acceleration and voltage may be used as the target data.

[0112] (Modification 2) The data analysis device according to the above embodiment uses a DNN as the machine learning model, but is not limited to this. For example, any machine learning model such as linear regression, multiple regression, SVM (Support Vector Machine), and decision trees may be used.

[0113] (Variation 3) The data analysis apparatus according to the above embodiment compared the differences in the sets of transformation methods that constitute the first data augmentation condition and the second data augmentation condition, but is not limited to this. As mentioned above, the first data augmentation condition and the second data augmentation condition are each composed of the same set of transformation methods, and the parameters relating to the degree of transformation associated with one or more of the transformation methods in the set may differ.

[0114] For example, let's consider a case where three data augmentation conditions are set as transformation methods: "scaling," "image rotation," and "black and white inversion." The first data augmentation condition is defined as having a transformation degree associated with "scaling" of "scaling ratio from 0.5x to 2.0x," a transformation degree associated with "image rotation" of "rotation angle from -180 degrees to +180 degrees," and a transformation degree associated with "black and white inversion" of "inversion probability of 50%." The second data augmentation condition is defined as having the same transformation degrees associated with "scaling" and "black and white inversion" as the first data augmentation condition, but with a transformation degree associated with "image rotation" of "rotation angle from -90 degrees to +90 degrees." In other words, the first and second data augmentation conditions are each composed of the sets of "scaling," "image rotation," and "black and white inversion," but with different parameters regarding the transformation degree associated with "image rotation."

[0115] (Modification 4) The data analysis device according to Modification 4 may train a second machine learning model by performing unsupervised learning on multiple target data sets, using the parameters of the first trained model, which is a first machine learning model whose training has been completed, as initial values. This allows the data analysis device according to Modification 4 to shorten the time required to train the second machine learning model.

[0116] (Variation 5) The data analysis device according to Modification 5 may train a second machine learning model by using the parameters of the first trained model as initial values ​​and performing additional training on the selected cluster. Specifically, the data analysis device according to Modification 5 may train a second machine learning model by using the parameters of the first trained model as initial values ​​and performing additional training limited to the target data (samples) included in the selected cluster, or samples around the selected cluster. This allows the data analysis device according to Modification 5 to shorten the time required for analysis by narrowing down the clusters to be processed during training.

[0117] (Experimental variation 6) The data analysis device according to Modification 6 does not need to consider the cluster selection unit. Specifically, the data analysis device according to Modification 6 may have a configuration that excludes the cluster selection unit, or the cluster selection unit may not select any clusters. Not selecting a cluster is equivalent to selecting all clusters. As a result, the data analysis device according to Modification 6 generates comparison results for each cluster.

[0118] (Example 7) The data analysis device according to Modification 7 may specify two different types of target data (for example, image BC-2 in Figure 4 and image WC-3 in Figure 7) and calculate the distance between two feature vectors corresponding to the two specified different types of target data. This allows the data analysis device according to Modification 7 to check the change in the distance between the two feature vectors under different data augmentation conditions.

[0119] (Hardware configuration) Figure 16 is a block diagram illustrating the hardware configuration of a computer according to one embodiment. The computer 1600 includes, as hardware, a CPU (Central Processing Unit) 1610, RAM (Random Access Memory) 1620, program memory 1630, auxiliary storage device 1640, and input / output interface 1650. The CPU 1610 communicates with the RAM 1620, program memory 1630, auxiliary storage device 1640, and input / output interface 1650 via a bus 1660.

[0120] CPU 1610 is an example of a general-purpose processor. RAM 1620 is used by CPU 1610 as working memory. RAM 1620 includes volatile memory such as SDRAM (Synchronous Dynamic Random Access Memory). Program memory 1630 stores various programs, including data analysis programs. For example, ROM (Read-Only Memory), part of auxiliary storage device 1640, or a combination thereof may be used as program memory 1630. Auxiliary storage device 1640 stores data non-temporarily. Auxiliary storage device 1640 includes non-volatile memory such as HDD or SSD.

[0121] The input / output interface 1650 is an interface for connecting to or communicating with other devices. For example, the input / output interface 1650 is used for connecting or communicating between the acquisition unit 110 shown in Figure 1 and external devices (e.g., input / output devices and server devices), and between the display control unit 170 and external devices.

[0122] Each program stored in program memory 1630 contains a computer-executable instruction. When a program (computer-executable instruction) is executed by the CPU 1610, it causes the CPU 1610 to perform a predetermined process. For example, when a data analysis program is executed by the CPU 1610, it causes the CPU 1610 to perform a series of processes described with respect to the parts in Figures 1 and 2.

[0123] The program may be provided to the computer 1600 in a state where it is stored on a computer-readable storage medium. In this case, for example, the computer 1600 further includes a drive (not shown) for reading data from the storage medium and retrieving the program from the storage medium. Examples of storage mediums include magnetic disks, optical disks (CD-ROM, CD-R, DVD-ROM, DVD-R, etc.), magneto-optical disks (MO, etc.), and semiconductor memory. Alternatively, the program may be stored on a server on a communication network, and the computer 1600 may download the program from the server using the input / output interface 1650.

[0124] The processing described in the embodiments is not limited to being performed by a general-purpose hardware processor such as a CPU 1610 executing a program, but may also be performed by a dedicated hardware processor such as an ASIC (Application Specific Integrated Circuit). The term "processing circuit (processing unit)" includes at least one general-purpose hardware processor, at least one dedicated hardware processor, or a combination of at least one general-purpose hardware processor and at least one dedicated hardware processor. In the example shown in Figure 16, the CPU 1610, RAM 1620, and program memory 1630 correspond to the processing circuit.

[0125] Therefore, according to each of the above embodiments, it is possible to estimate the impact of data augmentation on clustering.

[0126] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0127] 100...Data analysis device, 110...Acquisition unit, 120...First learning unit, 130...Clustering unit, 140...Cluster selection unit, 150...Second learning unit, 160...Comparison unit, 170...Display control unit, 210...Feature vector calculation unit, 220...Loss calculation unit, 230...Model update unit, 240...Model storage unit, 1000,1100,1200,1500...Display image, 1010,1110,1210,1510...Scatter plot, 1020,1120,1220...Display information, 1300,1400...Comparison result, 16 00...Computer, 1630...Program memory, 1640...Auxiliary storage device, 1650...Input / output interface, 1660...Bus, BC-1, BC-2, BC-3, BC-n1...Image, BT-1, BT-2, BT-3, BT-n2...Image, CL1...First cluster, CL2...Second cluster, CL3...Third cluster, I11, I12, I21, I22, I31, I32...Representative image, WC-1, WC-2, WC-3, WC-n4, WR-1, WR-2, WR-3, WR-n3...Image.

Claims

1. A data acquisition unit that acquires multiple target data, A first learning unit learns a first learning model by unsupervising learning the plurality of target data using a first data augmentation condition, which is a condition relating to a data augmentation transformation method, and generates a plurality of first feature vectors corresponding to the plurality of target data. A clustering unit that generates a first clustering result by clustering the plurality of first feature vectors, A second learning unit learns a second learning model by unsupervising learning the plurality of target data using a second data augmentation condition which differs from the first data augmentation condition in terms of the conditions relating to the transformation method, and generates a plurality of second feature vectors corresponding to the plurality of target data. A comparison unit generates a comparison result by comparing the plurality of first feature vectors and the plurality of second feature vectors for each of the plurality of clusters based on the first clustering result. A data analysis device equipped with the following features.

2. A selection unit that selects one or more clusters from the plurality of clusters included in the first clustering result, and less than the number of clusters. It further comprises, The comparison unit generates the comparison results for each of the selected clusters. The data analysis apparatus according to claim 1.

3. The set of transformation methods that constitute the second data augmentation condition is a subset of the set of transformation methods that constitute the first data augmentation condition. The data analysis apparatus according to claim 1.

4. The first data augmentation condition and the second data augmentation condition each consist of the same set of transformation methods, and the parameters relating to the degree of transformation associated with one or more of the transformation methods in the set are different. The data analysis apparatus according to claim 1.

5. The comparison unit calculates the first dispersion of the plurality of first feature vectors included in the cluster to be compared and the second dispersion of the plurality of second feature vectors included in the cluster to be compared, and generates the comparison result including the first dispersion and the second dispersion. The data analysis apparatus according to claim 1.

6. The comparison unit calculates the difference between the first dispersion degree and the second dispersion degree, and generates the comparison result which further includes the difference in dispersion degrees. The data analysis apparatus according to claim 5.

7. The comparison unit calculates the difference between the first dispersion of the plurality of first feature vectors included in the cluster to be compared and the second dispersion of the plurality of second feature vectors included in the cluster to be compared, and generates the comparison result including the difference in dispersion. The data analysis apparatus according to claim 1.

8. Display control unit that displays the comparison results It further possesses, A data analysis apparatus according to any one of claims 1 to 7.

9. A display control unit displays a display image that includes a scatter plot in which at least one of the plurality of first feature vectors and the plurality of second feature vectors is represented by a plurality of different components, and each point of the feature vectors is grouped into clusters based on the first clustering result. It further possesses, The data analysis apparatus according to claim 1.

10. The aforementioned display image further includes display information relating to the scatter plot, The display information is at least one of the following: the type of display data included in the scatter plot, the type of transformation method included in the data augmentation conditions, and the representative image of each cluster. The data analysis apparatus according to claim 9.

11. The comparison unit calculates the first dispersion of the plurality of first feature vectors included in the cluster to be compared and the second dispersion of the plurality of second feature vectors included in the cluster to be compared, and generates the comparison result including the first dispersion and the second dispersion. The display control unit displays the display image and the comparison result. The data analysis apparatus according to claim 9.

12. The comparison unit calculates the difference between the first dispersion degree and the second dispersion degree, and generates the comparison result which further includes the difference in dispersion degrees. The data analysis apparatus according to claim 11.

13. The comparison unit calculates the difference between the first dispersion of the plurality of first feature vectors included in the cluster to be compared and the second dispersion of the plurality of second feature vectors included in the cluster to be compared, and generates the comparison result including the difference in dispersion. The display control unit displays the display image and the comparison result. The data analysis apparatus according to claim 9.

14. The aforementioned display image includes a representative image of each cluster on the scatter plot. The data analysis apparatus according to claim 9.

15. The second learning unit learns the second learning model by performing unsupervised learning on the multiple target data, using the parameters of the first learning model, which has completed learning, as initial values. The data analysis apparatus according to claim 1.

16. The second learning unit learns the second learning model by using the parameters of the first learning model, which has completed learning, as initial values, and by performing additional learning on one or more selected clusters. The data analysis apparatus according to claim 2.

17. The first learning unit outputs the plurality of first feature vectors by inputting the plurality of target data to the first learning model after learning is complete. The second learning unit outputs the plurality of second feature vectors by inputting the plurality of target data to the second learning model after the learning is complete. The data analysis apparatus according to claim 1.

18. Computers Acquiring multiple target data, Using a first data augmentation condition, which is a condition relating to the data augmentation transformation method, a first learning model is trained by unsupervised learning of the multiple target data, and a plurality of first feature vectors corresponding to the multiple target data are generated. The process involves generating a first clustering result by clustering the aforementioned plurality of first feature vectors, The process involves training a second learning model by unsupervised learning of the multiple target data using a second data augmentation condition that differs from the first data augmentation condition in terms of the conditions relating to the transformation method, thereby generating multiple second feature vectors corresponding to the multiple target data. Based on the first clustering result, a comparison result is generated by comparing the plurality of first feature vectors and the plurality of second feature vectors for each of the plurality of clusters. A data analysis method that includes the following features.

19. Computer A means of acquiring multiple target data, A means for learning a first learning model by unsupervising learning the plurality of target data using a first data augmentation condition, which is a condition relating to a data augmentation transformation method, and for generating a plurality of first feature vectors corresponding to the plurality of target data, A means for generating a first clustering result by clustering the plurality of first feature vectors, A means for learning a second learning model by unsupervising learning the plurality of target data using a second data augmentation condition which differs from the first data augmentation condition in terms of the conditions relating to the transformation method, and for generating a plurality of second feature vectors corresponding to the plurality of target data, Means for generating comparison results by comparing the plurality of first feature vectors and the plurality of second feature vectors for each of the plurality of clusters based on the first clustering result. A data analysis program that functions as such.

Images