Data analysis device, method, and program

Abstract

To accurately compare features of a plurality of pieces of data acquired under different conditions.SOLUTION: A data analysis device according to an embodiment includes: a first learning unit configured to generate a plurality of first feature vectors by performing unsupervised learning on a plurality of pieces of first data satisfying a first condition; a first clustering unit configured to generate a first clustering result by clustering the plurality of first feature vectors; a second learning unit configured to generate a plurality of second feature vectors by performing unsupervised learning on at least some of the plurality of pieces of first data and a plurality of pieces of second data satisfying a second condition different from the first condition; a second clustering unit configured to generate a second clustering result by clustering the plurality of second feature vectors; and a comparison unit configured to generate a comparison result related to the plurality of pieces of first data and the plurality of pieces of second data, by comparing the first clustering result and the second clustering result.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Embodiments of the present invention relate to a data analysis apparatus, method, and program.

Background Art

[0002] In the manufacturing field, efforts are underway to monitor the occurrence of defects and improve productivity by classifying inspection images of products through machine learning. As a method for classifying inspection images using machine learning, supervised learning is known, in which a teacher label serving as a classification criterion is manually assigned in advance, and a classification model is learned by a method such as deep learning. However, in order to learn a highly accurate classification model through supervised learning, it is necessary to accurately assign teacher labels to a large number of images.

[0003] As another method for classifying inspection images using machine learning, an unsupervised learning method that performs classification using the similarity or distance between images is known. Since the unsupervised learning method does not require manual labeling, it is possible to classify a large number of unknown images, and an overview of the image dataset can be grasped from the classification results. In recent years, unsupervised learning methods using deep learning have been proposed, and the performance has been greatly improved by automatically learning the features of images and performing classification using their similarity or distance.

[0004] Thus, in the manufacturing field, efforts are underway to grasp the occurrence status of defects and reduce the analysis work cost by classifying inspection images using the unsupervised learning method. For example, by classifying a large number of inspection images accumulated in a factory using the unsupervised learning method, the number of defective or defective images can be confirmed, and it is possible to know what types of defects and how many are occurring. In addition, since the unsupervised learning does not require manual work, the user can classify the inspection images at the necessary timing, and it can also be used to classify and check the inspection images of the previous day at the start of business.

[0005] In particular, at the factory, more detailed analysis is carried out by comparing the occurrence status of defects and deficiencies for each manufacturing condition. For example, by comparing the occurrence of defects before and after equipment maintenance, it is possible to confirm the effectiveness of maintenance and the presence or absence of adverse effects caused by maintenance. In addition, by comparing the occurrence of defects in two inspection devices that perform the same inspection, it is possible to confirm the inspection accuracy of each inspection device and the habits of taking pictures associated with the inspection. By grasping the defects specific to the manufacturing conditions in this way, the cause of the defects becomes clear and it becomes possible to implement countermeasures early.

[0006] Conventionally, a technique of performing unsupervised learning using existing data and new data assuming time-series data is known. This technique reduces the work cost of teacher labeling by using the classification results of unsupervised learning of existing data to batch-assign teacher labels to the existing data in the support for labeling time-series data with a teacher. In addition, this technique can assign a teacher label to new data by performing unsupervised learning by mixing existing data and new data and using the teacher label of the existing data classified into the same group. In addition, this technique can also assign a teacher label to time-series data of a newly emerging trend by checking a group containing only new data. In these cases, by considering the existing data and the new data as data under different manufacturing conditions, it becomes possible to reduce the analysis cost of the time-series data.

[0007] On the other hand, in the above technique, the time-series data acquired from sensors and the like is the target, and the difference in characteristics between the existing data and the new data is not considered. In a factory, completely different defects and deficiencies may occur due to differences in manufacturing conditions, and in order to classify the inspection images taken of them with high accuracy, it is necessary to learn features suitable for each. If a new inspection image is classified using features suitable for the existing image, it may not be possible to classify the defects and deficiencies that occur only in the new inspection image, and there is a possibility of overlooking them.

[0008] Moreover, the above technique aims to perform supervised learning by assigning teacher labels to new data and is not suitable for methods of analysis that compare existing data with new data.

[0009] In a factory, a large number of devices with different properties are operating, and the state of the devices is constantly changing, such as maintenance, manufacturing process changes, and the launch of new products. Therefore, a method is desired that can learn the characteristics of defects and deficiencies corresponding to differences in manufacturing conditions such as differences in devices and states, compare them with high precision, and assist in analysis.

Prior Art Documents

Patent Documents

[0010]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0011] The problem to be solved by the present invention is to provide a data analysis device, method, and program that can accurately compare the characteristics of a plurality of data obtained under different conditions.

Means for Solving the Problems

[0012] A data analysis apparatus according to an embodiment includes a first acquisition unit, a first learning unit, a first clustering unit, a second acquisition unit, a second learning unit, a second clustering unit, and a comparison unit. The first acquisition unit acquires a plurality of first data that satisfy a first condition. The first learning unit generates a plurality of first feature vectors by performing unsupervised learning on the plurality of first data. The first clustering unit generates a first clustering result by clustering the plurality of first feature vectors. The second acquisition unit acquires a plurality of second data that satisfy a second condition different from the first condition. The second learning unit generates a plurality of second feature vectors by performing unsupervised learning on at least some of the plurality of first data and the plurality of second data. The second clustering unit generates a second clustering result by clustering the plurality of second feature vectors. The comparison unit generates a comparison result regarding the plurality of first data and the plurality of second data by comparing the first clustering result and the second clustering result.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Embodiments for Carrying Out the Invention

[0014] Hereinafter, embodiments of a data analysis apparatus will be described in detail with reference to the drawings.

[0015] (Embodiment) FIG. 1 is a block diagram illustrating the configuration of a data analysis system including a data analysis apparatus according to an embodiment. The data analysis system 1 in FIG. 1 includes a data analysis apparatus 100, one or more data storage units, and an output device 120. In FIG. 1, as the one or more data storage units, a first data storage unit 111 and a second data storage unit 112 are illustrated. These two data storage units store data satisfying different conditions. This condition is, for example, different devices themselves if there are a plurality of devices performing the same inspection, and if one device is used, it is before and after the date and time, before and after maintenance, and before and after changes in manufacturing conditions (manufacturing process). In other words, the type of condition is any one of different devices, date and time, before and after maintenance, and before and after changes in the manufacturing process. The data analysis apparatus 100 analyzes the differences between different conditions based on a plurality of data satisfying different conditions. The output device 120 displays display data based on the analysis result.

[0016] In the present embodiment, as a specific example of data, an inspection image (hereinafter, may also be simply referred to as an image) taken in the inspection of a manufacturing process will be used for explanation. This inspection image is, for example, an image having a defect detected in the appearance inspection of a semiconductor product. Also, as a specific example of the condition, before and after maintenance is assumed.

[0017] The first data storage unit 111 stores a plurality of first data satisfying the first condition. The first condition is, for example, before maintenance. Specifically, the first data storage unit 111 stores a plurality of images before maintenance.

[0018] The second data storage unit 112 stores a plurality of second data satisfying a second condition different from the first condition. The second condition is, for example, after maintenance. Specifically, the second data storage unit 112 stores a plurality of images after maintenance.

[0019] The output device 120 is, for example, a monitor. The output device 120 receives display data from the data analysis device 100. The output device 120 displays the display data. Note that the output device 120 is not limited to a monitor as long as it can display the display data. For example, the output device 120 may be a projector and a printer. Also, the output device 120 may include a speaker.

[0020] Note that the data analysis device 100 may include at least any one of the first data storage unit 111, the second data storage unit 112, and the output device 120. Also, the first data storage unit 111 and the second data storage unit 112 may be configured by separate storage devices, or may be configured by one storage device.

[0021] FIG. 2 is a block diagram illustrating the configuration of the data analysis device according to the embodiment. The data analysis device 100 in FIG. 2 includes a first acquisition unit 210, a first learning unit 220, a first clustering unit 230, a second acquisition unit 240, a second learning unit 250, a second clustering unit 260, a comparison unit 270, and a display control unit 280.

[0022] The first acquisition unit 210 acquires a plurality of first data that satisfy the first condition. For example, the first acquisition unit 210 acquires a plurality of first data from the first data storage unit 111. The first acquisition unit 210 outputs the plurality of first data to the first learning unit 220, and outputs at least some of the plurality of first data to the second learning unit 250.

[0023] The number of at least some of the plurality of first data output from the first acquisition unit 210 to the second learning unit 250 may be based on the clustering result (clustering result) by the first clustering unit 230 described later. For example, it is desirable that the number of some data maintains the ratio of the first data included in each of the plurality of clusters generated by the clustering result. By doing so, it becomes easier for the user to compare with a plurality of second data that satisfy the second condition, and it also becomes easier to discover new features.

[0024] The first learning unit 220 receives a plurality of first data from the first acquisition unit 210. The first learning unit 220 iteratively learns a learning model by performing unsupervised learning on the plurality of first data. Also, the first learning unit 220 outputs a plurality of first feature vectors by inputting the plurality of first data to the learning model. In other words, the first learning unit 220 generates a plurality of first feature vectors by performing unsupervised learning on the plurality of first data. The first learning unit 220 outputs the plurality of first feature vectors to the first clustering unit 230. Hereinafter, the specific configuration of the first learning unit 220 will be described with reference to FIG. 3.

[0025] FIG. 3 is a block diagram illustrating the specific configuration of the first learning unit in FIG. 2. The first learning unit 220 in FIG. 3 includes a feature vector calculation unit 310, a loss calculation unit 320, a model update unit 330, and a model storage unit 340. Regarding each of these units, hereinafter, description will be made with respect to one piece of first data among the plurality of first data.

[0026] The feature vector calculation unit 310 calculates a first feature vector based on the first data. Specifically, the feature vector calculation unit 310 outputs (calculates) a first feature vector by inputting the first data to the learning model stored in the model storage unit 340. The feature vector calculation unit 310 outputs the first feature vector to the loss calculation unit 320.

[0027] In the present embodiment, as the learning model used for calculating the feature vector, a deep neural network model (DNN: Deep Neural Network) that outputs a feature vector by inputting an image is used. Regarding this DNN, the model structure and structure parameters are set in advance according to the learning conditions.

[0028] Also, the feature vector calculation unit 310 may output the feature vector output from the output layer of the DNN as the first feature vector, or may output the feature vector output from an intermediate layer before the output layer as the first feature vector.

[0029] The loss calculation unit 320 receives the first feature vector from the feature vector calculation unit 310. The loss calculation unit 320 calculates a first loss using the first feature vector. The loss calculation unit 320 outputs the first loss to the model update unit 330.

[0030] In the loss calculation unit 320, in order to output a feature vector suitable for clustering, for example, the method (IDFD) described in the reference document (Yaling Tao, Kentaro Takagi, Kouta Nakata, “Clustering-friendly Representation Learning via Instance Discrimination And Feature Decorrelation”, arXiv:2106.00131, ICLR2021.) is used to calculate the first loss. This IDFD combines a method called Instance Discrimination (ID) and a method called Feature Decorrelation (FD) as a loss function.

[0031] Specifically, ID is a method in which, for example, the greater the error between feature vectors obtained from different images, the smaller the loss, and it has a first temperature parameter for controlling the sensitivity of this error. FD is a method in which, for example, the lower the correlation between elements of different feature vectors, the smaller the loss, and it has a second temperature parameter for controlling the sensitivity of this correlation. When calculating the combined loss (combined loss) based on the loss by ID and the loss by FD, the IDFD that combines these methods has a balancing parameter for adjusting the influence degree of each loss.

[0032] Generally speaking, by using IDFD, the loss calculation unit 320 can learn a learning model that extracts feature quantities such that the distance between similar images becomes closer and the distance between dissimilar images becomes farther for images.

[0033] The model update unit 330 receives the first loss from the loss calculation unit 320. The model update unit 330 updates the learning model using the first loss. Specifically, the model update unit 330 applies the optimization parameters based on the first loss to the learning model to update the internal parameters of the learning model. The model update unit 330 outputs the parameters of the updated learning model to the model storage unit 340.

[0034] The model storage unit 340 receives the parameters of the learning model from the model update unit 330. The model storage unit 340 updates and stores the learning model based on the parameters.

[0035] As described above, the first learning unit 220 can perform unsupervised learning using a model that extracts feature amounts such that the distances between similar images become closer and the distances between dissimilar images become farther.

[0036] The first clustering unit 230 receives a plurality of first feature vectors from the first learning unit 220. The first clustering unit 230 generates a first clustering result by clustering the plurality of first feature vectors. The first clustering unit 230 outputs the first clustering result to the comparison unit 270.

[0037] As a clustering method, for example, the K-Means method (K-Means clustering) is used. The first clustering result includes a first cluster number that is the ID of the first cluster to which the first data corresponding to the first feature vector belongs. That is, by performing clustering in the first clustering unit 230, a first cluster number is assigned to each of the plurality of first data. Therefore, the first clustering result includes, for example, data in which the first feature vector is associated with the first cluster number for distinguishing each cluster.

[0038] In addition, the first clustering unit 230 may assign a first cluster label corresponding to the first cluster number. The assignment of the first cluster label includes manual assignment and assignment using machine learning. In manual assignment, the user checks the data (images) included in the cluster and assigns, for each cluster, a first cluster label indicating, for example, the characteristics of the image. In assignment using machine learning, the images included in the cluster are analyzed, and a first cluster label indicating the characteristics of the image is automatically assigned. Note that in the case of manual assignment of the first cluster label, the number of clusters (cluster number) is extremely small compared to the number of images input to the data analysis device, and the burden on the user is small.

[0039] The second acquisition unit 240 acquires a plurality of second data that satisfy a second condition different from the first condition. For example, the second acquisition unit 240 acquires a plurality of second data from the second data storage unit 112. The second acquisition unit 240 outputs the plurality of second data to the second learning unit 250.

[0040] The second learning unit 250 receives at least some of the plurality of first data from the first acquisition unit 210 and receives a plurality of second data from the second acquisition unit 240. The second learning unit 250 generates a plurality of second feature vectors by performing unsupervised learning on at least some of the plurality of first data and the plurality of second data. The second learning unit 250 outputs the plurality of second feature vectors to the second clustering unit 260. Note that since the specific configuration and processing of the second learning unit 250 are the same as those of the first learning unit 220, the description thereof is omitted.

[0041] The second clustering unit 260 receives a plurality of second feature vectors from the second learning unit 250. The second clustering unit 260 generates a second clustering result by clustering the plurality of second feature vectors. The second clustering unit 260 outputs the second clustering result to the comparison unit 270.

[0042] As a clustering method, for example, the K-Means method (K-Means clustering) is used. The second clustering result includes a second cluster number which is the ID of the second cluster to which the first and second data corresponding to the second feature vector belong. That is, by performing clustering in the second clustering unit 260, a second cluster number is assigned to each of the plurality of first and second data. Therefore, the second clustering result includes, for example, data in which the second feature vector is associated with the second cluster number for distinguishing each cluster.

[0043] Also, the second clustering unit 260 may perform clustering so that the number of clusters included in the second clustering result is larger than the number of clusters included in the first clustering result by the first clustering unit 230. For example, in the second data, when features clearly different from those of the first data are found, it is expected that the number of clusters will increase, so it is considered that comparison will be easier.

[0044] The comparison unit 270 receives the first clustering result from the first clustering unit 230 and receives the second clustering result from the second clustering unit 260. The comparison unit 270 generates a comparison result regarding the plurality of first data and the plurality of second data by comparing the first clustering result and the second clustering result. The comparison result is, for example, a graph showing the number or ratio of the first data and the second data included in the clusters in the second clustering result. The comparison unit 270 outputs the comparison result to the display control unit 280.

[0045] The display control unit 280 receives the comparison result from the comparison unit 270. The display control unit 280 generates display data based on the comparison result and causes it to be displayed on the display which is the output device 120.

[0046] The configuration of the data analysis system 1 and the data analysis apparatus 100 according to the embodiment has been described above. Next, the operation of the data analysis apparatus 100 will be described using the flowchart of FIG. 4.

[0047] Figure 4 is a flowchart illustrating the operation of the data analysis apparatus according to the embodiment. The processing of the flowchart in Figure 4 starts when a data analysis program is executed by the user.

[0048] (Step ST110) When the data analysis program is executed by the data analysis apparatus 100, the first acquisition unit 210 acquires a plurality of first data from the first data storage unit 111. Hereinafter, the plurality of first data will be described with reference to Figure 5.

[0049] Figure 5 is a diagram illustrating a plurality of first data in the embodiment. In Figure 5, as the plurality of first data, Image 1-1, Image 1-2, …, Image 1-N are shown. N is the total number of the first data. Image 1-1 and Image 1-2 include polygonal defects (hereinafter referred to as corner defects), and Image 1-N includes circular defects (hereinafter referred to as round defects).

[0050] (Step ST120) After the first acquisition unit 210 acquires a plurality of first data, the first learning unit 220 generates a plurality of first feature vectors by performing unsupervised learning on the plurality of first data. Hereinafter, the processing of step ST120 will be referred to as "first feature vector generation processing". A specific example of the first feature vector generation processing will be described with reference to the flowchart of Figure 6.

[0051] Figure 6 is a flowchart illustrating the first feature vector generation processing of Figure 4. The flowchart of Figure 6 transitions from step ST110 of Figure 4.

[0052] (Step ST121) After the first acquisition unit 210 acquires a plurality of first data, the feature vector calculation unit 310 calculates a first feature vector based on the first data.

[0053] (Step ST122) After the feature vector calculation unit 310 calculates the first feature vector, the loss calculation unit 320 calculates the first loss using the first feature vector.

[0054] (Step ST123) After the loss calculation unit 320 calculates the first loss, the model update unit 330 updates the learning model using the first loss.

[0055] Precisely, for all of the plurality of first data, "iterative learning" is performed by repeating the processing from step ST121 to step ST123. One round of processing for all of these plurality of first data is referred to as "1 epoch".

[0056] (Step ST124) After one round of processing for all of the plurality of first data, the first learning unit 220 determines whether to end the iterative learning. This determination may use, for example, a pre-determined number of epochs as an end condition. If it is determined not to end the iterative learning, the process returns to step ST121. If it is determined to end the iterative learning, the first learning unit 220 outputs (generates) a plurality of first vectors, and the process proceeds to step ST130. Hereinafter, the first feature vector generation process will be described with reference to FIG. 7, and a table associating the first data and the first feature vector will be described with reference to FIG. 8.

[0057] FIG. 7 is a diagram for explaining the first feature vector generation process in the embodiment. The first learning unit 220 outputs a first feature vector 700 by inputting Image 1-1 to the learning model. The first feature vector 700 is, for example, vector data having a 128-dimensional feature amount output from the output layer of the DNN in the learning model. Specifically, the first feature vector 700 is (x 1 1-1 , x 2 1-1 , …, x 128 1-1) is represented as such. At this time, the subscript of each element x represents the element number, and the superscript of each element x represents the image number.

[0058] FIG. 8 is a table associating the first data and the first feature vectors in the embodiment. The table 800 in FIG. 8 associates a first feature vector with each of the images 1-1 to 1-N as a plurality of first data as a result of the first vector generation process.

[0059] In the table 800, for example, the image "1-1" and the first feature vector "(x 1 1-1 , x 2 1-1 , …, x 128 1-1 )" are associated, the image "1-2" and the first feature vector "(x 1 1-2 , x 2 1-2 , …, x 128 1-2 )" are associated, and the image "1-N" and the first feature vector "(x 1 1-N , x 2 1-N , …, x 128 1-N )" are associated. For convenience of explanation, a plurality of images and a plurality of first feature vectors are represented in a table, but at least the image (first data) and the first feature vector need to be associated. This is the same hereinafter.

[0060] (Step ST130) After the first learning unit 220 generates a plurality of first feature vectors, the first clustering unit 230 generates a first clustering result by clustering the plurality of first feature vectors. The first clustering result includes a first cluster number. Further, the first clustering result may include a first cluster label corresponding to the first cluster number. Hereinafter, the first clustering result will be described with reference to FIG. 9.

[0061] FIG. 9 is a table associating first data with first cluster numbers in the embodiment. In table 900 of FIG. 9, first cluster numbers are respectively associated with images 1-1 to 1-N as a plurality of first data.

[0062] Specifically, in table 900, for example, the image "1-1" is associated with the first cluster number "cl1", the image "1-2" is associated with the first cluster number "cl1", and the image "1-N" is associated with the first cluster number "cl2".

[0063] Note that in step ST130, the user may assign a first cluster label corresponding to the first cluster number. Hereinafter, an example in which a first cluster label corresponding to the first cluster number is assigned will be described with reference to FIG. 10.

[0064] FIG. 10 is a table associating first data with first cluster numbers and first cluster labels in the embodiment. Table 1000 of FIG. 10 is obtained by adding an item of the first cluster label to table 900 of FIG. 9. In table 1000, for example, the image "1-1", the first cluster number "cl1", and the first cluster label "corner defect" are associated, the image "1-2", the first cluster number "cl1", and the first cluster label "corner defect" are associated, and the image "1-N", the first cluster number "cl2", and the first cluster label "round defect" are associated. Hereinafter, it is assumed that a first cluster label is assigned to the first data.

[0065] (Step ST140) After the first clustering unit 230 generates a first clustering result, the second acquisition unit 240 acquires at least a part of the plurality of first data from the first data storage unit 111 and acquires a plurality of second data from the second data storage unit 112. Hereinafter, the plurality of second data will be described with reference to FIG. 11.

[0066] FIG. 11 is a diagram illustrating a plurality of second data in the embodiment. In FIG. 11, as the plurality of second data, images 2-1, 2-2, …, 2-M are shown. M is the total number of the second data. Images 2-1 and 2-2 include corner defects, and image 1-M includes a new defect that is neither a corner defect nor a round defect.

[0067] Hereinafter, in step ST140, it is assumed that the second acquisition unit 240 has acquired all the data of the plurality of first data from the first data storage unit 111. Also, it is assumed that the total number N of the plurality of first data is the same as the total number M of the plurality of second data.

[0068] (Step ST150) After the second acquisition unit 240 has acquired the plurality of first and second data, the second learning unit 250 generates a plurality of second feature vectors by performing unsupervised learning on the plurality of first and second data. Hereinafter, the process of step ST150 will be referred to as the “second feature vector generation process”. Since a specific example of the second feature vector generation process is the same as that of the first feature vector generation process, the description thereof will be omitted. Hereinafter, the first feature vector generation process will be described with reference to FIG. 12, and a table associating the second data with the second feature vectors will be described with reference to FIG. 13.

[0069] FIG. 12 is a diagram for explaining the second feature vector generation process in the embodiment. The second learning unit 250 outputs a second feature vector 1200 by inputting image 2-1 to the learning model. The second feature vector 1200 is, for example, vector data having a 128-dimensional feature amount output from the output layer of the DNN in the learning model. Specifically, the second feature vector 1200 is represented as (X 1 2-1 , X 2 2-1 , …, X 128 2-1 ). At this time, the subscript of each element X represents the element number, and the superscript of each element X represents the image number.

[0070] FIG. 13 is a table associating the first and second data with the second feature vectors in the embodiment. Table 1300 in FIG. 13 shows, for a plurality of first and second data, namely images 1-1 to 1-N and images 2-1 to 2-M, the second feature vectors respectively associated with them as the processing results of the second vector generation process.

[0071] In table 1300, for example, the image "1-1" is associated with the second feature vector "(X 1 1-1 ,X 2 1-1 ,…,X 128 1-1 )", the image "1-2" is associated with the second feature vector "(X 1 1-2 ,X 2 1-2 ,…,X 128 1-2 )", the image "1-N" is associated with the second feature vector "(X 1 1-N ,X 2 1-N ,…,X 128 1-N )", the image "2-1" is associated with the second feature vector "(X 1 2-1 ,X 2 2-1 ,…,X 128 2-1 )", the image "2-2" is associated with the second feature vector "(X 1 2-2 ,X 2 2-2 ,…,X 128 2-2 )", and the image "2-M" is associated with the second feature vector "(X 1 2-M ,X 2 2-M ,…,X 128 2-M )".

[0072] (Step ST160) After the second learning unit 250 generates a plurality of second feature vectors, the second clustering unit 260 generates a second clustering result by clustering the plurality of second feature vectors. The second clustering result includes a second cluster number. Hereinafter, the second clustering result will be described with reference to FIG. 14.

[0073] FIG. 14 is a table associating the first and second data with the second cluster number in the embodiment. In the table 1400 of FIG. 14, for images 1-1 to 1-N and images 2-1 to 2-M as a plurality of first and second data, the second cluster numbers are respectively associated therewith.

[0074] Specifically, in the table 1400, for example, the image "1-1" is associated with the second cluster number "CL1", the image "1-2" is associated with the second cluster number "CL1", the image "1-N" is associated with the second cluster number "CL2", the image "2-1" is associated with the second cluster number "CL1", the image "2-2" is associated with the second cluster number "CL1", and the image "2-M" is associated with the second cluster number "CL3".

[0075] (Step ST170) After the second clustering unit 260 generates the second clustering result, the comparison unit 270 generates a comparison result by comparing the first clustering result and the second clustering result. Specifically, the comparison unit 270 generates, as the comparison result, a graph showing the ratio of the first data and the second data included in the clusters in the second clustering result. Hereinafter, examples of the comparison result will be described with reference to FIGS. 15 to 17.

[0076] FIG. 15 is a diagram illustrating a comparison result regarding cluster CL1 in the embodiment. The comparison result 1500 in FIG. 15 visualizes the ratios of the first data and the second data belonging to the second cluster number CL1 in a pie chart. The pie chart of the comparison result 1500 is composed of, for example, three pieces of data: the first data labeled with angular defects, the first data labeled with round defects, and the second data. The ratios of these three pieces of data are, for example, 45:5:50.

[0077] According to the pie chart of the comparison result 1500, it can be seen that the ratios of the first data and the second data are about the same. Also, focusing only on the first data, it can be seen that the proportion of the first data labeled with angular defects is large. From these facts, it is considered that the second data is also classified similarly to the first data, that is, it is considered to contain many characteristics of angular defects. Therefore, the second cluster number CL1 is considered to represent the characteristics of angular defects.

[0078] FIG. 16 is a diagram illustrating a comparison result regarding cluster CL2 in the embodiment. The comparison result 1600 in FIG. 16 visualizes the ratios of the first data and the second data belonging to the second cluster number CL2 in a pie chart. The pie chart of the comparison result 1600 is composed of, for example, three pieces of data: the first data having angular defects, the first data having round defects, and the second data. The ratios of these three pieces of data are, for example, 5:90:5.

[0079] According to the pie chart of the comparison result 1600, it can be seen that the ratio of the second data is extremely smaller than that of the first data. Also, focusing only on the first data, it can be seen that the proportion of the first data labeled with round defects is large. From these facts, it is considered that the second cluster number CL2 represents the characteristics of round defects and that the occurrence of round defects is greatly reduced in the second data.

[0080] FIG. 17 is a diagram illustrating a comparison result regarding cluster CL3 in the embodiment. The comparison result 1700 in FIG. 17 visualizes, in a pie chart, the ratios of the first data and the second data belonging to the second cluster number CL3. The pie chart of the comparison result 1700 is composed of, for example, three types of data: first data having angular defects, first data having round defects, and second data. The ratios of these three types of data are, for example, 5:5:90.

[0081] From the pie chart of the comparison result 1700, it can be seen that the ratio of the second data is extremely larger than that of the first data. From this, it is considered that the second cluster number CL3 represents a new feature (new defect) that is neither an angular defect nor a round defect, and it is considered that new defects are occurring in the second data.

[0082] (Step ST180) After the comparison unit 270 generates the comparison result, the display control unit 280 displays the comparison result. Specifically, the display control unit 280 generates display data based on the comparison result and causes it to be displayed on the display which is the output device 120. After step ST180, the processing of the flowchart in FIG. 4 ends.

[0083] As described above, the data analysis apparatus according to the embodiment acquires a plurality of first data satisfying a first condition, generates a plurality of first feature vectors by performing unsupervised learning on the plurality of first data, generates a first clustering result by clustering the plurality of first feature vectors, acquires a plurality of second data satisfying a second condition different from the first condition, generates a plurality of second feature vectors by performing unsupervised learning on at least some of the plurality of first data and the plurality of second data, generates a second clustering result by clustering the plurality of second feature vectors, and generates a comparison result regarding the plurality of first data and the plurality of second data by comparing the first clustering result and the second clustering result.

[0084] Therefore, the data analysis apparatus according to the embodiment can extract data (e.g., images) having unique features under different conditions, and thus can accurately compare the features of a plurality of data acquired under different conditions.

[0085] (Other specific examples of the embodiment) In the above embodiment, as a specific example, the use of inspection images has been described, but it is not limited thereto. In other specific examples, for example, the use of images of surveillance cameras (surveillance images) will be described. The first condition is, for example, before a predetermined time, and the second condition is, for example, after the predetermined time.

[0086] Specifically, the first data storage unit 111 stores first data satisfying a time length before a predetermined time. More specifically, the first data storage unit 111 stores a plurality of first data (a plurality of first surveillance images) captured during a period from 5 minutes before to 10 minutes before. Similarly, the second data storage unit 112 stores second data satisfying a time length that is before the current time and after the above-mentioned predetermined time. More specifically, the second data storage unit 112 stores a plurality of second data (a plurality of second surveillance images) captured during a period from 5 minutes before the current time.

[0087] Since the processing by the data analysis apparatus 100 is the same as that in the above embodiment, the description thereof is omitted. In other specific examples, when no change in the surveillance image is regarded as normal, the learning using a plurality of first surveillance images and the clustering using a plurality of first feature vectors corresponding to the plurality of first surveillance images result in substantially constant output results. Here, when there is a change in the surveillance image, that is, when an abnormality occurs in the surveillance image, the first surveillance image and the second surveillance image have different features, so the above learning and the above clustering result in output results different from those in the normal state. The comparison of such output results will be described with reference to FIG. 18.

[0088] FIG. 18 is a diagram illustrating a comparison result reflecting features including data under different conditions in another specific example of the embodiment. The comparison result 1800 in FIG. 18 includes a scatter diagram representing data obtained during normal times and data obtained during abnormal times with different first components and second components, respectively. That is, the comparison result in another specific example is a correlation diagram representing a plurality of first feature vectors and a plurality of second feature vectors with different components. The scatter diagram of the comparison result 1800 shows that different clusters are formed during normal times and abnormal times. Note that the data analysis apparatus 100 may notify the user that different clusters are formed, for example, when different clusters are formed.

[0089] As described above, this data analysis apparatus can be applied not only to inspection images but also to monitoring images.

[0090] (Hardware Configuration) FIG. 19 is a block diagram illustrating the hardware configuration of a computer according to an embodiment. The computer 1900 includes, as hardware, a CPU (Central Processing Unit) 1910, a RAM (Random Access Memory) 1920, a program memory 1930, an auxiliary storage device 1940, and an input / output interface 1950. The CPU 1910 communicates with the RAM 1920, the program memory 1930, the auxiliary storage device 1940, and the input / output interface 1950 via a bus 1960.

[0091] The CPU 1910 is an example of a general-purpose processor. The RAM 1920 is used by the CPU 1910 as a working memory. The RAM 1920 includes volatile memories such as SDRAM (Synchronous Dynamic Random Access Memory). The program memory 1930 stores various programs including a data analysis program. As the program memory 1930, for example, a ROM (Read-Only Memory), a part of the auxiliary storage device 1940, or a combination thereof is used. The auxiliary storage device 1940 stores data non-temporarily. The auxiliary storage device 1940 includes non-volatile memories such as HDDs or SSDs.

[0092] The input / output interface 1950 is an interface for connecting to or communicating with other devices. The input / output interface 1950 is used, for example, for connection to or communication with the first data storage unit 111, the second data storage unit 112, and the output device 120 shown in FIG. 1.

[0093] Each program stored in the program memory 1930 includes computer-executable instructions. When the program (computer-executable instructions) is executed by the CPU 1910, it causes the CPU 1910 to execute a predetermined process. For example, when the data analysis program is executed by the CPU 1910, it causes the CPU 1910 to execute a series of processes described with respect to each part of FIGS. 2 and 3.

[0094] The program may be provided to the computer 1900 in a state stored in a computer-readable storage medium. In this case, for example, the computer 1900 further includes a drive (not shown) for reading data from the storage medium and acquires the program from the storage medium. Examples of the storage medium include magnetic disks, optical disks (such as CD-ROM, CD-R, DVD-ROM, DVD-R), magneto-optical disks (such as MO), and semiconductor memories. Also, the program may be stored in a server on a communication network, and the computer 1900 may download the program from the server using the input / output interface 1950.

[0095] The processes described in the embodiments are not limited to being performed by a general-purpose hardware processor such as the CPU 1910 executing a program, and may be performed by a dedicated hardware processor such as an ASIC (Application Specific Integrated Circuit). The term processing circuit (processing unit) includes at least one general-purpose hardware processor, at least one dedicated hardware processor, or a combination of at least one general-purpose hardware processor and at least one dedicated hardware processor. In the example shown in FIG. 19, the CPU 1910, the RAM 1920, and the program memory 1930 correspond to the processing circuit.

[0096] Therefore, according to each of the above embodiments, it is possible to accurately compare the characteristics of a plurality of data acquired under different conditions.

[0097] Although some embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and its equivalent scope.

Description of Reference Numerals

[0098] 1…Data analysis system, 100…Data analysis device, 111…First data storage unit, 112…Second data storage unit, 120…Output device, 210…First acquisition unit, 220…First learning unit, 230…First clustering unit, 240…Second acquisition unit, 250…Second learning unit, 260…Second clustering unit, 270…Comparison unit, 280…Display control unit, 310…Feature vector calculation unit, 320…Loss calculation unit, 330…Model update unit, 340…Model storage unit, 700…First feature vector, 800, 900, 1000, 1300, 1400…Tables, 1200…Second feature vector, 1500, 1600, 1700, 1800…Comparison results, 1900…Computer, 1930…Program memory, 1940…Auxiliary storage device, 1950…Input / output interface, 1960…Bus.

Claims

1. A first acquisition unit that acquires a plurality of first data satisfying a first condition; A first learning unit that generates a plurality of first feature vectors by performing unsupervised learning on the plurality of first data; A first clustering unit that generates a first clustering result by clustering the plurality of first feature vectors; A second acquisition unit that acquires a plurality of second data satisfying a second condition different from the first condition; A second learning unit that generates a plurality of second feature vectors by performing unsupervised learning on at least some of the plurality of first data and the plurality of second data; A second clustering unit that generates a second clustering result by clustering the plurality of second feature vectors; A comparison unit that generates a comparison result regarding the plurality of first data and the plurality of second data by comparing the first clustering result and the second clustering result A data analysis apparatus comprising the above.

2. The first clustering result includes data in which information of the first feature vectors is associated with a first cluster number for distinguishing each cluster, The second clustering result includes data in which information of the second feature vectors is associated with a second cluster number for distinguishing each cluster, The data analysis apparatus according to Claim 1.

3. The first clustering result includes a first cluster label corresponding to the first cluster number, The data analysis apparatus according to Claim 2.

4. The first learning unit and the second learning unit perform unsupervised learning using a model that extracts feature amounts such that the distance between similar images becomes closer and the distance between dissimilar images becomes farther, The data analysis apparatus according to Claim 1.

5. The comparison result is a graph showing the ratio of the first data and the second data included in the clusters in the second clustering result, The data analysis apparatus according to Claim 1.

6. The comparison result is a correlation diagram representing the plurality of first feature vectors and the plurality of second feature vectors with different components, The data analysis apparatus according to Claim 1.

7. The second clustering unit performs clustering so that the number of clusters included in the second clustering result is larger than the number of clusters included in the first clustering result, The data analysis apparatus according to Claim 1.

8. The types of the first condition and the second condition are any one of different devices, dates and times, before and after maintenance, and before and after changes in the manufacturing process. The data analysis device according to claim 1.

9. The first data and the second data are images. The data analysis device according to any one of claims 1 to 8.

10. A computer acquires a plurality of first data satisfying a first condition; generates a plurality of first feature vectors by performing unsupervised learning on the plurality of first data; generates a first clustering result by clustering the plurality of first feature vectors; acquires a plurality of second data satisfying a second condition different from the first condition; generates a plurality of second feature vectors by performing unsupervised learning on at least some of the plurality of first data and the plurality of second data; generates a second clustering result by clustering the plurality of second feature vectors; generates a comparison result regarding the plurality of first data and the plurality of second data by comparing the first clustering result and the second clustering result A data analysis method comprising the steps of:

11. A computer means for acquiring a plurality of first data satisfying a first condition; means for generating a plurality of first feature vectors by performing unsupervised learning on the plurality of first data; means for generating a first clustering result by clustering the plurality of first feature vectors; means for acquiring a plurality of second data satisfying a second condition different from the first condition; means for generating a plurality of second feature vectors by performing unsupervised learning on at least some of the plurality of first data and the plurality of second data; means for generating a second clustering result by clustering the plurality of second feature vectors; means for generating a comparison result regarding the plurality of first data and the plurality of second data by comparing the first clustering result and the second clustering result A data analysis program for causing the computer to function as such.

Images