Data classification device, data classification method, and labeling method

Abstract

To provide a data classifier for performing data classifying work in a highly-accurate and easy manner.SOLUTION: A data classifier of the present invention comprises: a data input section for inputting data to be classified; a feature amount extraction section for extracting a feature amount of the data by a trained model which has learned using a neural network; a dimensional compression section for compressing a dimension of the extracted feature amount; and a display section for displaying the data whose compressed feature amounts are close to each other to form a group, at positions visually close to each other in the group.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a data classification device, a data classification method, and a labeling method. Give It concerns the method. [Background technology]

[0002] In recent years, data classification technology using deep learning has made remarkable progress. This technology generates classification rules by learning the characteristics of data that has been previously labeled with a label indicating the data's attributes, and classifies the data according to the classification rules. In general, in order to improve classification accuracy, a large amount of data with accurate labels is required, and the labels are assigned by humans. Therefore, there are problems in that labeling by humans requires a lot of effort and is prone to labeling errors. To address this problem, devices and methods shown in Patent Document 1 and Patent Document 2, for example, are known.

[0003] Patent Document 1 discloses a technique for calculating feature amounts of an image used for learning and using the feature amounts as indices for labeling.

[0004] Patent Document 2 discloses a technique for extracting feature amounts of an image used for learning and evaluating the validity of a label that has been previously assigned to the image based on the feature amounts. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 11-344450 [Patent Document 2] JP 2019-215698 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, Patent Document 1 and Patent Document 2 have the following problems. Patent Document 1 requires a person to define the features to be calculated, which is labor-intensive. Patent Document 2 requires a person to assign labels to images used for learning in advance, which is labor-intensive.

[0007] The present invention has been made in view of the above, and provides a data classification device and a data classification method that can perform data classification work accurately and easily. The present invention also provides a labeling support method that can perform labeling of data accurately and easily. [Means for solving the problem]

[0008] The data classification device according to the present invention, which solves the above-mentioned problems, a data input section into which data to be classified is input; A feature extraction unit that extracts features of the data using a trained model trained using a neural network; a dimensionality reduction unit that reduces the dimensions of the extracted feature quantity; and a display unit that displays a group of data having similar compressed features in a visually close position.

[0009] A preferred aspect of the present invention is a data classification device in which, in the above invention, the trained model is an autoencoder trained to reproduce input data in output, and the feature extraction unit extracts the output of an intermediate layer of the autoencoder as the feature.

[0010] In the data classification device according to a preferred embodiment of the present invention, the display unit displays a scatter diagram.

[0011] The data classification method according to the present invention, which solves the above-mentioned problems, comprises: a data input step for inputting data to be classified; A feature extraction step of extracting features of the data using a trained model trained using a neural network; a dimensionality reduction step of reducing the dimensions of the extracted features; and a display step of displaying a group of data having similar compressed feature amounts in a visually close position.

[0012] A preferred aspect of the data classification method of the present invention is such that, in the above invention, the trained model is an autoencoder trained to reproduce input data in output, and in the feature extraction step, the output of an intermediate layer of the autoencoder is extracted as the feature.

[0013] In the data classification method according to the above-mentioned invention, a scatter diagram is displayed in the display step.

[0014] The labeling assistance method of the present invention, which solves the above-mentioned problems, uses any of the above-mentioned data classification methods, and treats groups of data that are displayed in a visually close position in the display step as candidates to be assigned the same label.

[0015] In the labeling assistance method according to a preferred embodiment of the present invention, data to which no labels have been added in advance is used as the data to be input in the data input step. Effect of the Invention

[0016] According to the data classification device and data classification method of the present invention, data to be classified into the same category is displayed in a visually close position, so that data classification work can be performed accurately and easily.

[0017] Furthermore, according to the labeling assistance method of the present invention, since the data classification method of the present invention is used, labels can be assigned to data accurately and easily. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a block diagram showing a functional configuration of a data classification device according to an embodiment of the present invention. [Diagram 2]FIG. 2 is a diagram showing a configuration of an autoencoder according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a flowchart showing an outline of the process performed by the data classification device according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing a display example of the display result. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, a mode for carrying out the present invention (hereinafter, referred to as an "embodiment") will be described with reference to the accompanying drawings. Note that the present invention is not limited to the embodiment described below.

[0020] Fig. 1 is a block diagram showing the functional configuration of a data classification device according to an embodiment of the present invention. The data classification device 1 shown in the figure is a device that classifies defect image data obtained by capturing images of defects that occur during a product manufacturing process according to their characteristics. The defect image data can be obtained, for example, by a defect inspection device.

[0021] The following describes the functional configuration of the data classification device 1. The data classification device 1 includes an input unit 11, a data input unit 12, an image processing unit 13, a feature extraction unit 14, a dimensional compression unit 15, a display unit 16, a control unit 17, and a storage unit 18.

[0022] The input unit 11 receives input of various information and is configured using a user interface such as a keyboard, a mouse, and a microphone.

[0023] The data input unit 12 inputs defect image data sent from a defect inspection machine that images a product, extracts defects, and generates defect image data including the defects, and outputs the data to the memory unit 18 for storage.

[0024] The image processing unit 13 performs known image processing on the data of the defect image. For example, it is preferable to perform image processing in which the parts of the data of the defect image other than the defects are used as a common background, and the defect parts are superimposed on the background. The background is, for example, a single color of gray.

[0025] The feature extraction unit 14 extracts features of the defect image by inputting the defect image data processed by the image processing unit 13 to a trained model trained using a neural network. The trained model is a neural network consisting of an input layer, an intermediate layer, and an output layer, each layer having one or more nodes. The trained model is generated by performing training to optimize the network parameters of the neural network. The network parameters include information on the weights and biases between layers of the neural network. The input of the trained model is defect image data, and the output is a reproduction of the input defect image data. This neural network is an autoencoder.

[0026] Here, the autoencoder will be described with reference to FIG. 2. FIG. 2 is a diagram showing the configuration of the autoencoder. As shown in FIG. 2, the autoencoder 2 is a neural network model consisting of an encoder 21 and a decoder 22. In the encoder 21, the features of each node 23 are weighted with a unique weight at each edge 24, and the number of dimensions is reduced while inputting the features to the nodes 23 of the next layer. This allows the features of the input data 26 to be compressed to a low dimension. In the decoder 22, output data 27 is generated by performing the reverse process of the encoder 21. In other words, the output data 27 reproduces the input data 26 from the features compressed by the encoder 21. At this time, a feature amount according to the input data 26 is obtained in the intermediate layer 25 located between the encoder 21 and the decoder 22. The feature amount is generally defined as a multidimensional vector consisting of multiple components that give the features of the image data. Note that the number of layers of the autoencoder 2 in the present invention and the number of nodes 23 in each layer are not limited.

[0027] The dimension compression unit 15 compresses the dimensions of the features extracted by the feature extraction unit 14. Although the features are compressed to a low dimension by the autoencoder 2, the number of dimensions far exceeds what a human can handle in most cases. Therefore, it is preferable to compress the dimensions to a number of dimensions that a human can handle. As a method for dimensional compression, for example, t-Distributed Stochastic Neighbor Embedding (t-SNE) or Principal Component Analysis (PCA) can be applied, but is not limited to these. In addition, the number of dimensions after dimensional compression is preferably two or three.

[0028] The display unit 16 displays groups of data with similar features compressed by the dimensionality compression unit 15 in visually close positions. The display method is not particularly limited as long as it is visually easy to recognize, but a scatter diagram is preferable. Note that "close features" refers to a small value representing the distance between multidimensional vectors, such as Euclidean distance or Mahalanobis distance.

[0029] Control unit 17 comprehensively controls the operation of data classification device 1. It also displays the processing results on a display and outputs them to the outside.

[0030] The storage unit 18 includes a trained model storage unit 181. The trained model storage unit 181 stores the trained model described above.

[0031] The storage unit 18 stores the defect image data input by the data input unit 12 and the results obtained by the processing of each unit. The storage unit 18 also stores various programs for operating the data classification device 1, and data including various parameters necessary for the operation of the data classification device 1. The storage unit 18 is configured using a ROM (Read Only Memory) in which various programs, etc. are pre-installed, and a RAM (Random Access Memory) for storing calculation parameters and data for each process, etc.

[0032] The various data including the trained model stored in the storage unit 18 may be recorded on a computer-readable recording medium such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a CD-ROM, a DVD-ROM, or a Blu-ray (registered trademark), or may be downloadable via a communication network. The communication network referred to here is configured using at least one of an existing public line network, a local area network (LAN), and a wide area network (WAN), for example, and may be wired or wireless.

[0033] Data classification device 1 having the above-described functional configuration is a computer configured using one or more hardware components such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field Programmable Gate Array).

[0034] 3 is a flowchart showing an outline of the processing performed by the data classification device 1. First, the data input unit 12 inputs data of a defect image (step S1).

[0035] Next, the image processing unit 13 performs image processing (step S2). At this time, the image processing unit 13 performs known image processing. The purpose here is to remove unnecessary areas other than the defect portion in the image, to eliminate images in which it is clear that no defect is captured, and so on. In the processing from step S3 onwards, all images may be used, or the images to be processed may be selected in the image processing of step S2. Here, processing is performed to cut out a specific size with the defect portion at the centre, and processing is performed to make the background area other than the defect a monochrome background such as grey, and the defect portion is superimposed on this background.

[0036] Thereafter, the feature extraction unit 14 extracts features using the trained model stored in the trained model storage unit 181 (step S3). Specifically, the feature extraction unit 14 inputs the data of the defect image processed by the image processing unit 13 to the trained model, and extracts the output of the intermediate layer as a feature.

[0037] Next, the dimension compression unit 15 compresses the dimensions of the feature extracted in step S3 (step S4). Specifically, the dimension compression unit 15 compresses the dimensions of the feature extracted in step S3 by a method such as t-SNE or PCA. At this time, the number of dimensions after compression is preferably two or three so that it is easy to visually recognize when displaying a group of data in the next step S5.

[0038] Thereafter, the display unit 16 displays groups of data having similar features in visually close positions based on the features compressed in step S4 (step S5). The display method will be described with reference to FIG. 4. FIG. 4 is a diagram showing an example of display by the display unit 16. On screen 4 shown in FIG. 4, the features compressed three-dimensionally in step S4 are associated with an X-axis 41, a Y-axis 42, and a Z-axis 43, and plotted in a three-dimensional scatter diagram. Furthermore, the plot points display data of corresponding defect images. Groups 44, 45, and 46 of defect image data are groups of data having similar features, and data having features far from any of the data groups is defect image data 47.

[0039] According to this display example, a group of data with similar features is displayed in a scatter diagram together with defect image data based on the features compressed in step S4, so that the user can visually understand the classification results.

[0040] After step S4, the display result may be adopted as a candidate for labeling in addition to being adopted as a classification result. When the display result is adopted as a candidate for labeling, a group of data with similar features is displayed in a visually close position together with the defect image data, so that the labor of labeling can be reduced and labeling errors can be prevented. Also, like defect image data 47, data that may be judged at first glance to have the same features as the defect image data group 44, so-called outliers, are visually easy to understand. Therefore, it is possible to suppress the assignment of inappropriate labels, and improve the accuracy of the data used for learning, and thus the classification accuracy of the classification rule generated from that data. In general, using data including outliers for learning to generate a classification rule leads to a decrease in classification accuracy.

[0041] According to one embodiment of the present invention described above, features of the defect image data are extracted using the input defect image data and a trained model, the dimensions of the extracted features are compressed, and based on the compressed features, groups of data with similar features are displayed in a visually close position together with the defect image data, allowing the user to visually understand the classification results.

[0042] Furthermore, according to this embodiment, when the display results are considered as candidates for labeling, a group of data with similar features is displayed in a visually close position together with the defect image data, which can reduce the effort required for labeling and prevent labeling errors.

[0043] Furthermore, according to this embodiment, when the display results are considered as candidates for labeling, outlier data is visually easy to understand, which makes it possible to prevent inappropriate labels from being assigned, thereby improving the accuracy of the data used for learning, and ultimately the classification accuracy of the classification rules generated from that data.

[0044] Although the embodiment of the present invention has been described above, the present invention should not be limited to the above-mentioned embodiment. Furthermore, the present invention can be applied to time-series data, document data, etc., in addition to data of defect images during product manufacturing. [Explanation of symbols]

[0045] 1 Data Classifier 2. Autoencoder 11 Input section 12 Data Entry Section 13 Image processing section 14 Feature Extraction Unit 15 Dimensional compression section 16 Display section 17 Control section 18 Memory section 21 Encoder 22 Decoder 23 Nodes 24 Edge 25 Middle Class 26 Input Data 27 Output Data 4 Screens 41 X-axis 42 Y-axis 43 Z axis 44, 45, 46 Group of defect image data 47 Defect image data 181 Trained model memory

Claims

1. a data input unit for inputting data that is not assigned a label to be classified; A feature extraction unit that extracts features of the data using a trained model trained using a neural network; a dimensionality compression unit that compresses the dimensions of the extracted feature quantity; a display unit that displays a group of the data themselves having similar compressed feature amounts in a visually close position, and displays the data themselves having distant compressed feature amounts in a visually distant position; A data classification device comprising:

2. The trained model is an autoencoder trained to reproduce input data in an output; The data classification device according to claim 1 , wherein the feature extraction unit extracts an output of an intermediate layer of the autoencoder as the feature.

3. 3. The data classification device according to claim 1, wherein the display unit displays a scatter diagram.

4. A data classification device according to any one of claims 1 to 3, wherein the data is image data.

5. A data input step of inputting data that has not been assigned a label to be classified; A feature extraction step of extracting features of the data using a trained model trained using a neural network; a dimensionality reduction step of reducing the dimensions of the extracted feature quantity; a display step of displaying a group of the data themselves having similar compressed feature amounts in a visually close position and displaying the data themselves having distant compressed feature amounts in a visually distant position; Data classification methods, including:

6. The trained model is an autoencoder trained to reproduce input data in an output; The data classification method according to claim 5 , wherein in the feature extraction step, an output from an intermediate layer of the autoencoder is extracted as the feature.

7. 7. The data classification method according to claim 5, wherein a scatter diagram is displayed in said display step.

8. A data classification method according to any one of claims 5 to 7, wherein the data is image data.

9. A method for labeling unlabeled data, comprising: Using the data classification method according to any one of claims 5 to 8, the same label is assigned to the group of data displayed in a visually close position in the display step, and the label assigned to the group of data is not assigned to the data displayed in a visually distant position from the group of data; Labeling method.

Images