Learning data generation device, learning data generation method, learning data generation program, and classification device

Abstract

To provide a learning data generation device capable of generating sufficient learning data for creating a predictive model through machine learning.SOLUTION: A learning data generation device comprises: raw data creation means 3 for acquiring a feature amount from waveforms already obtained from a device and creating raw data for balancing classes along with the waveform class; minority data increase means 4 for increasing minority data using the raw data to obtain a plurality of sets of new set data composed of waveform feature amounts and class sets; waveform generation means 5 for generating waveforms using the feature amounts set for each set of set data obtained by the minority data increase means 4; and learning data generation means 6 for displaying the generated waveforms on a display device 25, accepting an input of the displayed waveform class, and generating learning data by combining the entered class and the corresponding feature amounts as one set.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a training data generating device, a training data generating method, a training data generating program, and a classification device. [Background technology]

[0002] The design of semiconductors and various devices requires the evaluation of signal waveforms using measuring instruments such as oscilloscopes. Currently, signal waveforms acquired by measuring instruments such as oscilloscopes are checked by workers who visually judge whether they pass or fail. However, when analyzing signals such as voltage acquired from measuring instruments as numerical values, there is a problem in that the data cannot be used as is due to precision errors in the measuring instruments. Furthermore, when creating a predictive model using machine learning, signal waveforms that are judged to be abnormal signals occur very rarely, so there are very few samples of abnormal signals to learn from, making it difficult to create a machine learning judgment model.

[0003] Patent Document 1 discloses a method, device, and computer program for creating category determination rules that can create determination rules with fewer erroneous determinations. In these category determination rule creating methods, devices, and computer programs, the teaching data included in one of the divided groups is set as verification data, a verification category determination rule is created based on the teaching data included in the remaining groups, and the operation of assigning a category determination to the verification data based on the verification category determination rule is repeated while sequentially changing the group set for the verification data, the verification data and the corresponding category determination are presented, and correction or deletion is prompted, the teaching data is corrected in response to the instruction, and a category determination rule is created and output based on the corrected teaching data.

[0004] Patent Document 2 discloses a monitoring method for collecting information measured by instruments and monitoring a monitoring target. This method includes the steps of detecting display information showing the state of a monitored object using a non-contact sensor, converting the detected data into digital data, collecting the digital data, and determining the state of the monitored object based on the digital data. Specifically, in the detecting step, a measuring instrument is photographed with a camera, and in the converting step, the parameter values ​​displayed on the measuring instrument are converted into digital data by analyzing the photographed image.

[0005] Patent Document 3 discloses an anomaly detection device that can perform anomaly detection in consideration of a process flow consisting of a plurality of operating states. This anomaly detection device includes a feature extraction unit that receives time series data of a target device and extracts feature quantities of the time series data; a state transition estimation unit that identifies an operating state of the target device from the extracted feature quantities and estimates the state transition of the target device from the extracted feature quantities by referring to state transition pattern information that defines state transition patterns between a plurality of operating states; and an anomaly detection unit that refers to the state transition pattern information and normal range information that defines normal ranges for each operating state, and determines whether the time series data is abnormal based on the identified operating state, the estimated state transition, and the feature quantities of the extracted time series data. According to this anomaly detection device, anomaly detection is performed by referring to state transition pattern information and normal range information, so that anomaly detection can be performed at two stages: the state transition stage and the stage of each operating state. Therefore, anomaly detection can be performed taking into account the flow of a process consisting of multiple operating states.

[0006] The above features include indicators such as long-term or short-term trends of the sensor value waveform, waveform length, frequency, update frequency of the sensor value, and similarity between multiple partial waveforms contained in a certain waveform (time series data). [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-13720 [Patent Document 2] Japanese Patent Publication No. 2022-155615 [Patent Document 3] WO2022 / 054256 publication Summary of the Invention [Problem to be solved by the invention]

[0008] As mentioned above, although there are known methods for determining waveform features and making judgments, they are not able to address the problem of not being able to use data directly from measuring instruments that have accuracy errors.

[0009] An object of the invention according to the embodiments of the present invention is to provide a training data generation device, a training data generation method, a training data generation program, and a classification device that can address the problem of being unable to handle data as is due to accuracy errors in measuring instruments, and that can generate sufficient training data for learning by machine learning to create a predictive model even when a signal waveform that is determined to belong to a certain class occurs very rarely. [Means for solving the problem]

[0010] A training data generation device according to an embodiment of the present invention is characterized by comprising: raw data creation means for acquiring features from a waveform already obtained from a device, and creating raw data for balancing the classes together with the class of the waveform; minor data increase means for increasing minor data using the raw data to obtain multiple sets of new set data consisting of sets of waveform features and classes; waveform generation means for generating a waveform for each set of set data obtained by the minor data increase means using the features that make up the set; and training data generation means for displaying the generated waveform on a display device and accepting input of the class of the displayed waveform, and generating training data using the input class and the feature as a set.

[0011] In the training data generation device according to an embodiment of the present invention, the raw data creation means creates a moving average by dividing the number of waveform data by the number of pixels and further reducing the resulting value in steps, and extracts features using the value of this moving average.

[0012] In the learning data generating device according to the embodiment of the present invention, the raw data creating means extracts maximum and minimum values ​​of a waveform as feature quantities.

[0013] In the learning data generating device according to the embodiment of the present invention, the raw data creating means extracts the number of steps at a predetermined height between the maximum and minimum values ​​of the waveform as the feature amount.

[0014] In the learning data generation device according to an embodiment of the present invention, the waveforms are waveforms of the output voltage of the device, the temperature of the device, the noise of the device, the sound generated from the device, the volume generated from the device, and the vibration generated from the device.

[0015] In the training data generation device according to the embodiment of the present invention, the minor data increasing means obtains a plurality of sets of new data sets using the SMOTE (Synthetic Minority Oversampling TEchnique) method.

[0016] A classification device according to an embodiment of the present invention is characterized by comprising: a training data creation device as set forth in claim 1; a prediction model creation means for creating a prediction model that predicts a class from features by learning using the training data created by the training data creation device; and a class acquisition means for determining a class using the prediction model based on features of a waveform obtained from a device under test. [Brief explanation of the drawings]

[0017] [Figure 1]FIG. 1 is a block diagram of a learning data generation device according to an embodiment of the present invention, which is configured using a computer. [Figure 2] FIG. 2 is a diagram showing components provided in an external storage device of the learning data generation device according to the embodiment of the present invention. [Figure 3] FIG. 2 is a diagram showing an example of a waveform handled by the learning data generating device according to the embodiment of the present invention. [Figure 4] FIG. 4 is an enlarged view of the main part of FIG. 3. [Figure 5] 3 is a flowchart showing the operation of the learning data generation device according to the embodiment of the present invention. [Figure 6] FIG. 2 is a diagram showing components included in an external storage device of the classification device according to the embodiment of the present invention. [Figure 7] 10 is a flowchart showing the operation of the classification device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, a training data generation device, a training data generation method, a training data generation program, and a classification device according to embodiments of the present invention will be described with reference to the accompanying drawings. In each drawing, the same components are designated by the same reference numerals, and duplicated explanations will be omitted.

[0019] A training data generation device according to an embodiment of the present invention can be configured using a computer, as shown in Fig. 1. That is, a CPU 10 configures the training data generation device using programs and data in a main memory 11. An external storage interface 13, an input interface 14, a display interface 15, and a data input interface 16 are connected to the CPU 10 via a bus 12.

[0020] An external storage device 23 is connected to the external memory interface 13. The external storage device 23 stores programs and data for the operation of the training data generation device, which the CPU 10 can read into the main memory 11 as needed for use. For this reason, as shown in FIG. 2, the external storage device 23 stores programs for implementing the raw data creation means 3, the minor data increase means 4, the waveform generation means 5, and the training data generation means 6. An input device 24, such as a keyboard or touch panel, and a pointing device 22, such as a mouse, are connected to the input interface 14. A display device 25 having a screen such as an LCD is connected to the display interface 15. An oscilloscope 26, which is a measuring instrument, is connected to the data input interface 16. The oscilloscope 26 functions as a measuring device for obtaining the waveform of a measurement target signal from a device being designed, such as a semiconductor. The waveform signal obtained by the oscilloscope 26 is input by the CPU 10 via the data input interface 16 and processed to extract feature values.

[0021] In this embodiment, the raw data creation means 3, minor data increase means 4, waveform generation means 5, and learning data generation means 6 provided in the external storage device 23 have the following functions: The raw data creation means 3 acquires features from waveforms already obtained from the device, and creates raw data for balancing the classes along with the classes of the waveforms. The minor data increase means 4 increases the minor data using the raw data to obtain multiple sets of new set data composed of sets of waveform features and classes.

[0022] The waveform generating means 5 generates a waveform using the feature amounts of the set for each set of set data obtained by the minor data increasing means 4. The learning data generating means 6 displays the generated waveform on the display device 25 and accepts input of the class of the displayed waveform, and generates learning data by combining the input class and the feature amount as one set. In this case, input can be made from the input device 24 or the pointing device 22.

[0023] The raw data creation means 3 creates a moving average by dividing the number of waveform data by the number of pixels, and then further reducing this value to create a smaller step, and uses this moving average value to extract features. Here, the waveform data obtained from the oscilloscope 26 is the power supply voltage VDD, and is displayed as shown in Figure 3. The vertical axis is V (volts), and the horizontal axis is time, expressed in steps of a predetermined time interval.

[0024] Even in the areas indicated by circles in FIG. 3 where the potential is monotonically increasing, the enlarged waveform constantly fluctuates up and down, as shown in FIG. 4. In FIG. 4, a portion of the waveform that is not enlarged is shown as sCH2. In this embodiment, the total number of plots (total number of steps) of the waveform data is N, and in FIGS. 3 and 4, N=250,000. In this case, a short-term moving average (MA_SHORT) is calculated. MA_SHORT can be obtained by dividing the number of data by the number of pixels and calculating the moving average at 1 / 10 resolution. In the examples shown in FIGS. 3 and 4, the value obtained when 250,000 lines of plot data are expressed (divided) by 1,000 pixels can be used to calculate the moving average every 25 steps at 1 / 10 resolution.

[0025] Also, calculate the long-term moving average (MA_LONG). MA_LONG can be calculated using a resolution of the value obtained by dividing the number of data points by the number of pixels. In the example shown in Figures 3 and 4, the resolution is the value obtained when 250,000 lines of plotted data are expressed (divided) by 1,000 pixels, and the moving average can be calculated every 250 steps.

[0026] In this embodiment, waveform data is obtained using the moving average described above, and feature quantities are obtained from this data. Examples of feature quantities include VDD (power supply voltage), the maximum voltage value (VMAX) and minimum voltage value (VMIN) of the waveform data, Transition (transition rate), Δpct, EDGE, overshoot, undershoot (at rising edge), and Drop 1 to 4.

[0027] The above VDD (power supply voltage) is calculated by rounding the voltage data to two decimal places (0.00). The mode of this rounded data is then extracted, and if the remaining voltage data represents a stable section (GND / VDD state) of the screen (40% or more), the extraction of the mode is repeated, and the highest of the list of modes is taken as VDD. The maximum voltage value (VMAX) and minimum voltage value (VMIN) of the waveform data are self-explanatory. In this way, the raw data creation means 3 extracts the maximum and minimum values ​​of the waveform as feature quantities. The transition (transition rate) is the percentage of the window size (= number of index steps) when the MA_LONG (moving average) waveform crosses (goes above or below) the 99% potential, and the percentage of the window size (= number of index steps) when the MA_LONG (moving average) waveform crosses the 1% potential. In this way, the raw data creating means 3 extracts the number of steps at a predetermined height between the maximum and minimum values ​​of the waveform as a feature amount.

[0028] Δpct can be calculated by subtracting the index at 1% from the index at which the potential crosses 5%, 10%, 30%, 50%, 70%, 90%, and 95% of the MA_LONG (moving average) waveform, and dividing the result by the total number of data points (N = 250,000). Δpct may also include the value calculated similarly for MA_SHORT. The EDGE waveform of the MA_LONG (moving average) can be defined as a rising edge if the index at 99% potential is greater than the index at 1% potential, and a falling edge otherwise. For overshoot and undershoot (rising edge), the maximum potential within a window size twice the transition from the index at 99% potential can be obtained as the overshoot, and the minimum potential within a window size twice the transition from the index at 1% potential can be obtained as the undershoot. Drops 1 to 4 can be calculated by dividing the step from the index at 99% potential to the end into four parts, and obtaining the potentials as monitoring values.

[0029] The processing of the training data creation device according to the embodiment of the present invention is realized by the processing by the raw data creation means 3 as described above, as well as the processing by the minor data increasing means 4, the waveform generation means 5, and the training data generation means 6. That is, as shown in FIG. 5, the CPU 10 acquires waveform data from the oscilloscope 26 via the data input interface 16 (S11). As already described, this waveform data is used for preprocessing such as obtaining a moving average (S12). Furthermore, as already described, feature extraction is performed (S13). At this time, the waveform data is classified as either normal or abnormal via the data input interface 16 or from the input device 24, and raw data is created to balance the classes.

[0030] Next, the original data is used to increase the minor data to obtain multiple sets of new set data composed of sets of waveform features and classes (S14). This step S14 is performed by the minor data increasing means 4. The minor data increasing means 4 obtains multiple sets of new set data using the SMOTE (Synthetic Minority Oversampling TEchnique) method, which is known as a method for balancing classes.

[0031] Next, for each set of the obtained set data, a waveform is generated using the feature quantities that make up the set (S15). The generated waveform is displayed on a display device, and an input of the class of the displayed waveform is accepted (S16). That is, a highly skilled inspector visually inspects the generated and displayed waveform for normality or abnormality, and the normality or abnormality is input from the input device 24.

[0032] The input class and the feature are treated as one set, and multiple sets of training data are generated by SMOTE (S17). The generated data sets are displayed and stored in the external storage device 23 for use in creating a classification device.

[0033] Next, a classification device according to an embodiment of the present invention will be described. When the classification device according to an embodiment of the present invention is configured using a computer, it can have the same configuration as the training data generation device according to an embodiment of the present invention, as shown in FIG.

[0034] However, the external storage device 23 stores programs and data for the operation of this classification device, which can be read out and used by the CPU 10 as needed into the main memory 11. For this reason, the external storage device 23 stores programs for realizing the raw data creation means 3, the minor data increasing means 4, the waveform generation means 5, the learning data generation means 6, the prediction model creation means 7, and the class acquisition means 8, as shown in Fig. 6.

[0035] The prediction model creation means 7 creates a prediction model that predicts a class from features by learning using the learning data created by the learning data creation device. The class acquisition means 8 uses the prediction model to determine a class based on the features of the waveform obtained from the device under test.

[0036] The classification device of this embodiment operates using a program corresponding to the flowchart shown in Fig. 7, and its operation will be explained using this flowchart. A prediction model that predicts classes from features is created by learning using the learning data created by the learning data creation device (S21). As a result, a prediction model (program) is created in the external storage device 23, as shown in Fig. 5.

[0037] From this point onward, the device to be inspected is inspected. First, waveform data from the new inspection target is read (S22). Here, the CPU 10 acquires the waveform signal obtained by the oscilloscope 26 via the data input interface 16. As already explained for the training data generation device, preprocessing such as obtaining a moving average is performed (S23). Furthermore, as already explained for the training data generation device, feature extraction is performed (S24). Next, a class is determined based on the feature using a prediction model (S25).

[0038] According to this embodiment, it is possible to create a device that automatically judges signal waveforms by machine learning using waveforms obtained with an inexpensive oscilloscope, even in situations where there is little abnormal waveform data.

[0039] The waveforms in this embodiment are not limited to the output voltage of a device, but may also be waveforms of the temperature of a device, noise from a device, sound generated by a device, volume generated by a device, or vibration generated by a device. Furthermore, the feature quantities are not limited to those described above, and various features can be appropriately adopted depending on the target waveform. Furthermore, although two classes (abnormal and normal) have been used, this is not limiting, and three classes, or even four or more classes, are also possible. [Explanation of symbols]

[0040] 3. Methods for creating original data 4. Methods for increasing My Number data 5 Waveform generation means 6. Training data generation method 7. Methods for creating predictive models 8. How to obtain a class 10 CPU 11 Main Memory 12 Bus 13 External memory interface 14 Input Interface 15 Display Interface 16 Data Input Interface 22 Pointing Device 23 External storage device 24 Input Devices 25 Display device 26 Oscilloscope

Claims

1. a raw data generating means for acquiring features from a waveform already obtained from a device and generating raw data for balancing the classes together with the class of the waveform; a minor data increasing means for increasing the minor data using the original data to obtain a plurality of new set data each composed of a set of waveform features and classes; a waveform generating means for generating a waveform using the feature quantities of each set of set data obtained by the minor data increasing means; a training data generating means for displaying the generated waveform on a display device, receiving an input of the class of the displayed waveform, and generating training data by combining the input class and the feature quantity as one set; A learning data creation device comprising:

2. 2. The learning data creation device according to claim 1, wherein the raw data creation means creates a moving average by dividing the number of waveform data by the number of pixels and further reducing the resulting value, and extracts features using the value of this moving average.

3. 3. The learning data generating device according to claim 2, wherein said raw data generating means extracts maximum and minimum values ​​of a waveform as feature quantities.

4. 3. The learning data generating device according to claim 2, wherein said raw data generating means extracts the number of steps at a predetermined height between a maximum value and a minimum value of the waveform as the feature amount.

5. The learning data creation device according to claim 1, characterized in that the waveforms are waveforms of the output voltage of the device, the temperature of the device, the noise of the device, the sound generated from the device, the volume generated from the device, and the vibration generated from the device.

6. 2. The learning data generating device according to claim 1, wherein the minor data increasing means obtains a plurality of sets of new set data using a SMOTE (Synthetic Minority Oversampling TEchnique) method.

7. A learning data creation method in which a computer constituting a learning data creation device executes each step, a raw data creation step of acquiring features from waveforms already obtained from the device and creating raw data for balancing the classes together with the classes of the waveforms; a minor data increasing step of increasing minor data using the original data to obtain a plurality of new set data composed of sets of waveform features and classes; a waveform generating step of generating a waveform using the feature quantities of each set of set data obtained by the minor data increasing step; a training data generation step of displaying the generated waveform on a display device, receiving an input of the class of the displayed waveform, and generating training data by combining the input class and the feature quantity as one set; A learning data creation method comprising the steps of:

8. 8. The learning data creation method according to claim 7, wherein the original data creation step is configured to create a moving average by dividing the number of waveform data by the number of pixels, further reducing the resulting value, and extracting features using the value of this moving average.

9. 9. The method for creating learning data according to claim 8, wherein said original data creating step comprises extracting maximum and minimum values ​​of a waveform as feature quantities.

10. 9. The method for creating learning data according to claim 8, wherein the original data creation step is configured by extracting the number of steps at a predetermined height between the maximum and minimum values ​​of the waveform as a feature.

11. The method for creating learning data according to claim 7, characterized in that the waveforms are waveforms of the output voltage of the device, the temperature of the device, the noise of the device, the sound generated from the device, the volume generated from the device, and the vibration generated from the device.

12. 8. The method for creating training data according to claim 7, wherein the minor data increasing step is configured by obtaining a plurality of sets of new set data using the SMOTE (Synthetic Minority Oversampling TEchnique) method.

13. Computer, raw data creation means for acquiring features from waveforms already obtained from a device and creating raw data for balancing the classes together with the classes of the waveforms; a minor data increasing means for increasing the minor data using the original data to obtain a plurality of new sets of data each composed of a set of waveform features and classes; a waveform generating means for generating a waveform using the feature quantities of each set of set data obtained by the minor data increasing means; a training data generating means for displaying the generated waveform on a display device, receiving an input of the class of the displayed waveform, and generating training data by combining the input class and the feature quantity as one set; A program for creating learning data, characterized in that it functions as

14. The program for creating learning data according to claim 13, characterized in that the computer is configured to function as the raw data creation means, to create a moving average by dividing the number of waveform data by the number of pixels and further reducing the value, and to extract features using the value of this moving average.

15. 15. The program for generating learning data according to claim 14, wherein the computer is made to function as the original data generating means so as to extract maximum and minimum values ​​of a waveform as feature quantities.

16. The program for creating learning data according to claim 14, characterized in that the computer is made to function as the original data creation means by extracting the number of steps at a predetermined height between the maximum and minimum values ​​of the waveform as a feature.

17. The program for creating learning data according to claim 13, characterized in that the waveforms are waveforms of the output voltage of the device, the temperature of the device, the noise of the device, the sound generated from the device, the volume generated from the device, and the vibration generated from the device.

18. 14. The program for creating learning data according to claim 13, wherein the computer is caused to function as a minor data increasing means to obtain multiple sets of new set data using the SMOTE (Synthetic Minority Oversampling TEchnique) method.

19. The learning data creation device according to claim 1 ; a prediction model creation means for creating a prediction model that predicts a class from a feature by learning using the learning data created by the learning data creation device; a class obtaining means for obtaining a class using the prediction model based on the feature amount of a waveform obtained from a device under test; A classification device comprising:

Images