Outlier detection method and apparatus

Abstract

To provide an outlier detection method and apparatus capable of properly detecting an outlier.SOLUTION: A method includes the steps of: determining a coefficient of variation of values of objective variable for each training data 31, the training data 31 each including a plurality of values of objective variable, and defining outlier candidate data 32 which is data larger in coefficient of variation than a reference value set in advance and defining normal data 33 which is other than those above; creating a first regression model and calculating a first evaluation index, using a part of the normal data 33 as test data and the other part of the normal data 33 as first training data, and setting an evaluation index threshold for outlier determination, based on the first evaluation index; and creating a second regression model and calculating a second evaluation index using second training data obtained by adding each data included in the outlier candidate data 32 to the first training data, and comparing the second evaluation index with the evaluation index threshold to determine an outlier.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an outlier detection method and apparatus. [Background technology]

[0002] There are known methods for making various predictions using machine learning. For example, when predicting the physical properties of a material with an unknown composition, machine learning is performed using data already obtained through test manufacturing or the like as training data (teacher data), and the correlation between the composition of the material and its physical properties is learned, and predictions are made using the regression model obtained as a result of the learning.

[0003] Prior art document information related to the invention of this application includes Patent Document 1. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-33544 Summary of the Invention [Problem to be solved by the invention]

[0005] However, if the training data contains outliers, which are incorrect data or data with large errors, the predictive accuracy of the regression model obtained using the training data will decrease. Therefore, it is desirable to remove outliers from the training data prior to machine learning.

[0006] Conventionally, statistical methods have been used, such as identifying data whose objective variable value is outside the range of the mean value ±3σ or data whose value is outside the interquartile range as outliers. However, because there are cases where data with large variance is not an outlier, these methods have the risk of detecting data that is not an outlier as an outlier.

[0007] Therefore, an object of the present invention is to provide an outlier detection method and device that can appropriately detect outliers. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the present invention provides a method for detecting outliers contained in training data including data on explanatory variables and dependent variables used in machine learning, wherein each piece of data contained in the training data includes a plurality of values ​​of the dependent variable, and the method includes a data classification step of calculating a coefficient of variation of the value of the dependent variable for each piece of data contained in the training data, and classifying, among the data contained in the training data, data having the calculated coefficient of variation greater than a set reference value as candidate outlier data and other data as normal data; and a step of using the first training data to create a first regression model showing a correlation between the explanatory variables and the dependent variable. and an outlier detection method comprising: a threshold setting step of calculating a first evaluation index for the first regression model using the test data to evaluate the prediction accuracy of the first regression model, and setting an evaluation index threshold for outlier determination based on the first evaluation index; and an outlier determination step of adding each piece of data included in the outlier candidate data to the first training data to make the second training data, creating a second regression model using the second training data, and calculating a second evaluation index for the second regression model using the test data to evaluate the prediction accuracy of the second regression model, and comparing the second evaluation index with the evaluation index threshold to determine whether the added data is an outlier.

[0009] Furthermore, in order to solve the above-mentioned problems, the present invention provides an apparatus for detecting outliers included in training data containing data on explanatory variables and dependent variables used in machine learning, wherein each piece of data included in the training data contains a plurality of values ​​of the dependent variable, and the apparatus includes a data classification processing unit that calculates a coefficient of variation of the value of the dependent variable for each piece of data included in the training data, and classifies data included in the training data whose calculated coefficient of variation is greater than a set reference value as outlier candidate data, and other data as normal data; and a data classification processing unit that uses a part of the normal data as test data and the other part of the normal data as first training data, and creates a first regression model showing a correlation between the explanatory variables and the dependent variable using the first training data. an outlier detection device comprising: a threshold setting processing unit that creates a first regression model using the test data, calculates a first evaluation index for evaluating the prediction accuracy of the first regression model, and sets an evaluation index threshold for outlier detection based on the first evaluation index; and an outlier determination processing unit that, for each data included in the outlier candidate data, adds the respective data to the first teacher data to make the second teacher data, creates a second regression model using the second teacher data, calculates a second evaluation index for the second regression model using the test data, and determines whether the added data is an outlier by comparing the second evaluation index with the evaluation index threshold. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide an outlier detection method and device that can appropriately detect outliers. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic configuration diagram of an outlier detection device according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram illustrating an example of learning data. [Figure 3]10(a) is a diagram showing an example of the calculation results of the coefficient of variation, and FIG. 10(b) is a diagram showing the calculation results of FIG. 10(a) as a histogram. [Figure 4] FIG. 10 is a diagram illustrating the calculation of an evaluation index reference value. [Figure 5] 10A and 10B are diagrams illustrating the calculation of an evaluation index in the outlier determination process. [Figure 6] FIG. 4 is a diagram showing the results of calculating MAPE when each of the 87 outlier candidate data extracted in FIG. 3(a) is excluded from the training data. [Figure 7] FIG. 7 is a diagram showing the calculation results of MAPE when the evaluation index threshold is changed in FIG. 6. [Figure 8] FIG. 1 is a flow diagram of an outlier detection method according to an embodiment of the present invention. [Figure 9] FIG. 10 is a flow diagram of a data classification process. [Figure 10] FIG. 10 is a flowchart of a threshold setting process. [Figure 11] FIG. 10 is a flowchart of an outlier determination process. DETAILED DESCRIPTION OF THE INVENTION

[0012] [Embodiment Mode] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0013] 1 is a schematic diagram of an outlier detection device 1 according to this embodiment. The outlier detection device 1 is a device that detects outliers contained in training data 31 used for machine learning. An outlier is a value that deviates significantly from other data due to, for example, a measurement error, a human error such as an incorrect reading of an instrument or an input error, or the influence of noise. By detecting and removing outliers from the training data 31, it is expected that the prediction accuracy will improve when machine learning is performed using the training data 31.

[0014] The outlier detection device 1 has a control unit 2 and a storage unit 3. The outlier detection device 1 is, for example, a computer such as a personal computer or a server device, and is equipped with a processing element such as a CPU, memories such as RAM and ROM, a storage device such as a hard disk, and a communication interface which is a communication device such as a LAN card.

[0015] The control unit 2 has a data acquisition processing unit 21, a data classification processing unit 22, a threshold setting processing unit 23, an outlier determination processing unit 24, and a detection result presentation processing unit 25. Details of each unit will be described later. The storage unit 3 is realized by a predetermined storage area of ​​a memory or a storage device.

[0016] The outlier detection device 1 also has a display 4 and an input device 5. The display 4 is, for example, a liquid crystal display, and the input device 5 is, for example, a keyboard or a mouse. The display 4 may be configured as a touch panel and also function as the input device 5. The display 4 and the input device 5 may also be configured separately from the outlier detection device 1 and configured to be able to communicate with the outlier detection device 1 via wireless communication or the like. In this case, the display 4 or the input device 5 may be configured as a mobile terminal such as a tablet or a smartphone.

[0017] (Data acquisition processing unit 21) The data acquisition processing unit 21 performs a data acquisition process to acquire training data 31 from an external device. In the data acquisition process, the training data 31 is acquired via a network, for example, from a prediction device or the like that uses the training data 31. Note that the training data 31 may be input to the outlier detection device 1 via a medium such as a USB memory, and the method for acquiring the training data 31 is not particularly limited.

[0018] (Training data 31) Here, the training data 31 will be explained. FIG. 2 is a diagram showing an example of the training data 31. The training data 31 is a database used as training data during machine learning, and includes data on explanatory variables and response variables used in machine learning. The example in FIG. 2 shows a case in which the blend amounts of materials such as polymers and fillers are used as explanatory variables, and the physical properties (here, tensile strength) of a composite material manufactured using these materials are used as the response variable. By performing machine learning using this training data 31 and creating a regression model that shows the correlation between the explanatory variables (the blend amounts of each material) and the response variable (physical properties), it becomes possible to predict the physical properties that will be obtained when a composite material is manufactured with an unknown blend of materials.

[0019] In this embodiment, each data item included in the training data 31 includes multiple values ​​of the objective variable. Here, each data item includes five objective variable values ​​(tensile strength values ​​in the illustrated example). These objective variable values ​​are obtained, for example, by manufacturing a composite material with the same composition, forming multiple samples (here, five samples No. 1 to No. 5), and measuring the physical properties such as tensile strength of each sample. Here, a case will be described in which the training data 31 initially includes 1,316 data items.

[0020] (Data classification processing unit 22) The data classification processing unit 22 performs a data classification process to pick out data that are candidates for outliers from the data included in the learning data 31. The data classification process corresponds to a data classification step of the present invention.

[0021] In the data classification process, first, the coefficient of variation of the value of the objective variable is calculated for each piece of data included in the learning data 31. The coefficient of variation can be calculated by the following formula (1). (coefficient of variation) = {(standard deviation) / (mean value)} × 100 (1) Then, among the data included in the learning data 31, data having the calculated coefficient of variation greater than a set reference value is classified as outlier candidate data 32, and the remaining data is classified as normal data 33 and stored in the storage unit 3. A large coefficient of variation means that the value of the objective variable varies greatly, and it is considered that there is a high possibility that an outlier is included. Note that here, the outlier candidate data 32 and the normal data 33 are stored in the storage unit 3 as data separate from the learning data 31. However, this is not limiting, and for example, the outlier candidate data 32 and the normal data 33 may be distinguished from each other by attaching a flag or marker to each data item in the learning data 31. In other words, part of the learning data 31 may be classified as outlier candidate data 32 or normal data 33.

[0022] The reference value of the coefficient of variation for determining outlier candidates may be set as appropriate. For example, this reference value can be set taking into consideration the target objective variable, the overall variability of the data, etc., and can be the mean value, median, mode, mean value ±σ (σ is standard deviation), mean value ±2σ, mean value ±3σ, etc., of the coefficients of variation of all data included in the training data 31. Here, the coefficient of variation of representative formulation data, taking into consideration manufacturing performance, etc., among the data included in the training data 31, is used as the reference value.

[0023] FIG. 3(a) is a diagram showing an example of the calculation results of the coefficient of variation. In FIG. 3(a), the coefficient of variation was calculated for 1,316 pieces of data and plotted in ascending order. In the illustrated example, the standard value was set to 17.7 using the coefficient of variation for a representative blend. As shown in FIG. 3(a), in the illustrated example, there were 87 pieces of data that exceeded the standard value. In the data classification process, these 87 pieces of data are designated as outlier candidate data 32, and the remaining 1,229 pieces of data are designated as normal data 33.

[0024] Figure 3(b) is a histogram showing the calculation results of the coefficient of variation in Figure 3(a). Figure 3(b) also shows the mean, median, mean + σ, mean + 2σ, and mean + 3σ values ​​of the coefficient of variation of all data included in the training data 31. As shown in Figure 3(b), the value of 17.7 used as the reference value in this study was greater than mean + σ and less than mean + 2σ.

[0025] (Threshold setting processing unit 23) The threshold setting processing unit 23 performs a threshold setting process to set a threshold value of an evaluation index (referred to as an evaluation index threshold value) used to determine outliers. The threshold setting process corresponds to a threshold setting step of the present invention. In the threshold setting process, first, a part of the normal data 33 is used as test data, and the other part of the normal data 33 is used as first training data, and a first regression model showing the correlation between the explanatory variable and the response variable is created using the first training data. Then, the test data is applied to the created first regression model to calculate a first evaluation index (referred to as an evaluation index reference value) for evaluating the prediction accuracy of the first regression model. After that, an evaluation index threshold value for outlier determination is set based on the evaluation index reference value obtained by the calculation.

[0026] The evaluation index used for the evaluation index reference value (first evaluation index) is the mean error (ME), mean absolute error (MAE), root mean square error (RMSE), mean percent error (MPE), mean absolute error rate (MAPE), root mean square error rate (RMSPE), or coefficient of determination R 2 Here, MAPE was used.

[0027] In this embodiment, as shown in FIG. 4, multiple division patterns are formed to divide the normal data 33 into test data and data used as first training data. In this case, it is preferable that each data item included in the normal data 33 be used as test data at least once to prevent a decrease in evaluation accuracy due to bias in the test data. Considering that the number of data items in the normal data 33 is approximately 1,000, the normal data 33 is divided into four parts, and division patterns A to D are formed, in which each part is used as test data. However, the method for creating the division patterns is not limited to this. For example, the division may be performed completely randomly, or, for example, the division may be performed so that only one data item is used as test data. When dividing the data, it is preferable to divide the normal data 33 so that 20% to 30% of all data items are used as test data.

[0028] Then, for each of patterns A to D, a first regression model is created using the first training data, and the test data is applied to the first regression model to calculate the MAPE. The average value of the four MAPEs obtained for each of patterns A to D is then calculated as the evaluation index reference value (first evaluation index). In this embodiment, since there are multiple values ​​for the objective variable, it is preferable to use the median of the multiple objective variable values ​​when creating the first regression model and calculating the MAPE. This is because, even if outliers are included among the multiple objective variable values, the median is less susceptible to the influence of the outliers. Therefore, it is desirable to use the median, especially when the objective variable values ​​vary greatly. However, this is not limiting, and for example, the average value of the multiple objective variable values ​​may also be used.

[0029] Then, based on the obtained evaluation index reference value (average value of MAPE), the evaluation index threshold is set. The evaluation index reference value obtained in Figure 4 can be used as the evaluation index threshold as it is, or a value larger than the evaluation index reference value (coefficient of determination R as evaluation index) can be used to make a more conservative judgment. 24 is used), the evaluation index threshold may be set to a value 0.005% larger than the evaluation index reference value (the average value of MAPE obtained in FIG. 4). In this embodiment, the evaluation index reference value is used as the evaluation index threshold. Note that an appropriate evaluation index threshold differs depending on the target learning data 31, so it is advisable to set an appropriate value after consideration in advance. Alternatively, a configuration may be adopted in which multiple evaluation index thresholds are set in stages, and the evaluation index threshold can be selected taking into consideration the final detection result (the detection result of outliers, or the calculation result of the evaluation index when the detected outliers are excluded).

[0030] (Outlier determination processing unit 24) The outlier determination processing unit 24 performs an outlier determination process to determine whether each piece of data included in the outlier candidate data 32 is an outlier. The outlier determination process corresponds to the outlier determination step of the present invention. In the outlier determination process, some of the data included in the outlier candidate data 32 is added to the first teacher data to be used as second teacher data, and an evaluation index is calculated. Here, for each piece of data included in the outlier candidate data 32, the data is added to the first teacher data to be used as second teacher data. In other words, it is confirmed how the evaluation index (here, the MAPE) changes when some data (here, any one piece of data) included in the outlier candidate data 32 is added to the first teacher data. When certain data is added to the first teacher data, the prediction accuracy decreases compared to when no data is added (the MAPE increases, or the coefficient of determination R 2 becomes smaller), then the data added to the first training data set is likely to be an outlier.

[0031] More specifically, as shown in FIG. 5, the outlier determination processing unit 24 adds each piece of data included in the outlier candidate data 32 to the first training data to generate second training data. Next, a second regression model is created using this second training data. Furthermore, a second evaluation index (here, MAPE) for evaluating the prediction accuracy of the second regression model is calculated using test data for this second regression model. Here, as shown in FIG. 5, a second regression model is created and a MAPE is calculated for each of the division patterns A to D used in the threshold setting process, and the average value of the calculated MAPE is calculated as the second evaluation index. In creating the second regression model and calculating the second evaluation index, the median value of the values ​​of multiple objective variables is used, as in the threshold setting process. Furthermore, the same index is used as the first evaluation index (evaluation index threshold) and the second evaluation index.

[0032] Thereafter, the outlier determination processing unit 24 compares the calculated second evaluation index with the evaluation index threshold (first evaluation index) to determine whether the added data (data added to the first teacher data) is an outlier. Since MAPE is used here, when the calculated second evaluation index is greater than the evaluation index threshold, it is determined to be an outlier. Note that the coefficient of determination R 2 When the second evaluation index obtained by the calculation is smaller than the evaluation index threshold, the data is determined to be an outlier. The data determined to be an outlier is stored in the storage unit 3 as outlier data 35.

[0033] 6 is a diagram showing the results of calculating the second evaluation index (here, MAPE) for the 87 outlier candidate data 32 extracted in FIG. 3(a) when each data is added to the training data. In the illustrated example, the evaluation index for 31 data is greater than the evaluation index threshold, and these 31 data are determined to be outliers. Note that if multiple evaluation index thresholds are set in stages, it is recommended to store data detected as outliers so as to correspond to each evaluation index threshold.

[0034] (Detection result presentation processing unit 25) The detection result presentation processing unit 25 presents the determination result in the outlier determination processing unit 24, i.e., the outlier detection result. In the detection result presentation processing, the data detected as an outlier is presented to the user by, for example, displaying the data detected as an outlier on the display device 4.

[0035] For comparison, the detection result presentation processing unit 25 may calculate an evaluation index (such as MAPE) when all detected outlier data (data determined to be outliers in the outlier determination process) is excluded from the learning data 31 in the same manner as in Fig. 4, and present the calculation result. In this case, by calculating and presenting an evaluation index when the outliers are not excluded, it is possible to present the extent to which the prediction accuracy has improved by excluding the outliers.

[0036] In addition, when multiple evaluation index thresholds are set in stages, the evaluation index (e.g., MAPE) when each evaluation index threshold is adopted may be calculated and displayed. This allows the evaluation index threshold that minimizes the evaluation index to be selected, eliminating the need to set evaluation index thresholds in advance.

[0037] 7 shows the MAPE value calculated by excluding all data detected as outliers for an example in which the evaluation index threshold value in FIG. 6 is set to a value equal to the evaluation index reference value. In the example, 31 data were detected as outliers. FIG. 7 also shows a comparative example in which no outliers were excluded (including all 32 outlier candidate data), and the evaluation index reference value (the value calculated in FIG. 4) in which all outlier candidate data were excluded.

[0038] As shown in FIG. 7, it was confirmed that the example according to this embodiment had significantly improved prediction accuracy (decreased error rate) compared to the comparative example in which outliers were not excluded and the evaluation index reference value in which all outlier candidate data 32 were excluded. Furthermore, the evaluation index reference value in which all 87 data included in the outlier candidate data 32 were excluded had almost the same prediction accuracy as the comparative example in which outliers were not excluded. This is thought to be because the prediction accuracy was not sufficiently improved because not only outliers but also data that were not outliers were excluded. In the detection result presentation process, a graph such as that shown in FIG. 7 may be displayed on the display device 4 or the like.

[0039] (Outlier detection method) 8 is a flow diagram of the outlier detection method according to this embodiment. As shown in FIG. 8, in the outlier detection method according to this embodiment, first, in step S1, a data acquisition process is performed. In the data acquisition process, a data acquisition processing unit 21 acquires learning data 31 from an external device or the like. The acquired learning data 31 is stored in the storage unit 3.

[0040] Then, in step S2, a data classification process is performed. In the data classification process, as shown in Fig. 9, first, in step S21, the coefficient of variation of each piece of data included in the learning data 31 is calculated. Then, in step S22, data whose coefficient of variation is greater than a set reference value is stored in the storage unit 3 as outlier candidate data 32. Then, in step S23, data whose coefficient of variation is equal to or less than the reference value is stored in the storage unit 3 as normal data 33. Then, the process returns and proceeds to step S3 in Fig. 8.

[0041] In step S3, a threshold setting process is performed. In the threshold setting process, as shown in FIG. 10, first, in step S31, the normal data 33 is divided into test data and first training data. Here, the normal data 33 is divided into four parts, and four divided patterns, patterns A to D, are formed so that each part is used as test data (see FIG. 4). Then, in step S32, a first regression model is created for each of patterns A to D using the first training data. Then, in step S33, the test data is applied to the created first regression model for each of patterns A to D to calculate the MAPE. Then, in step S34, the average value of the MAPE calculated for each of patterns A to D is calculated, and the obtained average value is set as the evaluation index reference value (first evaluation index). Then, in step S35, an evaluation index threshold is set based on the evaluation index reference value. Here, the evaluation index threshold is set to a value equal to the evaluation index reference value. The set evaluation index threshold is stored in the storage unit 3 as evaluation index threshold data 34. Then, the process returns and proceeds to step S4 in FIG. 8.

[0042] In step S4, an outlier determination process is performed. In the outlier determination process, as shown in FIG. 11, first, in step S41, an initial value of 1 is assigned to n, a variable representing a data number, and the number of pieces of outlier candidate data 32 is assigned to n_max. Then, in step S42, a second regression model is created for each of patterns A to D using second training data obtained by adding the nth piece of data to the first training data. Then, in step S43, a MAPE is calculated for each of patterns A to D by applying test data to the created second regression model. Then, in step S44, the average value of the MAPE calculated for each of patterns A to D is calculated as a second evaluation index. Then, in step S45, it is determined whether the second evaluation index calculated in step S44 is greater than the evaluation index threshold set in step S3. If step S45 returns YES (Y), in step S46, the nth piece of data is determined to be an outlier and stored in the storage unit 3 as outlier data 35, and the process proceeds to step S48. If the determination in step S45 is NO (N), the nth data is determined not to be an outlier in step S47, and the process proceeds to step S48. In step S48, it is determined whether the variable n is equal to or greater than n_max. If the determination in step S48 is NO (N), the process returns to step S42 after incrementing the variable n in step S49. If the determination in step S48 is YES (Y), the process returns and proceeds to step S5 in FIG. 8.

[0043] In step S5, a detection result presentation process is performed. In the detection result presentation process, the detected outlier is presented by, for example, displaying the data determined to be an outlier in step S4, i.e., the outlier data 35, on the display 4. Thereafter, the process ends. Note that an outlier removal step may be included in which the detected outlier is removed from the training data 31.

[0044] (Actions and Effects of the Embodiments) As described above, the outlier detection method according to this embodiment includes a data classification step of determining the coefficient of variation of the value of the dependent variable for each piece of data included in the training data 31, and classifying, from among the data included in the training data 31, data whose determined coefficient of variation is greater than a set reference value as outlier candidate data 32 and the remaining data as normal data 33; a data classification step of using part of the normal data 33 as test data and the other part of the normal data 33 as first training data, creating a first regression model showing the correlation between the explanatory variable and the dependent variable using the first training data, and applying the test data to the created first regression model; and an outlier determination step of: calculating a first evaluation index (MAPE, etc.) for evaluating the prediction accuracy of the first regression model using the first evaluation index; and setting an evaluation index threshold for outlier determination based on the calculated first evaluation index; and adding each piece of data included in the outlier candidate data 32 to the first teacher data to make it second teacher data, creating a second regression model using the second teacher data, applying test data to the second regression model to calculate a second evaluation index, and comparing the calculated second evaluation index with the evaluation index threshold to determine whether the added data is an outlier.

[0045] By identifying data with a large coefficient of variation as outlier candidates, it is possible to easily narrow down the outlier candidates when the training data 31 includes multiple values ​​of the objective variable. Then, by checking the impact (change in MAPE) of each piece of outlier candidate data when it is added to the training data (first training data), it becomes possible to properly detect outliers. Then, by properly detecting outliers, it is possible to improve the prediction accuracy of physical properties, etc. using the training data 31 from which the detected outliers have been removed.

[0046] (Variation) Although not mentioned in the above embodiment, the outlier detection device 1 may be incorporated as a function into a prediction device that predicts physical properties and the like using training data 31. In this case, the prediction device will be equipped with a regression model creation unit that performs machine learning using the training data 31 from which outliers have been removed and creates a regression model that indicates the correlation between explanatory variables and target variables, and a prediction unit that predicts physical properties and the like using the regression model created by the regression model creation unit.

[0047] (Summary of the embodiment) Next, the technical ideas grasped from the above-described embodiments will be described by using the reference numerals and the like in the embodiments. However, the reference numerals and the like in the following description do not limit the components in the claims to the members and the like specifically shown in the embodiments.

[0048] [1] A method for detecting outliers included in training data (31) including data on explanatory variables and dependent variables used in machine learning, wherein each data included in the training data (31) includes a plurality of values ​​of the dependent variable, and a data classification step of determining a coefficient of variation of the value of the dependent variable for each piece of data included in the training data (31), and classifying data included in the training data (31) whose determined coefficient of variation is greater than a set reference value as outlier candidate data (32), and other data as normal data (33); and a first training data analysis step of determining a correlation between the explanatory variables and the dependent variable using the first training data. an outlier detection method comprising: a threshold setting step of creating a first regression model, calculating a first evaluation index for evaluating the prediction accuracy of the first regression model using the test data, and setting an evaluation index threshold for outlier determination based on the first evaluation index; and an outlier determination step of adding each piece of data included in the outlier candidate data (32) to the first training data to make it second training data, creating a second regression model using the second training data, calculating a second evaluation index for evaluating the prediction accuracy of the second regression model using the test data, and determining whether the added data is an outlier by comparing the second evaluation index with the evaluation index threshold.

[0049] [2] The outlier detection method according to [1], wherein in the threshold setting step, a plurality of division patterns are formed to divide the normal data (33) into the test data and the first teacher data, and the evaluation index threshold is set using an average value of the first evaluation index calculated for each division pattern, and in the outlier determination step, an average value of the second evaluation index calculated using the second teacher data created based on the first teacher data of each division pattern is used to compare with the evaluation index threshold.

[0050] [3] The outlier detection method according to [1], wherein the first and second evaluation indices are any of mean error (ME), mean absolute error (MAE), root mean square error (RMSE), mean percent error (MPE), mean absolute error rate (MAPE), and root mean square error rate (RMSPE), and the outlier determination step determines that the added data is an outlier when the calculated second evaluation index is greater than the evaluation index threshold.

[0051] [4] The outlier detection method according to [1], wherein a coefficient of determination is used as the first and second evaluation indexes, and in the outlier determination step, when the calculated second evaluation index is smaller than the evaluation index threshold, the added data is determined to be an outlier.

[0052] [5] The outlier detection method according to [1], wherein in the threshold setting step and the outlier determination step, a median value of the plurality of objective variables is used to create the first and second regression models and calculate the first and second evaluation indexes.

[0053] [6] A device for detecting outliers included in training data (31) including data on explanatory variables and dependent variables used in machine learning, wherein each data included in the training data (31) includes a plurality of values ​​of the dependent variable, and a data classification processing unit (22) for calculating a coefficient of variation of the value of the dependent variable for each piece of data included in the training data (31), and classifying data included in the training data (31) having the calculated coefficient of variation greater than a set reference value as outlier candidate data (32), and other data as normal data (33); and a first regression model for classifying a correlation between the explanatory variables and the dependent variable using the first training data, the first regression model being configured to use a part of the normal data (33) as test data and the other part of the normal data (33) as first training data. an outlier detection device (1) comprising: a threshold setting processing unit (23) that creates a first regression model using the test data, calculates a first evaluation index for evaluating the prediction accuracy of the first regression model, and sets an evaluation index threshold for outlier determination based on the first evaluation index; and an outlier determination processing unit (24) that, for each of data included in the outlier candidate data (32), adds the respective data to the first teacher data to make the data second teacher data, creates a second regression model using the second teacher data, calculates a second evaluation index for evaluating the prediction accuracy of the second regression model using the test data, and determines whether the added data is an outlier by comparing the second evaluation index with the evaluation index threshold.

[0054] (Addendum) Although the embodiments of the present invention have been described above, the invention according to the claims is not limited to the above-described embodiments. It should be noted that not all of the combinations of features described in the embodiments are necessarily essential to the means for solving the problems of the invention. Furthermore, the present invention can be appropriately modified and implemented within the scope of its spirit. [Explanation of symbols]

[0055] 1...Outlier detection device 2...Control unit 21...Data acquisition processing unit 22...Data classification processing section 23...Threshold setting processing section 24...Outlier detection processing unit 25...Detection result presentation processing unit 3...Storage section 31...Learning data 32...Outlier candidate data 33...Normal data 34...Evaluation index threshold data 35...Outlier data

Claims

1. A method for detecting outliers contained in learning data including data on explanatory variables and objective variables used in machine learning, comprising: each data included in the learning data includes a plurality of values ​​of the objective variable; a data classification step of calculating a coefficient of variation of the value of the objective variable for each piece of data included in the training data, and classifying, among the pieces of data included in the training data, data whose calculated coefficient of variation is greater than a set reference value as candidate outlier data, and other data as normal data; a threshold setting step of using a part of the normal data as test data and another part of the normal data as first teacher data, creating a first regression model showing a correlation between the explanatory variable and the dependent variable using the first teacher data, calculating a first evaluation index for the first regression model using the test data, and setting an evaluation index threshold for outlier determination based on the first evaluation index; an outlier determination step of adding each piece of data included in the outlier candidate data to the first teacher data to form second teacher data, creating a second regression model using the second teacher data, calculating a second evaluation index for the second regression model using the test data, and comparing the second evaluation index with the evaluation index threshold to determine whether the added data is an outlier. Outlier detection methods.

2. In the threshold setting step, a plurality of division patterns are formed to divide the normal data into the test data and the first teacher data, and the evaluation index threshold is set using an average value of the first evaluation index calculated for each division pattern; In the outlier determination step, an average value of the second evaluation index calculated using the second teacher data created based on the first teacher data of each of the division patterns is compared with the evaluation index threshold value. The outlier detection method according to claim 1 .

3. As the first and second evaluation indexes, any one of mean error (ME), mean absolute error (MAE), mean squared error (RMSE), mean percent error (MPE), mean absolute error (MAPE), and mean squared error ratio (RMSPE) is used; In the outlier determination step, when the calculated second evaluation index is greater than the evaluation index threshold, the added data is determined to be an outlier. The outlier detection method according to claim 1 .

4. The first and second evaluation indexes are determined using a coefficient of determination, In the outlier determination step, when the calculated second evaluation index is smaller than the evaluation index threshold, the added data is determined to be an outlier. The outlier detection method according to claim 1 .

5. In the threshold setting step and the outlier determination step, a median value of the plurality of dependent variable values ​​is used for creating the first and second regression models and calculating the first and second evaluation indexes. The outlier detection method according to claim 1 .

6. An apparatus for detecting outliers contained in learning data including data on explanatory variables and objective variables used in machine learning, each data included in the learning data includes a plurality of values ​​of the objective variable; a data classification processing unit that calculates a coefficient of variation of the value of the objective variable for each piece of data included in the learning data, and classifies, among the pieces of data included in the learning data, data whose calculated coefficient of variation is greater than a set reference value as outlier candidate data, and classifies the other pieces of data as normal data; a threshold setting processing unit that uses a part of the normal data as test data and another part of the normal data as first teacher data, creates a first regression model that indicates a correlation between the explanatory variable and the dependent variable using the first teacher data, calculates a first evaluation index for the first regression model using the test data to evaluate the prediction accuracy of the first regression model, and sets an evaluation index threshold for outlier determination based on the first evaluation index; an outlier determination processing unit that, for each piece of data included in the outlier candidate data, adds the respective piece of data to the first teacher data to form second teacher data, creates a second regression model using the second teacher data, calculates a second evaluation index for the second regression model using the test data, and compares the second evaluation index with the evaluation index threshold to determine whether the added data is an outlier, Outlier detector.

Images