Data preparation support apparatus for machine learning, data preparation apparatus for machine learning, data preparation support method for machine learning, and data preparation support program for machine learning

Abstract

To provide a data preparation support apparatus for machine learning that enables a user to easily check feature quantities for machine learning in advance.SOLUTION: A data preparation support apparatus 10 comprises: a reference acquisition unit 11 that acquires, via a network 5, reference values of statistical quantities for each of an objective variable and explanatory variables specified by a user; a statistical quantity calculation unit 13 that calculates statistical quantities of feature quantity data 22 including a plurality of records each including the objective variable and the explanatory variables; a statistical quantity determination unit 14 that determines presence or absence of an abnormality in the feature quantity data 22 by comparing the statistical quantities of the feature quantity data 22 with the reference values; and a feature quantity evaluation unit 15 that determines presence or absence of an abnormality for each record of the feature quantity data 22 when the statistical quantity determination unit 14 determines that an abnormality is present.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a machine learning data preparation support device, a machine learning data preparation device, a machine learning data preparation support method, and a machine learning data preparation support program. [Background technology]

[0002] Artificial intelligence such as machine learning is rapidly evolving, with particularly remarkable technological developments in the field of image processing, and further applications in fields such as manufacturing and medicine. As a preprocessing step for machine learning, features, which are data corresponding to a machine learning model, are extracted or generated from input data (raw data), and these features are input into the machine learning model to execute machine learning (see, for example, Patent Document 1).

[0003] Input data may contain outliers or abnormal values, and using features extracted directly from such data in a machine learning model may result in inaccurate results. For this reason, technologies have been developed to prepare appropriate machine learning data from input data. For example, the technology disclosed in Patent Document 2 selects features from records that do not contain bias features from input data containing multiple records, trains a machine learning model with a relatively low processing load using the selected features, and then calculates the fairness and accuracy of the trained machine learning model for the selected features. This allows features with high fairness and accuracy to be used in a full-scale machine learning model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7068085 [Patent Document 2] Japanese Patent Application Publication No. 2023-37242 Summary of the Invention [Problem to be solved by the invention]

[0005] Machine learning can take days to complete. If the results show that the features used are inadequate, the features must be corrected and the process restarted, resulting in a significant loss of time. However, it is difficult for users to check the features of a huge amount of data.

[0006] Therefore, an object of the present invention is to provide a machine learning data preparation support device, a machine learning data preparation support method, and a machine learning data preparation support program that make it easy for users to check features for machine learning in advance. [Means for solving the problem]

[0007] In order to solve the above problem, a data preparation support device for machine learning of the present invention includes: a standard acquisition unit that acquires standard values ​​of statistics for the objective variable and standard values ​​of statistics for the explanatory variables based on an objective variable and explanatory variables specified by a user; a statistics calculation unit that calculates standard values ​​of statistics for a plurality of records for the objective variable and standard values ​​of statistics for the explanatory variables based on a plurality of records, each of which includes the objective variable and the explanatory variables; a statistics determination unit that determines the presence or absence of an abnormality in the plurality of records by comparing the standard values ​​of the statistics for the plurality of records; and a feature evaluation unit that, when the statistics determination unit determines the presence or absence of an abnormality, determines the presence or absence of an abnormality for each record of the plurality of records based on the comparison result by the statistics determination unit. Furthermore, a data preparation device for machine learning of the present invention includes the data preparation support device for machine learning and a feature generation unit that receives input data and outputs the plurality of records.

[0008] The machine learning data preparation support method of the present invention includes: a reference acquisition process that acquires reference values ​​for statistics for a target variable and reference values ​​for statistics for the explanatory variables based on a target variable and explanatory variables specified by a user; a statistics calculation process that calculates statistics for a plurality of records for the target variable and statistics for a plurality of records for the explanatory variables based on a plurality of records, each of which includes the target variable and the explanatory variables; a statistics determination process that determines whether or not there is an abnormality in the plurality of records by comparing the statistics for the plurality of records with the reference values; and, if it is determined that there is an abnormality in the plurality of records, a feature evaluation process that determines whether or not there is an abnormality for each of the plurality of records based on the comparison result in the statistics determination process.

[0009] The machine learning data preparation support program of the present invention causes a computer to execute the following steps: inputting a response variable and explanatory variables specified by a user, and acquiring reference values ​​for statistics for the response variable and reference values ​​for statistics for the explanatory variables; inputting a plurality of records, each including the response variable and the explanatory variables, and calculating statistics for the plurality of records for the response variable and statistics for the plurality of records for the explanatory variables; determining whether or not the plurality of records contain an abnormality by comparing the statistics for the plurality of records with the reference values; and, if it is determined that the plurality of records contain an abnormality, determining whether or not each of the plurality of records contains an abnormality based on a comparison result of the statistics for the plurality of records with the reference values. [Effects of the Invention]

[0010] According to the present invention, it is possible to avoid failure of machine learning due to low-accuracy data and reduce time loss. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram of a machine learning data preparation assistance device according to an embodiment of the present invention. [Figure 2]1 is an example of a system setting file to be input to a machine-learning data preparation support device according to an embodiment of the present invention. [Figure 3] 1 is an example of feature data to be input to a machine-learning data preparation support device according to an embodiment of the present invention. [Figure 4] 10 is a diagram illustrating an example of a basis and an external source for a reference value of a statistical quantity output by the machine-learning data preparation assistance device according to an embodiment of the present invention. [Figure 5] 10 is an example of table data comparing statistical quantities output by the machine-learning data preparation assistance device according to the embodiment of the present invention with their reference values. [Figure 6] 3 is table data in which the feature amount data shown in FIG. 2 is accompanied by evaluation results of the feature amounts. [Figure 7] 1 is an example of a list of features including abnormal values ​​output by the machine-learning data preparation support device according to an embodiment of the present invention. [Figure 8] 10 is an example of table data comparing statistical quantities output by the machine-learning data preparation assistance device according to the embodiment of the present invention with their reference values. [Figure 9] 1 is a flowchart illustrating a machine learning data preparation support method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, modes for carrying out the present invention (hereinafter referred to as "embodiments") will be described with reference to the drawings as appropriate. Elements of the same or similar structure will be given the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0013] [Data preparation support device] FIG. 1 is a block diagram of a machine learning data preparation support device (hereinafter, referred to as the data preparation support device) 10 according to an embodiment of the present invention. The data preparation support device 10 includes a functional unit 1, a memory unit 2, a connection interface (IF) 31, and a network interface (IF) 32. The functional unit 1 includes a reference acquisition unit 11, a reliability calculation unit 12, a statistics calculation unit 13, a statistics determination unit 14, a feature evaluation unit 15, and a UI display processing unit 16. To accommodate these elements of the functional unit 1, as well as the connection interface 31 and the network interface 32, the data preparation support device 10 includes a central processing unit (CPU), read-only memory (ROM), random access memory (RAM), and other memory, and performs processing by software programs stored in the memory. The memory unit 2 is a storage device that stores data input by a user, data acquired from an external device, data output by the functional unit 1, and the like, and is, for example, a high-speed storage device such as a solid-state drive (SSD). The connection interface 31 is an element for connecting to peripheral devices such as an input device 41 and an output device 42. The network interface 32 is a communication device that connects to the network 5 and transmits and receives data via the network 5. An information terminal such as a personal computer (PC) can be used as the data preparation support device 10. In addition, part or all of the storage unit 2 of the data preparation support device 10 may be provided in a server on the network 5. In this specification, the data preparation support device 10 checks features for a machine learning model that predicts the likelihood of an injury or illness occurring in a patient from the test data.

[0014] The following describes the system setting file 21 and feature amount data 22, which are among the data stored in the storage unit 2. These are input by user operations. The system setting file 21 indicates the name of the disease or injury whose likelihood of onset is predicted by the machine learning model, factors and drug effects of the disease or injury, and the priorities among these, and is created in advance. In this specification, as shown in FIG. 2 as an example, the system setting file 21 is table data including definitions 21a, user settings 21b, and their priorities 21c. In this embodiment, the definitions are, in descending order of priority, disease or injury, drug effect category, and factor. The disease or injury is diabetes, the drug effect category is prescription of diabetes medication, and the factors are, in descending order of priority, HbA1c value, fasting blood glucose, urinary glucose, and BMI (Body Mass Index) value. Priorities may be equal among two or more definitions, or may not be set.

[0015] The feature data 22 is input data for machine learning and is the target of evaluation by the data preparation support device 10. The feature data 22 is extracted in advance from the patient's test data, which is raw data, and generated in a format corresponding to the machine learning model. As shown in FIG. 3, the feature data 22 is table data including an ID (identification) represented by a four-digit number that identifies the patient, whether or not the patient has developed an illness or injury, the patient's attributes (gender, age group), test data, and whether or not the patient has received treatment. In this example, the illness or injury is diabetes, and whether or not the patient has developed the illness is represented by a binary value (1: yes, 0: no). The test data includes HbA1c (hemoglobin A1c) value, fasting plasma glucose (FPG) and urine sugar (not shown), and whether or not the patient has received treatment is whether or not the patient has been prescribed an antidiabetic drug. In the feature amount data 22, items are provided for gender (male / female), age and test data (by stage), and whether or not a diabetes medication has been prescribed (by the time (year) of prescription within the past four years), with "1" entered for applicable items and "0" for non-applicable items. The ID and whether or not an injury or illness has occurred in the feature amount data 22 are dependent variable data 22o, and the ID, gender, age, test data, and whether or not treatment has been received are explanatory variable data 22e. In FIG. 3, the dependent variable data 22o and explanatory variable data 22e are represented in one table data (feature amount data 22), and IDs are displayed in columns on both ends for ease of understanding, but the dependent variable data 22o and explanatory variable data 22e may be separate table data.

[0016] The reference acquisition unit 11 acquires reference values ​​for statistics for each item in the system setting file 21 from data stored in an external storage device such as a server via the network 5. In more detail, the reference acquisition unit 11 searches for data stored in a server or the like using a search engine based on the user settings 21b in the system setting file 21, and extracts reference values ​​for statistics from the detected data (external sources). The reliability calculation unit 12 calculates the reliability of the external sources detected by the reference acquisition unit 11 based on the number of searches (search volume) by the search engine. The number of searches can be checked using a tool installed on the network 5, just like the search engine.

[0017] Figure 4 shows the basis for the reference value of a statistic, the external source on which it is based, and its reliability. Figure 4 shows the basis 23a for the reference value of a statistic, the external source 23b, and its reliability 23c in addition to the definition 21a and user setting 21b in the system setting file 21 (Figure 3). Definition: The statistic for illness is the incidence rate of illness (diabetes) based on the objective variable. The reference value for the statistic for the incidence rate of diabetes is calculated based on the external source of 19.7% for men and 10.8% for women. For simplicity, 15% of the average of the two is used as the reference value for the statistic. Definition: The statistic for therapeutic classification is the prescription rate of diabetes medication, and the reference value is calculated based on the external source that indicates an HbA1c value of 7.0% or higher as the prescription guideline. Definition: The statistic for factor is the proportion of patients with a factor whose value indicates suspected diabetes. The calculated reference value 23r for the statistic is shown in Figure 5. Here, since the feature data 22 (explanatory variable data 22e) does not include a BMI value, a reference value for the BMI statistical value is not calculated. Furthermore, the reliability 23c of the external source is ranked in descending order as S, A, B, C, etc. In principle, external sources ranked B or higher are used, and if multiple external sources ranked A or higher are detected, the two with the highest reliability are used.

[0018] The statistical quantity calculation unit 13 calculates the statistical quantities of the objective variable data 22o and explanatory variable data 22e of the feature quantity data 22. The statistical quantity determination unit 14 compares the statistical quantities of the feature quantity data 22 with the reference values ​​extracted by the reference acquisition unit 11, determines whether they are normal values, and outputs the result. If the statistical quantities are determined to be abnormal values, an alert is issued along with the output of the result. The determination criteria for the statistical quantities of the feature quantity data 22 are set in advance, for example, so that the difference Δ from the reference value is within ±10 points. Furthermore, the threshold for the difference Δ may be set narrower when the reliability of the external source on which the reference value is based is high, and wider when it is low. The statistical quantities and their reference values ​​may also be calculated and determined, for example, by patient gender or age group.

[0019] FIG. 5 shows table data 241 including statistics 24a, reference values ​​23r, and differences Δ of the feature data 22. The statistics of the feature data 22 are the incidence rate of diabetes, the prescription rate of diabetes medication, and the percentage of patients in each range of appropriate, caution, and suspected diabetes for HbA1c values, fasting blood glucose, and urine glucose. Furthermore, as an alert, for example, characters or fields indicating abnormal values ​​are displayed with a marking such as a different color from others (in FIG. 5, they are displayed with a bold frame). Here, a difference Δ within ±10 points is determined to be normal. As shown in FIG. 5, the feature data 22 has determined that the prescription rate of diabetes medication and the percentage of HbA1c values ​​of 6.5% or higher (suspected diabetes) are abnormal. Furthermore, if the statistics determining unit 14 determines that the statistics include abnormal values, it may display an external source (FIG. 4) of the reference value for the item related to the abnormal value as evidence along with the table data 241.

[0020] When the statistical quantity determination unit 14 determines that a statistical quantity of the feature data 22 is an abnormal value, the feature quantity evaluation unit 15 determines whether the objective variable and explanatory variable of each feature (record) of the feature data 22 are abnormal, and extracts records that are suspected of having an abnormal value. To do this, the feature quantity evaluation unit 15 comprehensively evaluates the objective variable and explanatory variable of the feature quantity based on the determination result by the statistical quantity determination unit 14. As described above, in the determination of the statistical quantities of the feature data 22 by the statistical quantity determination unit 14, the prescription rate of diabetes medication and the proportion of HbA1c values ​​of 6.5% or more are significantly higher than the reference values. Therefore, for example, the feature quantity that indicates the absence of diabetes but the prescription of diabetes medication and / or the HbA1c value of 6.5% or more is determined to be abnormal. Such determination criteria may be configured to be set by the feature quantity evaluation unit 15 based on the determination result by the statistical quantity determination unit 14, or may be configured to be set and input by the user based on table data 241 shown in FIG. 5.

[0021] FIG. 6 shows the feature evaluation result 221 output by the feature evaluation unit 15. In FIG. 6, records of the feature data 22 shown in FIG. 3 that indicate no onset of diabetes, prescription of antidiabetic medication, and / or an HbA1c value of 6.5% or higher are marked with a bold frame and are considered to be abnormal. Furthermore, as shown in FIG. 7, records containing suspected abnormal values ​​are extracted from the feature data 22 and listed in descending order of the degree of suspicion. Specifically, if a priority is set in the system configuration file 21 (FIG. 2), records with prescription of antidiabetic medication are listed higher than records without prescription based on the priority. Furthermore, the more recently the antidiabetic medication was prescribed and the higher the HbA1c value, the higher the record is listed. Furthermore, records corresponding to items with a high reliability of the external source (FIG. 4) on which the reference value is based may be listed higher.

[0022] The user checks each record in the feature list 242 shown in Fig. 7, corrects any clerical errors, and removes the record from the feature data 22 if it is determined to be an outlier. The user checks, for example, for clerical errors in the program for generating the feature data 22 from the raw data or in the raw data. By displaying records that are highly suspected of being outliers at the top, the user can efficiently check the feature data 22. The data preparation support device 10 re-inputs the feature data 22 corrected by the user, and the statistics calculation unit 13 calculates statistics, which are then judged by the statistics determination unit 14.

[0023] Fig. 8 shows table data 243 including the statistical quantities 24b, the reference values ​​23r, and the differences Δ of the feature amount data 22. In Fig. 8, none of the statistical quantities deviates significantly from the reference values, and it is determined that the feature amount data 22 does not include any abnormal values.

[0024] The UI display processing unit 16 converts the outputs of the statistical quantity determination unit 14 and the feature quantity evaluation unit 15 into a format (see FIGS. 5, 7, and 8) for display on the output device 42. The input device 41 is a well-known input device such as a keyboard or a mouse operated by a user of the data preparation support device 10. The output device 42 is a well-known display device such as a liquid crystal display that displays table data 241, 242, 243, etc., or a speaker.

[0025] [Data preparation support method] A data preparation support method by the data preparation support device 10 will be described with reference to Fig. 9. Fig. 9 is a flowchart illustrating a machine learning data preparation support method according to an embodiment of the present invention.

[0026] (Standard acquisition processing, statistical calculation processing) First, a system setting file 21 (FIG. 2) is input by a user operation (step S11), and the reference acquisition unit 11 extracts reference values ​​for each user-set statistic from an external source acquired from an external server or the like (steps S12 and S13). At this time, the reliability calculation unit 12 may calculate the reliability of the external source. Next, feature data 22 (FIG. 3) is input by a user operation (step S21), and the statistics calculation unit 13 calculates the statistics of each of the objective variable data 22o and explanatory variable data 22e of the feature data 22 (step S22). Note that steps S11 to S13 and steps S21 to S22 can be executed in reverse order or in parallel.

[0027] (Statistical value determination processing) The statistics determination unit 14 compares the statistics calculated in step S22 with the corresponding reference values ​​to determine whether or not there is an abnormality (step S31). If there is no abnormality in any of the statistics of the feature amount data 22 (step S31: YES), the statistics (FIG. 8) are output (step S32) and the process is completed. If there is an abnormal statistical amount included (step S31: NO), the statistics are output together with an alert (FIG. 5) (step S33), and the processes of steps S41 to S44 are executed.

[0028] (Feature evaluation processing) The feature evaluation unit 15 selects one feature (record) from the feature data 22 (step S41), comprehensively evaluates the objective variable and explanatory variables, and determines whether or not there is an abnormal value (step S42). If the feature evaluation unit 15 determines that the feature includes an abnormal value (step S42: NO), it extracts the feature from the feature data 22 (step S43). After evaluating all features in the feature data 22 (step S44), the feature evaluation unit 15 ranks the features extracted in step S43 (step S45), attaches an alert to any abnormal values, and outputs a feature list 242 ( FIG. 7 ) (step S46), completing the process.

[0029] If the user modifies the feature amount data 22 based on the feature amount list 242 output in step S46, the process can be repeated from step S21 where the feature amount data 22 is input.

[0030] [Data preparation device for machine learning] The data preparation support device 10 (FIG. 1) may be installed in a machine learning data preparation device that prepares machine learning data (feature data). Such a machine learning data preparation device includes a feature generation unit that generates feature data 22 from raw data input by a user. The feature generation unit may be configured to generate, from the raw data, the feature data 22 including the objective variable and explanatory variables defined in the system configuration file 21, based on the system configuration file 21.

[0031] The present invention is not limited to the above-described embodiments, and includes various modifications and applications without departing from the spirit of the present invention as set forth in the claims. For example, the above-described embodiments have been described in detail to aid in understanding the present invention, and the present invention is not necessarily limited to those having all of the described configurations.

[0032] The above-described configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware such as an integrated circuit. The above-described configurations and functions may also be realized by software, with a processor interpreting and executing a program that realizes each function. Information such as the program, table, and file that realizes each function can be stored in a recording device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as a flash memory card or a DVD (Digital Versatile Disk). [Explanation of symbols]

[0033] 10 Data preparation support device (machine learning data preparation support device) 11 Standard Acquisition Department 12 Reliability calculation unit 13 Statistics calculation section 14 Statistics judgment section 15 Feature evaluation unit 16 UI display processing section 21 System Configuration Files 22 Feature data (objective variables, explanatory variables) S12~S13 Reference acquisition process S22 Statistical calculation processing S31 Statistical value judgment processing S41~S44 Feature evaluation processing

Claims

1. a standard acquisition unit that acquires standard values ​​of statistics for the objective variable and standard values ​​of statistics for the explanatory variables based on the objective variable and explanatory variables designated by a user; a statistics calculation unit that calculates statistics of a plurality of records for the objective variable and statistics of a plurality of records for the explanatory variables based on a plurality of records each including the objective variable and the explanatory variables; a statistical quantity determination unit that determines whether or not there is an abnormality in the plurality of records by comparing the statistical quantities of the plurality of records with the reference value; and a feature evaluation unit that, when the statistical quantity determination unit determines that an abnormality exists, determines the presence or absence of an abnormality for each record of the plurality of records based on the comparison result by the statistical quantity determination unit.

2. a user designates a priority between the objective variable and the explanatory variables; The machine learning data preparation support device according to claim 1 , wherein the feature evaluation unit ranks and outputs records determined to contain an abnormality based on the priority of the objective variable and / or explanatory variable determined to be abnormal in the record.

3. the reference acquisition unit searches for information stored in an external storage device using a search engine to detect information including the reference value; The machine learning data preparation support device according to claim 1 , further comprising a reliability calculation unit that calculates the reliability of the information detected by the criteria acquisition unit based on the number of searches for the information.

4. 3. A machine learning data preparation device comprising: the machine learning data preparation support device according to claim 1; and a feature generation unit that receives data as input and outputs the plurality of records.

5. a reference acquisition process for acquiring reference values ​​of statistics for the objective variable and the explanatory variables based on the objective variable and the explanatory variables designated by a user; a statistics calculation process for calculating statistics of a plurality of records for the objective variable and statistics of a plurality of records for the explanatory variables based on a plurality of records each including the objective variable and the explanatory variables; a statistical quantity determination process for determining whether or not the plurality of records are abnormal by comparing the statistical quantities of the plurality of records with the reference value; a feature evaluation process for determining the presence or absence of an abnormality for each of the plurality of records based on the comparison results in the statistical determination process when the plurality of records are determined to have an abnormality.

6. On the computer, a step of inputting a response variable and an explanatory variable designated by a user and acquiring reference values ​​of statistics for the response variable and the explanatory variables; a step of receiving a plurality of records each including the response variable and the explanatory variables, and calculating statistics of the plurality of records for the response variable and statistics of the plurality of records for the explanatory variables; a step of determining whether or not there is an abnormality in the plurality of records by comparing the statistical values ​​of the plurality of records with the reference values; a step of determining whether or not each of the plurality of records has an abnormality based on a comparison result between the statistics of the plurality of records and the reference value when the plurality of records are determined to have an abnormality; A data preparation support program for machine learning that runs

Images