Information processing system and information processing method

The system generates multiple weak learners and calculates cutoff values to optimize ensemble calculations for individual samples, addressing prediction robustness issues across facilities with diverse data characteristics.

JP2024157876APending Publication Date: 2024-11-08HITACHI HIGH TECH CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023072502
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-26
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing predictive models generated from data of a single facility may lose robustness when applied to data from other facilities due to differences in data characteristics, such as gender, age structure, and incidence rate, leading to weakened prediction accuracy.

Method used

An information processing system that generates multiple weak learners from different training data groups, calculates cutoff values for each learner, and performs ensemble calculations based on these values to enhance prediction robustness.

Benefits of technology

The system achieves robust predictions even with varying data characteristics across facilities by optimizing ensemble calculations for individual samples, improving discrimination between positive and negative outcomes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157876000001_ABST
    Figure 2024157876000001_ABST
Patent Text Reader

Abstract

To provide an information processing system capable of making robust predictions even when there are differences in data characteristics between facilities.SOLUTION: An information processing system according to the present invention is configured to have: a weak learner generation unit that generates, from a plurality of data groups for learning collected from different targets related to an analysis target, a plurality of weak learners corresponding to the respective targets; a cutoff value calculation unit that calculates, from the plurality of data groups for learning, cutoff values for prediction values in relation to the plurality of weak learners corresponding thereto, respectively; and an ensemble calculation unit that compares and analyzes prediction values calculated by the plurality of generated weak learners using prediction data of the analysis target, and the corresponding cutoff values, and calculates a prediction value of the analysis target according to the results.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing system that performs prediction by ensemble calculation using weak learners. [Background technology]

[0002] In machine learning, if there are differences in the data characteristics of the target population between facilities, applying a predictive model trained on data from a single facility to data from other facilities may weaken the robustness of the prediction. In such cases, the predictive models generated from the data from each facility may be treated as weak learners, and ensemble calculations may be performed on the predicted values ​​of each weak learner.

[0003] Patent Document 1 describes a method for estimating the probability of a rare event with high accuracy even with a small data set by performing ensemble calculations using information on gate ensemble weights for the predicted values ​​of each weak learner. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-174330 A Summary of the Invention [Problem to be solved by the invention]

[0005] In order to improve the robustness of prediction, it is necessary to perform ensemble calculations appropriate for each sample, taking into account the characteristics of each weak learner. For example, when predicting the presence or absence of disease onset, such as positive or negative, there are differences between facilities in data characteristics such as gender, age composition, and incidence rate, so depending on the facility, weak learners with different characteristics may be generated, such as weak learners that are good at predicting positives and weak learners that are good at predicting negatives.

[0006] Therefore, in prediction, rather than uniformly performing ensemble calculations such as the average value on the predicted values ​​of each weak learner, it is desirable to perform ensemble calculations according to each individual sample, such as adopting the predicted value of a weak learner with excellent positive predictions for samples with a high probability of being positive. [Means for solving the problem]

[0007] In order to solve the above problems, the information processing system of the present invention is configured to include a weak learner generation unit that generates a plurality of weak learners corresponding to each of the targets from a plurality of learning data groups collected from different targets related to the target of analysis, a cutoff value calculation unit that calculates cutoff values ​​of predicted values ​​for the plurality of weak learners corresponding to each of the plurality of learning data groups from the plurality of learning data groups, and an ensemble calculation unit that compares and analyzes predicted values ​​calculated using the prediction data of the target of analysis with the corresponding cutoff values ​​using the generated plurality of weak learners, and calculates a predicted value of the target of analysis based on the result of the comparison. Effect of the Invention

[0008] According to one aspect of the present invention, predictions based on ensemble calculations using weak learners can be made robustly even when there are differences in data characteristics between facilities. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a hardware configuration of an information processing system according to an embodiment of the present invention. [Figure 2A] 2 is a diagram illustrating a configuration of data stored in a learning data storage unit 112 according to the embodiment of the present invention. FIG. [Figure 2B] 10 is a diagram illustrating a configuration of data stored in a prediction data storage unit 113 according to the embodiment of the present invention. FIG. [Diagram 3] 1 is a flowchart of a weak learner generation process according to an embodiment of the present invention. [Figure 4] 13 is a flowchart of a cutoff value calculation process according to an embodiment of the present invention. [Diagram 5]11 is an example of a cutoff value created in the cutoff value calculation process according to the embodiment of the present invention. [Figure 6] 1 is a flowchart of an ensemble calculation process according to an embodiment of the present invention. [Figure 7A] 1 shows an ensemble calculation result created in the ensemble calculation process according to an embodiment of the present invention. [Figure 7B] FIG. 2 is a diagram illustrating an ensemble calculation processing method according to an embodiment of the present invention on the XY axis. [Figure 8] 11 is a flowchart of an output process according to the embodiment of the present invention. [Figure 9] 11 is an example of a display screen output in the output process according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Overview of the embodiment> In the embodiment of the present invention, a case will be described as an example in which a weak learner is generated by learning data from two facilities and prediction is performed as a two-class classification problem such as the presence or absence of disease onset. Each weak learner calculates a cutoff value that maximizes classification performance, and performs ensemble calculations according to the individual sample data and the prediction values ​​of the weak learners based on the cutoff value. EXAMPLES

[0011] 1 is a block diagram showing a hardware configuration of an information processing system according to this embodiment, and the information processing system includes a server 101 and a database 102. The server 101 and the database 102 are connected so that the server 101 can access data stored in the database 102.

[0012] The server 101 is a computer having an input device 103, an output device 104, an arithmetic device 105 that executes programs, a memory 106 that stores the programs, and a storage device 107. Here, the input device 103 is a mouse, a keyboard, or the like, and is an interface that accepts input to the server 101. The output device 104 is a display device, a printer, or the like, and outputs the results of arithmetic operation by the arithmetic device 105.

[0013] The arithmetic device 105 is a CPU, a GPU, or the like, and executes a program loaded into the memory 106. The memory 106 includes a ROM, which is a non-volatile storage element, and a RAM, which is a volatile storage element. The ROM stores immutable programs (e.g., BIOS) and the like. The RAM is a high-speed, volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores the programs stored in the storage device 107 and data used when the programs are executed.

[0014] The storage device 107 is a non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD), and stores the programs executed by the arithmetic device 105 and data used when the programs are executed.

[0015] Specifically, the storage device 107 stores programs for implementing each of the weak learner generation unit 108, the cutoff value calculation unit 109, the ensemble calculation unit 110, and the output unit 111. Here, the weak learner generation unit 108 generates a prediction model serving as a weak learner from the learning data of each facility by executing a predetermined program, as shown in Fig. 3 described later. As shown in FIG. 4, which will be described later, the cutoff value calculation unit 109 calculates a cutoff value for each weak learner by executing a predetermined program. As shown in FIG. 6, which will be described later, the ensemble calculation unit 110 executes a predetermined program to calculate an ensemble calculation result according to the predicted value and the cutoff value of each weak learner. As shown in FIG. 8, which will be described later, the output unit 111 outputs a predicted value for the calculation data by executing a predetermined program.

[0016] The database 102 stores data to be analyzed by the server 101, that is, a learning data storage unit 112 (see FIG. 2A) and a prediction data storage unit 113 (see FIG. 2B). FIG. 2A is a diagram for explaining the configuration of the learning data of each facility stored in the learning data storage unit 112 in this embodiment. The learning data includes data of an analysis subject ID 201, variables A202 to D205, and variable Y206. Here, the analysis subject ID 201 is an identifier that uniquely identifies the analysis subject. The variables A202 to D205 are explanatory variables related to the analysis subject, such as the gender, age group, and various test results of the analysis subject. In addition, the variable Y206 is an objective variable, and for example, when the presence or absence of disease onset is used as the objective variable, if the analysis subject does not develop a disease, the result is negative, so 0, and if the analysis subject develops a disease, the result is positive, so 1. Note that in this embodiment, since two different facilities are assumed, the learning data shown in FIG. 2A may be prepared in different tables for each facility, or the facility name or the like may be added to one table so that the difference between the facilities can be understood.

[0017] 2B is a diagram for explaining the configuration of prediction data used for predicting an analysis subject, stored in the prediction data storage unit 113 of this embodiment. Here, the analysis subject ID 201B is an identifier that uniquely identifies the analysis subject. Variables A202B to D205B are the gender and age group of the analysis subject, various test results, etc., and can be input values ​​when performing prediction processing in this embodiment. Also, it differs from the learning data storage unit 112 (FIG. 2A) in that the contents of the table do not have an item equivalent to the objective variable Y206.

[0018] FIG. 3 is a flowchart of the weak learner generation process of this embodiment. This weak learner generation process is executed by the weak learner generation unit 108 of the server 101. First, learning data of each facility is acquired (S301). The learning data is acquired from the learning data storage unit 112. Next, a prediction model serving as a weak learner is generated from the learning data of each facility by machine learning (S302), and this process ends. At this time, a different machine learning algorithm may be used depending on the learning data of each facility.

[0019] FIG. 4 is a flowchart of the cutoff value calculation process of this embodiment. This cutoff value calculation process is executed by the cutoff value calculation unit 109 of the server 101. First, learning data and weak learners for each facility are acquired (S401). The learning data is acquired from the learning data storage unit 112. The weak learners are generated by a weak learner generation process. Next, a receiver operating characteristic (ROC) analysis is performed on the learning data for each weak learner (S402). For example, when the range of the predicted value is 0 to 1, the threshold for classifying whether the predicted target is negative or positive is moved between 0 and 1, and the true positive rate (sensitivity) and false positive rate (1-specificity) at each threshold are calculated.

[0020] Next, a cutoff value is calculated for each weak learner from the ROC analysis result (S403), and this process is terminated. Specifically, in order to maximize the classification performance of the weak learner for the learning data, the threshold at which Youden's index (sensitivity + specificity - 1) is maximized is set as the cutoff value. At this time, the cutoff value may be calculated using other methods. For example, an ROC curve with the true positive rate (sensitivity) on the vertical axis and the false positive rate (1 - specificity) on the horizontal axis may be drawn, and the threshold value of the coordinate at which the distance between the coordinate at which sensitivity = 1 and specificity = 1 and the ROC curve is the smallest may be set as the cutoff value. This makes it possible to identify whether the predicted value of the analysis subject is likely to be negative or positive in each weak learner.

[0021] Fig. 5 is an example of the cutoff value generated in step S403 in Fig. 4. Here, weak learner 501 is a weak learner generated from the learning data of each facility, and in this embodiment, two weak learners, weak learner 1 and weak learner 2, are generated. Cutoff value 502 is a cutoff value calculated for each weak learner.

[0022] FIG. 6 is a flowchart of the ensemble calculation process of this embodiment. This ensemble calculation process is executed by the ensemble calculation unit 110 of the server 101. First, the predicted value and cutoff value by the weak learner of each facility are obtained (S601). That is, the weak learner is generated by the weak learner generation process (FIG. 3) described above, and the predicted value (e.g., positive: 1, negative: 0) of the analysis subject is calculated using the prediction data recorded in the prediction data storage unit 113. Also, the cutoff value is generated by the cutoff value calculation process (FIG. 4) described above.

[0023] Next, the predicted value of each weak learner is compared with the cutoff value (S602). Depending on the magnitude relationship between the predicted value and the cutoff value, different statistical processing is performed to calculate the final predicted value (S603), and this process ends. Specifically, (1) If the prediction values ​​of each weak learner are all equal to or greater than the cutoff value, each weak learner tends to make positive predictions, so the maximum prediction value among the weak learners is calculated as the prediction value based on the ensemble calculation. (2) If the prediction values ​​of each weak learner are all less than the cutoff value, each weak learner tends to make negative predictions, so the smallest prediction value among the weak learners is calculated as the prediction value based on the ensemble calculation. (3) When the relationship between the predicted value and the cutoff value is different for each weak learner, that is, when the predicted value by one weak learner is greater than or equal to the cutoff value and the predicted value by the other weak learner is less than the cutoff value, one weak learner tends to predict positive and the other weak learner tends to predict negative. Therefore, the average of each predicted value of each weak learner is calculated and this is calculated as the prediction based on the ensemble calculation.

[0024] Fig. 7A is an example of the ensemble calculation generated in step S603 of Fig. 6 described above. A predicted value 701 of weak learner 1 and a predicted value 702 of weak learner 2 are predicted values ​​calculated by each weak learner for the subject of analysis. A cutoff value 703 of weak learner 1 and a cutoff value 704 of weak learner 2 are cutoff values ​​for each weak learner. An ensemble calculation result 705 is a predicted value calculated by an ensemble calculation based on the predicted values ​​and cutoff values ​​of each weak learner.

[0025] For example, if we focus on the record whose analysis subject ID 700 is P001, the predicted value 701 (=0.8) of weak learner 1 is greater than or equal to the cutoff value 703 (=0.4) of weak learner 1, and the predicted value 702 (=0.7) of weak learner 2 is greater than or equal to the cutoff value 704 (=0.6) of weak learner 2. Since each weak learner tends to predict positive, the ensemble calculation result 705 is 0.8, which is the maximum value of the predicted value 701 of weak learner 1 and the predicted value 702 of weak learner 2.

[0026] Furthermore, when focusing on the record whose analysis subject ID 700 is P002, the predicted value 701 (= 0.2) of weak learner 1 is less than the cutoff value 703 (= 0.4) of weak learner 1, and the predicted value 702 (= 0.1) of weak learner 2 is less than the cutoff value 704 (= 0.6) of weak learner 2, and since each weak learner tends to predict negative, the ensemble calculation result 705 is 0.1, which is the minimum value of the predicted value 701 of weak learner 1 and the predicted value 702 of weak learner 2.

[0027] Furthermore, if we focus on the record whose analysis subject ID 700 is P003, the predicted value 701 (=0.6) of weak learner 1 is greater than or equal to the cutoff value 703 (=0.4) of weak learner 1, while the predicted value 702 (=0.4) of weak learner 2 is less than the cutoff value 704 (=0.6) of weak learner 2. Since the relationship between the predicted value and the cutoff value differs for each learner, the ensemble calculation result 705 is 0.5, which is the average value of the predicted value 701 of weak learner 1 and the predicted value 702 of weak learner 2.

[0028] 7B illustrates the above-mentioned calculation example in a two-dimensional graph on the XY axis, where the X axis is the predicted value by weak learner 1 and the Y axis is the predicted value by weak learner 2, plotted for each analysis subject (P001, P002, P003). In addition, the cutoff value of each weak learner is used to divide the data into four regions (region I 710, region II 720, region III 730, region IV 740).

[0029] The subject P001 plotted in area I 710 shows a positive tendency in each learning device, so the maximum value of 0.8 is adopted. The subject P002 plotted in area II 720 shows a negative tendency in each learning device, so the minimum value of 0.1 is adopted. The subject P003 plotted in area III 730 shows different tendencies in each learning device, so the average value of both, 0.5, is adopted (the same applies when plotted in area IV 740).

[0030] This allows for a better separation between predicted values ​​for subjects likely to be positive and predicted values ​​for subjects likely to be negative, thereby improving the ability to distinguish between positive and negative results.

[0031] FIG. 8 is a flowchart of the output process of this embodiment. This output process is executed by the output unit 111 of the server 101. First, prediction data is acquired (S801). The prediction data is acquired from the input device 103 or the prediction data storage unit 113 (FIG. 2B). Next, the acquired prediction data is input to each weak learner, and the ensemble calculation result calculated in the ensemble calculation process (FIG. 6) is output as a final predicted value (S802), and this process ends.

[0032] 9 shows a display screen 900 that is output in the output process of this embodiment, and is displayed on the display of the output device 104. The display screen is made up of a graph display area 901 and a prediction analysis area 902.

[0033] The graph display area 901 is composed of a graph display information input area 903, a graph display execution button 904, and a display graph 905. The user can input information for setting the vertical and horizontal axes of the display graph 905 in the graph display information input area 903 and click the graph display execution button 904 to output the display graph 905.

[0034] The display graph 905 is an example of visualization of the ensemble calculation results of FIG. 7A and FIG. 7B. As described above, by setting the horizontal and vertical axes of the graph in the graph display information input area 903, it is possible to easily confirm the prediction results of each weak learner (weak learner 1, weak learner 2) for the prediction data of each analysis subject input from the prediction data storage unit 113, and the positive / negative tendency and distribution of the entire prediction data population. In addition, as in FIG. 7B, the prediction results (numerical values) of the weak learner 1 and the weak learner 2 may be displayed near the plot of the prediction result of each analysis subject, and further, the final prediction value by the above-mentioned ensemble calculation method may also be displayed. In addition, if there are many samples and all the prediction values ​​are overlapped on the screen and difficult to see, the prediction value may be displayed only when the user clicks on the plot with a mouse or the like.

[0035] On the other hand, the prediction analysis area 902 is where the user manually inputs prediction data of other subjects to be analyzed to obtain prediction results, and is composed of a prediction data input area 906, a prediction execution button 907, and prediction results 908. The user can input prediction data of the subject to be analyzed in the prediction data input area 906 and output prediction results 908 by clicking the prediction execution button 907. The prediction results 908 present the predicted values ​​of each weak learner, the ensemble calculation results, and the ensemble calculation method. This makes it possible to confirm the ensemble calculation method applied in the prediction.

[0036] As described above, in this embodiment, a cutoff value that maximizes the classification performance of each weak learner is calculated, and an ensemble calculation is performed based on the cutoff value according to the individual sample data and the predicted values ​​of the weak learners, thereby enabling robust predictions to be made even when there are differences in data characteristics between facilities.

[0037] In this embodiment, a case has been described in which a weak learner is generated by learning data from two facilities, but data from three or more facilities may be used. For example, if the number of facilities is N (≧3), N weak learners and N cutoff values ​​are generated, and the following ensemble calculation may be performed in the ensemble calculation process. That is, (1) If the prediction values ​​of the majority of weak learners are equal to or greater than the cutoff value, then the majority of weak learners tend to predict positive results, and therefore the maximum value of the prediction values ​​of each weak learner is calculated as the prediction value based on the ensemble calculation. (2) If the prediction values ​​of the majority of weak learners are less than the cutoff value, since the majority of weak learners tend to predict negative, the minimum of the prediction values ​​of each weak learner is calculated as the prediction value by ensemble calculation. (3) In other cases, since the number of weak learners that tend to predict positive is equal to the number of weak learners that tend to predict negative, the average of the prediction values ​​of each weak learner is calculated as the prediction value by ensemble calculation. This enables robust predictions even when there are a large number of facilities. [Explanation of symbols]

[0038] 101:Server 102: Database 103: Input device 104: Output device 105: Arithmetic device 106: Memory 107: Storage device 108: Weak learner generation unit 109: Cutoff value calculation unit 110: Ensemble Calculation Section 111: Output section 112: Learning data storage unit 113: Prediction data storage unit

Claims

1. An information processing system for calculating a predicted value of an analysis target, a weak learner generation unit that generates a plurality of weak learners corresponding to each of a plurality of targets from a plurality of learning data groups collected from different targets related to the analysis target; a cutoff value calculation unit that calculates cutoff values ​​of predicted values ​​for the weak learners corresponding to each of the plurality of learning data groups; an ensemble calculation unit that performs a comparative analysis between a predicted value calculated using the prediction data of the analysis target and the corresponding cutoff value by the generated weak learners, and calculates a predicted value of the analysis target according to the result of the comparison; An information processing system comprising:

2. 2. The information processing system according to claim 1, The cutoff value calculation unit performs an ROC analysis on the learning data group corresponding to each of the weak learners, and calculates a threshold value that maximizes the classification performance of the weak learner as a cutoff value of the weak learner. An information processing system comprising:

3. 3. The information processing system according to claim 1, the ensemble calculation unit calculates different statistical values ​​depending on the magnitude relationship between the predicted values ​​of the plurality of weak learners and the corresponding cutoff values; An information processing system comprising:

4. 4. The information processing system according to claim 3, The ensemble calculation unit is (1) when the predicted values ​​of each of the plurality of weak learners are all equal to or greater than the corresponding cutoff value, the maximum of the predicted values ​​of each of the weak learners is calculated as the predicted value of the analysis target; (2) when the predicted values ​​of each of the plurality of weak learners are all less than the corresponding cutoff value, the smallest one among the predicted values ​​of each of the weak learners is calculated as the predicted value of the analysis target; (3) When the magnitude relationship between the predicted values ​​of the weak learners and the corresponding cutoff values ​​differs among the weak learners, an average value of the predicted values ​​of the weak learners is calculated as the predicted value of the analysis target. An information processing system comprising:

5. 5. The information processing system according to claim 4, an output unit that displays a graph of the calculated predicted value of the analysis target; An information processing system comprising:

6. 6. The information processing system according to claim 5, the output unit inputs the prediction data to be analyzed and displays a prediction result; An information processing system comprising:

7. An information processing method for calculating a predicted value of an analysis target, comprising: generating a plurality of weak learners corresponding to each of the targets from a plurality of learning data groups collected from different targets related to the analysis target; Calculating cutoff values ​​of predicted values ​​for the weak learners corresponding to each of the plurality of learning data groups; a prediction value calculated using the prediction data of the analysis target and the corresponding cutoff value are compared and analyzed by the generated weak learners, and a prediction value of the analysis target is calculated according to the result of the comparison; 23. An information processing method comprising:

Citation Information

Patent Citations

  • Prediction device by ensemble learning of heterogeneous machine learning

    JP2021174330A