MODEL CONSTRUCTION DEVICE, MODEL CONSTRUCTION METHOD, AND PROGRAM

Abstract

To construct a prediction model of an earthquake damage which is difficult to estimate, by narrowing down significant variables.SOLUTION: A model construction device (1) comprises: a data acquisition unit (11) that inputs past damage data (21) and divides the input past damage data (21) into learning data and evaluation data; and a learning unit (12) that performs machine learning of learning data to construct model candidates, inputs evaluation data into the model candidates to calculate evaluation values of prediction accuracy of the model candidates, confirms change of the evaluation values by randomly replacing unspecified variables of the learning data to input them into the model candidates, and generates variable selection data in which variables having smallest change of the evaluation values are removed from the learning data. A model candidate having the largest evaluation value, of a plurality of model candidates that are constructed by repeating removing of the variables for the variable in the leaning data to become the last one is output as a final prediction model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a model construction device, a model construction method, and a program for constructing a model for predicting earthquake damage to underground facilities. [Background technology]

[0002] Conventionally, there are methods to construct damage prediction models from inspection data of underground facilities, and there are several cases where they have been applied to earthquake damage prediction. However, because some variables may cause unintended behavior, it is difficult to predict damage from another earthquake based on a damage prediction model obtained from another earthquake if all variables are simply used. The variables here include seismic intensity, peak ground velocity (PGV), peak ground acceleration (PGA), spectral intensity, displacement converted from PGV / PGA, equivalent dominant period, ground strain, average shear wave velocity up to 30m, average ground height, average slope, estimated artificial flat land, buried years, conduit length, etc.

[0003] Non-Patent Document 1 describes the results of constructing an earthquake damage prediction model that uses a machine learning algorithm to predict damage to underground telecommunications pipelines with high accuracy. In addition, the contribution of each of the above-mentioned variables was confirmed in this prediction model.

[0004] Non-Patent Document 2 describes the results of verifying the effectiveness of a prediction model constructed using machine learning in predicting damage to underground pipelines during earthquakes.

[0005] Furthermore, in the field of statistics, several algorithms for removing variables have been proposed. Non-Patent Document 3 describes a stepwise method for creating a significant model by incorporating and removing significant variables one by one. With the stepwise method, variables are automatically selected. For this reason, the stepwise method is an objective and convenient method. However, there is currently no damage prediction model that uses the stepwise method. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Yo Ito et al., "Earthquake Damage Prediction of Underground Steel Pipe with Screw Joint Using Machine Learning," ASCE lifelines Proceedings, p. 284, published March 22, 2021 [Non-Patent Document 2] Yo Ito et al., "Study of damage prediction model using machine learning based on the results of earthquake inspection of underground telecommunication pipelines," 40th Earthquake Engineering Research Conference C13-1657, published on September 24, 2020 [Non-Patent Document 3] EBPT Glossary Variable Increment Method, [online], [searched September 13, 2021], Internet <URL:http: / / jspt.japanpt.or.jp / ebpt_glossary / forward-backward-stepwise-selection-method.html> Summary of the Invention [Problem to be solved by the invention]

[0007] However, in order to construct a robust earthquake damage prediction model based on data on damage to underground facilities caused by past earthquakes (hereinafter referred to as "past damage data"), it is important to narrow down significant variables using a rational method, but there has been an issue in that this has not been done yet.

[0008] In view of the above circumstances, an object of the present invention is to construct a model for predicting earthquake damage that is difficult to estimate by narrowing down significant variables. [Means for solving the problem]

[0009] In order to solve the above problems, a model construction device according to one embodiment is a model construction device that constructs a prediction model of earthquake damage to an underground facility, and includes: a data acquisition unit that inputs past damage data and divides the input past damage data into learning data and evaluation data; and a learning unit that performs machine learning on the learning data to construct model candidates, inputs the evaluation data into the model candidates to calculate an evaluation value of the prediction accuracy of the model candidates, checks changes in the evaluation value by randomly replacing unspecified variables of the learning data and inputting the randomly replaced variables into the model candidates, and generates variable selection data in which the variable with the smallest change in evaluation value is deleted from the learning data. Each time the variable is deleted by the learning unit, the data acquisition unit newly divides the variable selection data into learning data and evaluation data, and the learning unit repeatedly deletes the variable in the learning data until only one variable remains, and outputs the model candidate with the largest evaluation value among the model candidates as a final prediction model.

[0010] In order to solve the above problems, a model construction method according to one embodiment is a model construction method for constructing a prediction model of earthquake damage to an underground facility, the method including: a data acquisition step of inputting past damage data by a model construction device and dividing the input past damage data into learning data and evaluation data; and a learning step of constructing a model candidate by machine learning the learning data, inputting the evaluation data into the model candidate, calculating an evaluation value of the prediction accuracy of the model candidate, confirming a change in the evaluation value by randomly replacing unspecified variables of the learning data and inputting the randomly replaced variables into the model candidate, and generating variable selection data in which the variable with the smallest change in evaluation value is deleted from the learning data, wherein the data acquisition step newly divides the variable selection data into learning data and evaluation data each time the variable is deleted, and the learning step repeatedly deletes the variable until there is only one variable left in the learning data, and among the model candidates, the model candidate with the largest evaluation value is output as the final prediction model.

[0011] In order to solve the above problem, a program according to one embodiment causes a computer to function as the model building device. Effect of the Invention

[0012] According to the present disclosure, by narrowing down significant variables, it becomes possible to automatically construct a robust earthquake damage prediction model and make highly accurate predictions. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing an example of the configuration of a model construction device according to an embodiment. [Diagram 2] 1 is a flowchart illustrating an example of a model construction method executed by a model construction device according to an embodiment. [Diagram 3] 1 is a table showing the structure of past damage data. [Figure 4A] 13 is a table showing an example of constructing a prediction model for a communication pipe. [Figure 4B] 13 is a graph showing an example of constructing a prediction model for a communication pipe. [Diagram 5] FIG. 2 is a block diagram showing a schematic configuration of a computer functioning as a model construction device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0015] Fig. 1 is a block diagram showing an example of the configuration of a model construction device according to an embodiment. The model construction device 1 shown in Fig. 1 includes a data acquisition unit 11, a learning unit 12, and a prediction unit 13. The model construction device 1 constructs a prediction model of earthquake damage to underground facilities. The data acquisition unit 11, the learning unit 12, and the prediction unit 13 form a control and arithmetic circuit (controller). The control and arithmetic circuit may be formed of dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array), may be formed of a processor, or may be formed including both.

[0016] The data acquiring unit 11 inputs past damage data 21. The data acquiring unit 11 divides the input past damage data 21 into learning data a and evaluation data b. In addition, as described below, the data acquiring unit 11 inputs data obtained by deleting from the learning data a a variable that causes the smallest change in the evaluation value of the model candidate F(x) created by the learning unit 12 as all data c from the learning unit 12.

[0017] The learning unit 12 receives the learning data a and the evaluation data b and generates a prediction model F * (x). The learning unit 12 constructs a prediction model F * The learning unit 12 outputs the prediction model F * (x) is stored in a storage unit, and a prediction model F * (x) may be output to the prediction unit 13.

[0018] The prediction unit 13 applies the prediction target data 22 to the prediction model F constructed by the learning unit 12. * (x) and output the prediction result (predicted value) 23 to predict earthquake damage. * The same variables used in constructing (x) must be present.

[0019] 2 is a flowchart showing an example of a model construction method executed by a model construction device according to an embodiment. As shown in FIG 2, the procedure for constructing a predictive model will be explained in detail by dividing it into steps S01 to S09. <Step S01>

[0020] FIG. 3 is a table showing the structure of past damage data. As shown in FIG. 3, the past damage data 21 is divided into records for each underground facility, and each record has a plurality of variables (hereinafter, for convenience of explanation, it is assumed that the number is m) indicating the presence or absence of damage, earthquakes, facilities, ground characteristics, etc. Any variables may be used, but it is desirable to introduce all variables that may be related to earthquake damage. Based on the plurality of records (hereinafter, for convenience of explanation, it is assumed that the number is n) having these m variables, a final prediction model F * Multiple candidates for (x) (hereinafter referred to as "candidate models F(x)") are constructed.

[0021] In step S01, the data acquisition unit 11 inputs past damage data 21 and divides the past damage data 21 into learning data a and evaluation data b. The data acquisition unit 11 divides the past damage data 21, which is divided into n records having m variables, into the learning data a and the evaluation data b. The learning data a is data used for machine learning, which will be described later, and the evaluation data b is data used for evaluating the accuracy of a candidate model F(x) constructed by machine learning. <Step S02>

[0022] In step S02, the learning unit 12 constructs a model candidate F(x) by machine learning. The learning unit 12 inputs learning data a from the data acquisition unit 11. The learning unit 12 performs machine learning on the learning data a to generate a final prediction model F *The learning unit 12 constructs a plurality of candidate models F(x) that are candidates for (x). The learning unit 12 constructs a model to fit the learning data a as a teacher. Machine learning refers to a technique for improving the intelligence of a computer by having the computer read data and giving commands, etc. The learning unit 12 is included in a computer that performs machine learning. Since the process in which a computer improves its intelligence requires complex calculations, a machine learning algorithm is built into the learning unit 12. For example, the learning unit 12 may construct the candidate model F(x) by machine learning using XGboost or Random forest. XGboost and Random forest are machine learning algorithms. The learning unit 12 uses the machine learning algorithm to iteratively "learn" from the learning data a, and constructs a final prediction model F * Construct a candidate model F(x) for (x).

[0023] XGBoost is a type of model building method that performs ensemble learning, called gradient boosting, by combining multiple decision trees. A decision tree is a model that expresses conditional branching in a tree structure, and is configured so that an answer can be obtained by tracing the conditions from the root node to the end. Ensemble learning is a learning method that combines multiple decision trees (learners) to build a comprehensively useful model. Boosting is a method of preparing multiple decision trees (learners) and proceeding with learning in series, and building the next model while referring to the results of the previous model. First, the first decision tree is created based on the training data. The first decision tree does not accurately explain all the training data, so a certain amount of error occurs. The second decision tree is created to cancel this error. In other words, the second decision tree is created so that the accuracy will increase if the results of the first and second decision trees are added together. In the same manner, new decision trees are created to cancel the errors generated by all the decision trees created up to that point. XGBoost is an algorithm that creates models by applying penalties to the size of trees, etc., and can build complex models by performing ensemble learning using serially connected decision trees.

[0024] Random forest is a model building method that uses multiple decision trees for ensemble learning, called bagging. Bagging is a method of training multiple decision trees (learners) in parallel, and outputs prediction results based on majority vote or average. <Step S03>

[0025] In step S03, the learning unit 12 calculates an evaluation value of the model candidate F(x). The learning unit 12 inputs evaluation data b to the constructed model candidate F(x) to calculate an evaluation value for evaluating the prediction accuracy of the model candidate F(x). The evaluation value is an evaluation index of the prediction result. At this time, the evaluation data b must contain the same variables as the learning data a used to construct the model candidate F(x). The learning unit 12 may calculate the evaluation value using ROC-AUC or F score (F value).

[0026] ROC (Receiver Operating Characteristic Curve) is a prediction curve, which is plotted with TPR (True Positive Rate) on the vertical axis and FPR (False Positive Rate) on the horizontal axis, and AUC (Area Under the Curve) is the area under the curve. AUC ranges from 0 to 1, and the closer the value is to 1, i.e., the larger the AUC area, the better the machine learning performance.

[0027] F score (F value) is the harmonic mean of precision and recall. Precision, which is an evaluation index of machine learning, is the proportion of data predicted to be positive that is actually positive. Recall, which is also an evaluation index, is the proportion of data predicted to be positive among data that is actually positive. However, precision and recall, which are evaluation indexes of machine learning, are in a trade-off relationship. For this reason, F score is calculated as the harmonic mean of precision and recall using the following formula (1), and is output as a number between 0 and 1. An F score (F value) of 0 gives the worst evaluation, and a value of 1 gives the best evaluation. F score = 2 * (precision * recall) / (precision + recall) (1) <Step S04>

[0028] In step S04, the learning unit 12 temporarily stores the model candidate F(x) and the evaluation value in association with each other. Each time a model candidate F(x) is constructed and its evaluation value is calculated, the learning unit 12 temporarily stores the model candidate F(x) and the evaluation value of the model candidate F(x) in association with each other. As will be described later, the learning unit 12 selects a final prediction model F(x) from among the multiple pieces of data in which the model candidate F(x) and the evaluation value are associated with each other and are temporarily stored. * Select (x). <Step S05>

[0029] Next, the learning unit 12 compares the importance of each variable with respect to the original learning data a. In step S05, the learning unit 12 randomly replaces unspecified variables in the learning data a and inputs them to the model candidate F(x), and confirms the change in the evaluation value of the model candidate F(x). The learning unit 12 may compare the importance of each variable according to the change in evaluation value using permutation importance or feature importance. Permutation importance and feature importance are interpretation methods for calculating the importance of features (variables).

[0030] Feature importance is a quantification of the importance of features using a value that evaluates how much a model has improved due to branching when creating a decision tree. Permutation Importance is an interpretation method that calculates the importance of features (variables) based on how much performance deteriorates when a trained model is used to shuffle certain features (variables) and make a prediction. Permutation Importance calculates the score reduction rate from a score calculated using all variables and a score calculated by randomly shuffling only certain variables, and determines that a variable with a larger score reduction rate is more important and a variable with a smaller score reduction rate is less important. <Step S06>

[0031] In step S06, the learning unit 12 generates variable selection data c by deleting the variable with the smallest change in evaluation value from the learning data a. If there are multiple variables with the same smallest rate, data in which all of the multiple variables have been deleted at the same time is set as the variable selection data c. <Step S07>

[0032] In step S07, the learning unit 12 repeatedly deletes variables until only one variable remains in the learning data a, and inputs the variable selection data c to the data acquiring unit 11. Each time a variable is deleted by the learning unit 12, the data acquiring unit 11 divides the variable selection data c into learning data a and evaluation data b (step S01), and the learning unit 12 constructs a model candidate F(x) (step S02), calculates an evaluation value for the model candidate F(x) (step S03), and temporarily stores the constructed model candidate F(x) in association with the evaluation value for the model candidate F(x) (step S04), repeating a series of operations. <Step S08>

[0033] In step S08, when the variables in the learning data run out, the learning unit 12 selects the model candidate F(x) with the maximum evaluation value from among the multiple model candidates temporarily stored as the final prediction model F(x).* (x) is recognized. <Step S09>

[0034] In step S09, the learning unit 12 generates the final prediction model F * (x) is output to the prediction unit 13. (Earthquake damage prediction)

[0035] Thereafter, as described above, the prediction unit 13 applies the prediction target data 22 to the prediction model F obtained from the learning unit 12. * (x) and output the prediction result (predicted value) 23 to predict earthquake damage. * The same variables used in constructing (x) must be present.

[0036] Figure 4A shows the prediction model F for communication pipes. * FIG. 4B is a table showing an example of constructing a prediction model F(x) for a communication pipe. *4B is a graph showing the results of FIG. 4A. In FIG. 4A, "no" is a number indicating the number of times a variable has been deleted, "no.0" indicates the evaluation value of the model candidate F(x) before the variable deletion and the variables used, and "no.1" indicates the evaluation value of the model candidate F(x) after the variable has been deleted once and the variables used. For example, "no.0" has a total of 13 variables, including the average inclination angle, average elevation, basic natural period, average S-wave velocity, number of years since the facility was constructed, facility (pipe) length, equivalent predominant period, measured seismic intensity, maximum velocity, maximum acceleration, SI (Spectral Intensity) value, converted displacement, and ground strain, while in "no.1", a total of four variables, including the average inclination angle, average elevation, basic natural period, and ground strain, have been deleted from "no.0". Figures 4A and 4B show that the variables with the smallest change in evaluation value were repeatedly deleted from learning data a until only one variable remained in the learning data a, and the evaluation value was calculated. The result shows that the Fscore, which is the evaluation index of "no. 8", which had variables deleted eight times, was the highest at 0.197. This shows that, of the 13 variables, the significant variables were the average S-wave velocity and the converted displacement. Therefore, the model candidate F(x), which had the highest evaluation value, "no. 8", was selected as the final prediction model F. * It is identified as (x).

[0037] As described above, the model construction device 1 according to this embodiment repeatedly deletes the variables with the smallest change in evaluation value until only one variable remains in the learning data, i.e., by narrowing down the significant variables, it becomes possible to automatically construct a robust earthquake damage prediction model and make highly accurate predictions.

[0038] A computer capable of executing program instructions can be used to function the above-mentioned model construction device 1. Fig. 5 is a block diagram showing a schematic configuration of a computer functioning as the model construction device. Here, the computer functioning as the model construction device 1 may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, etc. The program instructions may be program code, code segments, etc. for performing necessary tasks.

[0039] 5, the computer 100 includes a processor 110, a memory unit including a ROM (Read Only Memory) 120, a RAM (Random Access Memory) 130, and a storage 140, an input unit 150, an output unit 160, and a communication interface (I / F) 170. Each component is connected to each other via a bus 180 so as to be able to communicate with each other.

[0040] The ROM 120 stores various programs and various data. The RAM 130 temporarily stores programs or data as a working area. The storage 140 is configured with a hard disk drive (HDD) or a solid state drive (SSD), and stores various programs including an operating system and various data. In the present disclosure, the programs related to the present disclosure are stored in the ROM 120 or the storage 140.

[0041] Specifically, the processor 110 is a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a SoC (System on a Chip), etc., and may be composed of multiple processors of the same or different types. The processor 110 reads a program from the ROM 120 or the storage 140, and executes the program using the RAM 130 as a working area, thereby controlling each of the above components and performing various arithmetic processing. At least a part of the processing contents may be realized by hardware.

[0042] The program may be recorded on a recording medium readable by the model construction device 1. If such a recording medium is used, the program can be installed in the model construction device 1. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, a USB (Universal Serial Bus) memory, or the like. Furthermore, the program may be in a form that is downloaded from an external device via a network.

[0043] Although the above-mentioned embodiment has been described as a representative example, it is obvious to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present disclosure. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications or changes are possible without departing from the scope of the claims. [Explanation of symbols]

[0044] 1. Model building equipment 11 Data Acquisition Section 12 Learning Department 13 Prediction Department 21 Past damage data 22 Data to be predicted 23 Prediction results (predicted values)

Claims

1. A model construction device for constructing a prediction model of earthquake damage to underground facilities, a data acquisition unit that inputs past damage data and divides the input past damage data into learning data and evaluation data; constructing a candidate model by performing machine learning on the learning data; inputting the evaluation data into the model candidate to calculate an evaluation value of the prediction accuracy of the model candidate; Checking the change in the evaluation value by randomly replacing unspecified variables of the learning data and inputting the randomly replaced variables into the candidate model; a learning unit that generates variable selection data by deleting a variable with the smallest change in evaluation value from the learning data; the data acquisition unit divides the variable selection data into new learning data and new evaluation data each time the learning unit deletes the variable; The learning unit repeatedly deletes the variables in the learning data until only one variable remains, and outputs the model candidate with the highest evaluation value among the model candidates as a final prediction model.

2. The model building device according to claim 1 , wherein the learning unit builds the model candidates by machine learning using XGboost or Random forest.

3. The model building device according to claim 1 , wherein the learning unit calculates the evaluation value using a ROC-AUC or an F score.

4. The model construction device according to claim 1 , wherein the learning unit compares the importance of each variable according to a change in the evaluation value by using permutation importance or feature importance.

5. A method for constructing a prediction model of earthquake damage to underground facilities, comprising: With the model construction device, a data acquisition step of inputting past damage data and dividing the input past damage data into learning data and evaluation data; constructing a candidate model by performing machine learning on the learning data; inputting the evaluation data into the model candidate to calculate an evaluation value of the prediction accuracy of the model candidate; Checking the change in the evaluation value by randomly replacing unspecified variables of the learning data and inputting the randomly replaced variables into the candidate model; A learning step of generating variable selection data by deleting the variable with the smallest change in evaluation value from the learning data, The data acquisition step divides the variable selection data into new learning data and new evaluation data each time the variable is deleted; The learning step repeats the deletion of variables in the learning data until only one variable remains, and the model candidate with the highest evaluation value is output as the final predictive model.

6. A program for causing a computer to function as the model building device according to any one of claims 1 to 4.

Images