Prediction device, estimation method, prediction method, estimation program, and prediction program
The prediction device enhances accuracy in indirect prediction by using a second-order differential correction to mitigate noise-induced bias in intermediate variables, improving the estimation of objective variables.
Patent Information
- Application Number
- JP2023082527
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-18
- Publication Date
- 2024-11-28
AI Technical Summary
Conventional indirect prediction methods suffer from decreased prediction accuracy due to noise in intermediate variables, leading to bias in the target variable output from known calculation formulas.
A prediction device that includes an intermediate variable calculation unit using a learned prediction model and an estimator to correct the output value using a second-order differential of the function, thereby calculating a predicted value of the objective variable.
Improves prediction accuracy in indirect prediction by correcting for noise, particularly in complex output functions, reducing bias and enhancing the precision of target variable estimation.
Smart Images

Figure 2024165928000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a prediction device, an estimation method, an estimation program, and a prediction program. [Background technology]
[0002] Conventionally, a method is known in which the output of a machine learning model is input to a known calculation formula to obtain a final result. For example, Patent Document 1 describes inputting the output of a recurrent neural network to a softmax function.
[0003] There is also a known method called indirect prediction that uses a machine learning model. Indirect prediction is used to predict indicators in chemical plants, agriculture, the economy, etc. In indirect prediction, the machine learning model learns the relationship between explanatory variables and intermediate variables, rather than the relationship between explanatory variables and target variables.
[0004] During prediction, the machine learning model outputs intermediate variables based on the explanatory variables. Then, the objective variable is calculated from the intermediate variables using a known formula.
[0005] For example, consider the case of predicting yield using indirect prediction. In this case, the machine learning model that has learned the relationship between explanatory variables and intermediate variables outputs two intermediate variables, yield and theoretical yield, based on the input explanatory variables. Then, based on the two intermediate variables, yield and theoretical yield, the yield is calculated using the known formula, "yield = yield ÷ theoretical yield." [Prior art documents] [Patent documents]
[0006] [Patent Document 1] JP 2020-052611 A Summary of the Invention [Problem to be solved by the invention]
[0007] However, the conventional techniques have a problem in that the prediction accuracy in indirect prediction may decrease.
[0008] For example, intermediate variables output by a machine learning model may contain noise. In such a case, the intermediate variables containing noise are input to the known formula. As a result, a bias due to noise occurs in the objective variable output from the known formula, and the prediction accuracy decreases. [Means for solving the problem]
[0009] In order to solve the above-mentioned problems and achieve the objective, the prediction device is characterized by having an intermediate variable calculation unit that calculates intermediate variables from explanatory variables using a trained prediction model, and an estimation unit that uses the second derivative of a function that calculates an output value from the intermediate variables to estimate parameters for correcting the output value to obtain a predicted value of the dependent variable.
[0010] In order to solve the above-mentioned problems and achieve the objective, the prediction device is characterized by having an intermediate variable calculation unit that calculates intermediate variables from explanatory variables using a trained prediction model, and an objective variable calculation unit that calculates a predicted value of an objective variable by correcting an output value obtained by inputting the intermediate variables into a function using the second derivative of the function. Effect of the Invention
[0011] According to the present invention, it is possible to improve prediction accuracy in indirect prediction. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a prediction device. [Diagram 2] FIG. 2 is a diagram illustrating a conventional indirect prediction. [Diagram 3] FIG. 3 is a diagram illustrating the bias. [Figure 4] FIG. 4 is a diagram illustrating the flow of the learning process and the prediction process. [Diagram 5]FIG. 5 is a flowchart showing the flow of processing performed by the prediction device in the learning phase. [Figure 6] FIG. 6 is a flowchart showing the flow of processing by the prediction device in the inference phase. [Figure 7] FIG. 7 is a diagram illustrating an example of the configuration of a computer that executes a prediction program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, embodiments of a prediction device, an estimation method, a prediction program, and a prediction program according to the present application will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiments described below.
[0014] [First embodiment] [Configuration of the first embodiment] The configuration of a prediction device 10 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of a prediction device. The prediction device 10 predicts a response variable from an explanatory variable by indirect prediction.
[0015] As shown in FIG. 1, the prediction device 10 includes a communication unit 11, an input unit 12, an output unit 13, a storage unit 14, and a control unit 15.
[0016] The communication unit 11 performs data communication with other devices via a network. For example, the communication unit 11 is a network interface card (NIC).
[0017] The input unit 12 receives input of data and may be an interface connected to input devices such as a mouse and a keyboard.
[0018] The output unit 13 outputs data. The output unit 13 may be an interface connected to an output device such as a display, a speaker, or the like.
[0019] The storage unit 14 is a storage device such as a hard disk drive (HDD), a solid state drive (SSD), an optical disk, etc. The storage unit 14 may be a semiconductor memory in which data can be rewritten, such as a random access memory (RAM), a flash memory, or a non-volatile static random access memory (NVSRAM).
[0020] The storage unit 14 stores an OS (Operating System) and various programs executed by the prediction device 10. The storage unit 14 has a training data accumulation unit 141, model information 142, correction parameter information 143, and a prediction data accumulation unit 144.
[0021] The training data storage unit 141 stores training data for training a machine learning model. The training data is data that combines explanatory variables and intermediate variables.
[0022] Hereafter, the machine learning model will be referred to as the predictive model. A predictive model takes explanatory variables as input and outputs intermediate variables.
[0023] The model information 142 is information such as parameters for constructing a prediction model. For example, if the prediction model is a neural network, the model information 142 includes parameters such as weights and biases.
[0024] The correction parameter information 143 is information on parameters for correcting the calculation result of a known calculation formula when calculating a target variable from an intermediate variable.
[0025] Hereinafter, a function that performs a calculation according to a known formula is called an output function. The prediction device 10 obtains a response variable by correcting an output value of the output function using a correction parameter.
[0026] The prediction data accumulation unit 144 accumulates new data. The new data is explanatory variables whose corresponding intermediate variables are unknown.
[0027] The control unit 15 controls the entire prediction device 10. The control unit 15 is, for example, an electronic circuit such as a central processing unit (CPU), a micro processing unit (MPU), or a graphics processing unit (GPU), or an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).
[0028] The control unit 15 also has an internal memory for storing programs and control data that define various processing procedures, and executes each process using the internal memory. The control unit 15 also functions as various processing units by the operation of various programs. For example, the control unit 15 has an acquisition unit 151, an intermediate variable calculation unit 152, a target variable calculation unit 153, an output control unit 154, and an update unit 155.
[0029] The acquisition unit 151 acquires training data and prediction data. For example, the acquisition unit 151 acquires sensor values of sensors installed in a chemical plant as explanatory variables or objective variables. The acquisition unit 151 stores the acquired data in the training data storage unit 141 or the prediction data storage unit 144.
[0030] The intermediate variable calculation unit 152 inputs the explanatory variables to the prediction model and calculates the intermediate variables.
[0031] A response variable calculation unit 153 calculates a response variable by inputting an intermediate variable to an output function and correcting the obtained output value using a correction parameter.
[0032] The output control unit 154 outputs the prediction result obtained in the inference phase. For example, the output control unit 154 causes the display device to display the objective variables (predicted values) calculated by the objective variable calculation unit 153 and the sensitivity of the prediction model in the form of a table, a graph, or the like.
[0033] The update unit 155 updates the parameters of the prediction model and the correction parameters in the learning phase.
[0034] In this embodiment, it is possible to prevent bias from occurring in the objective variables of the prediction results by correcting the output values of the output functions by the objective variable calculation unit 153. The correction process by the objective variable calculation unit 153 will be described later.
[0035] Conventional indirect prediction will now be described with reference to Fig. 2. Fig. 2 is a diagram for explaining conventional indirect prediction.
[0036] The explanatory variable x includes multiple values such as sensor value_1, sensor value_2, ..., sensor value_p. The trained prediction model ^g (^ directly above g) outputs intermediate variable z based on the explanatory variable x. The intermediate variable z includes multiple values such as sensor value_A, ..., sensor value_L.
[0037] A known output function f calculates the response variable y, which is the target of prediction, from the intermediate variable z. Such indirect prediction is expressed as in equation (1).
[0038]
number
[0039] Here, the objective variable y is expressed as in equation (2), and the intermediate variable z is expressed as in equation (3).
[0040]
number
[0041]
number
[0042] As shown in equation (3), the intermediate variable z may contain noise ε. Let ε be a component with independent mean 0 and variance σ k2 If the noise is , the expected value of the objective variable is approximated as equation (4).
[0043]
number
[0044] H in Equation (4) f is the second derivative of the output function. The second term on the right-hand side of equation (4) is the second derivative (correction term) when the objective variable shown in equation (2) is approximated by Taylor expansion taking into account the distribution of ε.
[0045] In the indirect prediction explained in Figure 2 and equation (1), the first term on the right-hand side of equation (4) is taken into account, but the second term is not. This causes a bias in the predicted value of the objective variable.
[0046] For example, consider a case where the objective variable is expressed by equation (5) and the intermediate variable is expressed by equation (6). An example of a prediction result by conventional indirect prediction in this case is shown in Figure 3. Figure 3 is a diagram for explaining bias.
[0047]
number
[0048]
number
[0049] The horizontal axis in Figure 3 corresponds to the explanatory variable x, and the vertical axis corresponds to the objective variable y. Plug-in is a predicted value (objective variable) based on conventional indirect prediction. Moreover, conditional expectation is a predicted value based on the true values of the explanatory variables and objective variable. Due to the influence of noise, a difference occurs between plug-in and conditional expectation. Taking noise into account, it is desirable to obtain a predicted value close to the conditional expectation.
[0050] The prediction device 10 of this embodiment calculates a response variable that takes into account the influence of noise by correcting the output of the output function.
[0051] The learning process and prediction process performed by the prediction device 10 will be described with reference to Fig. 4. Fig. 4 is a diagram illustrating the flow of the learning process and prediction process.
[0052] [Learning phase: learning the predictive model] The upper part of FIG. 4 shows the flow of the learning process using training data. First, the prediction device 10 calculates the intermediate variable z j For (j=1,2,…,p), the prediction model ^g j Learn about the following.
[0053] Specifically, the intermediate variable calculation unit 152 calculates a prediction model ^g based on the explanatory variable x stored in the training data storage unit 141. j Using the intermediate variable z j Then, the update unit 155 compares the intermediate variables stored in the training data storage unit 141 with the intermediate variables calculated by the intermediate variable calculation unit 152, and calculates the prediction model ^g j Update the parameters of
[0054] The update unit 155 can update the parameters by a known machine learning method (for example, backpropagation). The update unit 155 stores the updated parameters as the model information 142 in the storage unit 14.
[0055] [Learning phase: Estimation of correction parameters] Next, the prediction device 10 estimates correction parameters for correcting the output of the output function. The intermediate variable calculation unit 152 calculates the prediction model ^g j Using the intermediate variable ^z as shown in (7), j Calculate.
[0056]
number
[0057] Then, the update unit 155 calculates the residual r of the plug-in estimator as shown in equation (8). y is the objective variable of the training data. The objective variable of the training data may be stored in the training data storage unit 141, or may be calculated from the intermediate variables of the training data. In addition, ^z=(^z1, ^z2, ..., ^z p ) is a p-dimensional vector.
[0058]
number
[0059] In addition, the update unit 155 updates the second-order differential h j Calculate.
[0060]
number
[0061] Furthermore, the update unit 155 uses linear regression to calculate the correction parameter β j Estimate.
[0062]
number
[0063] This allows us to calculate the correction parameter β for each of the p intermediate variables. j The update unit 155 obtains the correction parameter β j is stored in the storage unit 14 as the correction parameter information 143. j is σ in Eq. (4). k 2 Corresponds to.
[0064] In this way, the update unit 155 estimates the correction parameters for correcting the output value and obtaining the predicted value of the objective variable by using the second derivative of the output function that calculates the output value from the intermediate variable. Note that the update unit 155 functions as an estimation unit.
[0065] In addition, when there are multiple intermediate variables, the update unit 155 estimates the parameters by linear regression so that the sum of the products of the second derivatives of the output function for each of the multiple intermediate variables and the parameters is equal to the residual of the output value calculated by the output function.
[0066] [Inference phase: Calculating predicted values] The lower part of Fig. 4 shows the flow of prediction processing using new data. The intermediate variable calculation unit 152 calculates the predicted value ^y of the objective variable by equation (11). In equation (11), the trained prediction model ^g and the correction parameter β j is used.
[0067]
number
[0068] In this way, the objective variable calculation unit 153 calculates the predicted value of the objective variable by correcting the output value obtained by inputting the intermediate variable to the output function using the second-order differential of the output function.
[0069] [Processing flow of the first embodiment] The flow of processing by the prediction device 10 in the learning phase will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of processing by the prediction device in the learning phase.
[0070] 5, first, the prediction device 10 acquires training data (step S101). Next, the prediction device 10 learns a prediction model of intermediate variables (step S102).
[0071] Next, the prediction device 10 calculates the residual (equation (8)) and the second-order differential (equation (9)) of the Plug-in estimated amount (step S103).
[0072] Then, the prediction device 10 estimates correction parameters by linear regression (equation (10)) (step S104).
[0073] The flow of processing by the prediction device 10 in the inference phase will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of processing by the prediction device in the inference phase.
[0074] 6, the prediction device 10 first acquires prediction data (new data) (step S201). Next, the prediction device 10 calculates a corrected prediction value using the prediction data and the correction parameters (Equation (11)) (step S202).
[0075] Furthermore, the prediction device 10 calculates the sensitivity (step S203). The sensitivity is the first derivative of the output function. Then, the prediction device 10 outputs the calculation result (step S204).
[0076] [Advantages of the first embodiment] As described above, the intermediate variable calculation unit 152 calculates intermediate variables from explanatory variables using a trained prediction model. The update unit 155 estimates parameters for correcting the output value and obtaining a predicted value of the objective variable using the second derivative of the output function that calculates the output value from the intermediate variables.
[0077] Furthermore, the objective variable calculation unit 153 calculates a predicted value of the objective variable by correcting an output value obtained by inputting the intermediate variable to the output function using the second-order differential of the output function.
[0078] In this way, the prediction device 10 can correct the output function using the estimated parameters, and can prevent bias from occurring in the response variable of the prediction result in indirect prediction, thereby improving the prediction accuracy in indirect prediction.
[0079] In particular, the more complex the output function is, the greater the effect of noise, so the more likely it is that the effect of this embodiment will be achieved. For example, this embodiment is effective when the output function is a twice-differentiable function (a function whose second-order differentiation does not become 0). The twice-differentiable function includes an n-th order function (n≧2), an exponential function, a logarithmic function, etc.
[0080] [System configuration, etc.] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or a part of it can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic. Note that the program may be executed not only by the CPU but also by other processors such as a GPU.
[0081] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.
[0082] [program] As an embodiment, the prediction device 10 can be implemented by installing a prediction program that executes the above information processing as package software or online software on a desired computer. For example, the above prediction program can be executed by an information processing device, so that the information processing device can function as the prediction device 10. The information processing device here includes desktop or notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as tablet terminals, smartphones, mobile phones and PHS (Personal Handy-phone System), and slate terminals such as PDA (Personal Digital Assistant).
[0083] The prediction device 10 may also be implemented as a server that provides the above-mentioned information processing-related services to a client, the client being a terminal device used by a user. For example, the server is implemented as a server device that provides an information processing service that uses explanatory variables as input and outputs a predicted value of a target variable taking noise into consideration. In this case, the server may be implemented as a Web server or as a cloud that provides the above-mentioned information processing-related services by outsourcing.
[0084] 7 is a diagram showing an example of the configuration of a computer that executes a prediction program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.
[0085] The memory 1010 includes a read only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.
[0086] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the prediction device 10 is implemented as a program module 1093 in which a computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing a process similar to the functional configuration of the prediction device 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).
[0087] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.
[0088] Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)). The program module 1093 and the program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070. [Explanation of symbols]
[0089] 10 Prediction Device 11 Communications Department 12 Input section 13 Output section 14 Storage section 15 Control section 141 Training Data Storage Unit 142 Model Information 143 Correction Parameter Information 144 Prediction Data Storage Unit 151 Acquisition Department 152 Intermediate variable calculation section 153 Objective variable calculation unit 154 Output control section 155 Update Department
Claims
1. an intermediate variable calculation unit that calculates intermediate variables from explanatory variables using a trained prediction model; an estimation unit that estimates a parameter for correcting the output value and obtaining a predicted value of a response variable by using a second-order differential of a function that calculates an output value from the intermediate variable; A prediction device comprising:
2. The estimation unit estimates the parameters by linear regression such that a sum of products of the second derivatives of the function for each of the intermediate variables and the parameters is equal to a residual of the output value calculated by the function. The prediction device according to claim 1 .
3. an intermediate variable calculation unit that calculates intermediate variables from explanatory variables using a trained prediction model; an objective variable calculation unit that calculates a predicted value of an objective variable by correcting an output value obtained by inputting the intermediate variable into a function using a second-order differential of the function; A prediction device comprising:
4. 1. A computer-implemented estimation method comprising: an intermediate variable calculation step of calculating intermediate variables from explanatory variables using a trained prediction model; an estimation step of estimating a parameter for correcting the output value to obtain a predicted value of the objective variable, using a second-order differential of a function that calculates an output value from the intermediate variable; The estimation method according to claim 1,
5. 1. A computer-implemented prediction method, comprising: an intermediate variable calculation step of calculating intermediate variables from explanatory variables using a trained prediction model; a response variable calculation step of calculating a predicted value of a response variable by correcting an output value obtained by inputting the intermediate variable into a function using a second-order differential of the function; A prediction method comprising:
6. an intermediate variable calculation step of calculating intermediate variables from explanatory variables using a trained predictive model; an estimation step of estimating a parameter for correcting the output value to obtain a predicted value of the objective variable, using a second derivative of a function that calculates an output value from the intermediate variable; An estimation program characterized by causing a computer to execute the above.
7. an intermediate variable calculation step of calculating intermediate variables from explanatory variables using a trained predictive model; an objective variable calculation step of calculating a predicted value of an objective variable by correcting an output value obtained by inputting the intermediate variable into a function using a second-order differential of the function; A prediction program that causes a computer to execute the above steps.
Citation Information
Patent Citations
Tag estimation device, tag estimation method and program
JP2020052611A
