Information processing device and information processing program

Abstract

An information processing device and an information processing program are provided that can further improve the estimation accuracy of a dependent variable. [Solution] The information processing device 10 includes an acquisition unit 24 that acquires a variable value set 54 corresponding to multiple explanatory variables used for estimation, and an estimation unit 28 that executes a first estimation process that estimates a single dependent variable from the variable value set 54 using a simple regression model for each explanatory variable and obtains an intermediate value set 56 which is a collection of provisional estimated values ​​of the dependent variable, and a second estimation process that estimates the dependent variable using the intermediate value set 56 and obtains an estimated value 58 of the dependent variable.

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing program for estimating a single objective variable from a plurality of explanatory variables. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there has been known an artificial intelligence technique for performing a learning process on a learning device including a neural network and optimizing each value of a group of learning parameters.

[0003] Patent Document 1 discloses a controller that optimizes control parameters in an inverted pendulum control system using a neural network learning algorithm (for example, an error backpropagation method using mean square error). Patent Document 1 also discloses a hierarchical neural network as an estimation model for estimating a single objective variable from multiple explanatory variables. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 06-095704 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, the estimation accuracy of the objective variable may differ depending on the characteristics of the training dataset, more specifically, the magnitude or tendency of the interaction between the explanatory variables. However, the estimation model disclosed in Patent Document 1 does not take into account the influence of the interaction at all, and the estimation accuracy may decrease depending on the characteristics of the training dataset.

[0006] The present invention has been made in consideration of such problems, and has a general purpose to provide an information processing apparatus and an information processing program capable of further improving the estimation accuracy of a dependent variable. [Means for solving the problem]

[0007] The information processing device of the present invention includes an acquisition unit that acquires variable value sets corresponding to a plurality of explanatory variables used for estimation, and an estimation unit that executes a first estimation process that estimates a single dependent variable from the variable value sets acquired by the acquisition unit using a simple regression model for each of the explanatory variables and obtains an intermediate value set that is a collection of provisional estimates of the dependent variable, and a second estimation process that estimates the dependent variable using the intermediate value set and obtains an estimate of the dependent variable.

[0008] Furthermore, the regression equation indicated by the simple regression model may be a broken line function formed by connecting a plurality of line segments.

[0009] In addition, the information processing device may further include a UI unit for displaying a scatter plot showing a distribution of the objective variable with respect to the explanatory variables, and for accepting an input operation via the scatter plot to divide the scatter plot into a plurality of intervals corresponding to the plurality of line segments.

[0010] The second estimation process may include a first calculation of calculating a plurality of interaction values ​​indicating interactions between the intermediate value sets, and a second calculation of applying a contribution rate to the plurality of interaction values ​​calculated by the first calculation to obtain the estimated value of the dependent variable.

[0011] Furthermore, the estimation unit may estimate the dependent variable using an estimation model in which a first estimator for performing the first estimation process and a second estimator for performing the second estimation process are connected in series, and the first estimator is a hierarchical neural network including a first input layer, a first hidden layer, and a first output layer, and an activation function of an arithmetic unit constituting the first hidden layer may have a linear or polygonal functional shape.

[0012] In addition, the information processing device may further include a regression calculation unit that uses a learning dataset that is a collection of the explanatory variables and the target variable to determine a plurality of intervals for each of the explanatory variables, performs a simple regression calculation for each of the intervals, and calculates a regression coefficient set that is a simple regression coefficient for each of the explanatory variables and for each of the intervals, and a learning processing unit that sets the regression coefficient set obtained by the regression calculation unit as an initial value of a variable parameter that specifies a calculation rule in the first estimator, and performs a learning process to update the variable parameter for the estimation model.

[0013] The information processing program of the present invention causes one or more computers to execute an estimation step of executing: an acquisition step of acquiring variable value sets corresponding to a plurality of explanatory variables used for estimation; a first estimation process of estimating a single objective variable from the acquired variable value sets using a simple regression model for each of the explanatory variables to obtain an intermediate value set that is a collection of provisional estimates of the objective variable; and a second estimation process of estimating the objective variable using the intermediate value set to obtain an estimate of the objective variable. Effect of the Invention

[0014] According to the present invention, it is possible to further improve the estimation accuracy of the dependent variable. [Brief description of the drawings]

[0015] [Figure 1] 1 is a block diagram showing an example of a configuration of an information processing device according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a diagram illustrating an example of a network structure of the estimation model of FIG. [Diagram 3] 4 is a first flowchart showing an example of a learning operation by the information processing device of FIG. [Figure 4] 10 is a second flowchart showing an example of a learning operation by the information processing device of FIG. [Diagram 5] FIG. 13 is a diagram illustrating an example of a scatter plot relating to explanatory variables and a response variable. [Figure 6]FIG. 5 is a diagram showing an example of a regression equation obtained by a simple regression calculation for the scatter diagram of FIG. 4. [Figure 7] 2 is a diagram showing an example of a learning setting screen displayed on a UI unit in FIG. 1. [Figure 8] FIG. 7 is a diagram illustrating an example of variable parameters corresponding to the broken line function in FIG. 6. [Figure 9] 4 is a flowchart showing an example of an estimation operation performed by the information processing device of FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In order to facilitate understanding of the description, the same components in each drawing are denoted by the same reference numerals as much as possible, and duplicated descriptions are omitted. In addition, the term "part" may be replaced with other terms such as unit, module, device, or element.

[0017] [Configuration of information processing device 10] <Overall composition> FIG. 1 is a block diagram showing an example of the configuration of an information processing device 10 according to an embodiment of the present invention. The information processing device 10 may be a "general-purpose computer" designed for various applications, or a "dedicated computer" designed for a specific application. Examples of general-purpose computers include stationary devices such as personal computers, and portable devices such as tablets and laptops. Examples of dedicated computers include embedded computers and PLCs (Programmable Logic Controllers).

[0018] The information processing device 10 may be a server computer that provides an information processing service. The server computer may be either a cloud type or an on-premise type. Although the information processing device 10 is shown as a single computer in FIG. 1, the information processing device 10 may be a group of computers that configure a distributed system instead.

[0019] Specifically, the information processing device 10 includes a communication unit 12, an input unit 14, a display unit 16, a control unit 18, and a storage unit 20.

[0020] The communication unit 12 is an interface that transmits and receives electrical signals to and from an external device (not shown). This allows the information processing device 10 to acquire a learning dataset 40 from the external device and provide the estimation result information 46 generated by the information processing device 10 to the external device.

[0021] The input unit 14 is composed of an input device such as a mouse, a keyboard, a touch sensor, a microphone, or a combination of these. The display unit 16 is composed of an output device including a liquid crystal panel, an organic EL (Electro Luminescence) panel, or electronic paper. A graphical user interface (hereinafter, UI unit 22) is constructed by combining the input function of the input unit 14 and the display function of the display unit 16.

[0022] The control unit 18 is composed of [1] a general-purpose processor including a CPU (Central Processing Unit) and a GPGPU (General-Purpose computing on Graphics Processing Unit), and [2] a dedicated processor including an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array). The control unit 18 reads out and executes programs and data stored in the storage unit 20, thereby functioning as an acquisition unit 24, a regression calculation unit 26, an estimation unit 28, a learning processing unit 30, a display control unit 32, and an output processing unit 34.

[0023] The acquisition unit 24 acquires various data necessary for the learning process or the estimation process. For example, the acquisition unit 24 acquires a learning dataset 40 and a variable value set 54. The acquisition unit 24 may acquire data from an external device via communication, or may acquire data via each unit of the information processing device 10 (for example, the storage unit 20 or the UI unit 22).

[0024] The regression calculation unit 26 performs a regression calculation (so-called simple regression calculation) that predicts a single objective variable from a single explanatory variable, using the learning dataset 40 acquired by the acquisition unit 24. In this simple regression calculation, a broken line function formed by connecting multiple line segments is used as a regression formula. That is, the regression calculation unit 26 determines multiple intervals for each explanatory variable using the learning dataset 40, performs a simple regression calculation for each interval, and calculates a simple regression coefficient (a regression coefficient set 42 described later) for each explanatory variable and for each interval. Note that since the multiple variables have different definitions, the regression calculation unit 26 may align the scales of the variables through standardization or normalization.

[0025] The estimation unit 28 performs information processing (hereinafter, estimation processing) for estimating a single objective variable from a plurality of explanatory variables. For this estimation processing, an estimation model 36 that has been subjected to machine learning is used. The estimation model 36 is a mathematical model in which a first estimator E1 that performs a first estimation processing and a second estimator E2 that performs a second estimation processing are connected in series. The first estimation processing is information processing for estimating objective variables from a variable value set 54 corresponding to a plurality of explanatory variables used in the estimation using a simple regression model for each explanatory variable, and obtaining a set of tentative estimates of the objective variables (hereinafter, intermediate value set 56). The second estimation processing is information processing for estimating objective variables using the intermediate value set 56, and obtaining an estimate value 58 of the objective variable. The second estimation processing includes, for example, a "first calculation" for calculating a plurality of interaction values ​​indicating interactions between the intermediate value sets 56, and a "second calculation" for obtaining an estimate value of the objective variable by applying a contribution rate to the plurality of interaction values. The estimation unit 28 may adjust the scale of each variable through standardization or normalization, as in the case of the regression calculation unit 26. Furthermore, this simple regression model may be a mathematical model that completely reproduces the simple regression calculation by the regression calculation unit 26, or may be a mathematical model (so-called derived model) that substantially reproduces the simple regression calculation.

[0026] The first estimator E1 is a computing unit for reproducing a simple regression model for each explanatory variable. The regression equation indicated by the simple regression model is a broken line function formed by connecting a plurality of line segments. That is, the first estimator E1 corresponds to a computing unit that arranges the simple regression calculations for each explanatory variable by the regression calculation unit 26 in parallel and executes the simple regression calculations collectively. In this case, the first estimator E1 is, for example, a hierarchical neural network including a first input layer, a first intermediate layer, and a first output layer. When the number of explanatory variables is n (n≧2) and the number of intervals is m (n≧2), the number of calculation units in the first input layer is, for example, (n+1). The number of calculation units in the first intermediate layer is, for example, (n·m+1). The number of calculation units in the first output layer is, for example, n. Note that, when the threshold of the activation function is included in the variable parameters, the number of calculation units in the first input layer may be n, and the number of calculation units in the first intermediate layer may be n·m.

[0027] The activation function of the arithmetic units constituting the first hidden layer is a function shape that can express the broken line function, which is the regression formula in the simple regression calculation, by a linear sum. Specifically, the activation function has a linear or broken line function shape. Examples of this function shape include a ReLU (Rectified Linear Unit) function, a ReLU6 function, a Leaky ReLU function, a hard hyperbolic tangent function, a hard sigmoid function, an identity function, a soft shrink function, and a hard shrink function.

[0028] The second estimator E2 is a computing unit that takes into account at least interactions between tentative estimated values. The second estimator E2 is, for example, a hierarchical neural network including a second input layer, at least one second intermediate layer, and a second output layer. When the number of explanatory variables is n (n≧2), the number of computing units in the second input layer is, for example, n. The number of computing units in the second intermediate layer may be n or m (m≠n). The number of computing units in the second output layer is, in principle, one.

[0029] The second estimator E2 may execute the above-mentioned first and second calculations integrally, or may execute the first and second calculations separately and sequentially. When the first and second calculations are separated, the second estimator E2 is a mathematical model in which a first calculation unit E21 that performs the first calculation and a second calculation unit E22 that performs the second calculation are connected in series.

[0030] The first calculator E21 is provided to calculate an interaction value indicating an interaction between tentative estimated values. The interaction value may be a product of two values, or a product of three or more values. The type of interaction is not limited to a product, and may be another nonlinear operator. The first calculator E21 is, for example, a neural network including an input layer and an output layer. When the number of explanatory variables is n (n≧2), the number of calculation units in the input layer is, for example, n. The number of calculation units in the output layer is equal to the number (m) of interaction values. For example, when the product of two values ​​is calculated in all combinations, m=n(n−1) / 2.

[0031] The second calculator E22 is provided to apply a contribution rate to a plurality of interaction values ​​to obtain an estimate of the objective variable. Examples of the contribution rate include a weighted sum and a softmax calculation. The second calculator E22 is, for example, a neural network including an input layer and an output layer. The number of calculation units in the input layer is equal to the number (m) of interaction values. The number of calculation units in the output layer is, in principle, one.

[0032] The learning processing unit 30 performs learning processing (here, supervised learning) on ​​the estimation model 36 using the learning dataset 40. The learning processing unit 30 sets initial values ​​of variable parameters that specify the calculation rules in the estimation model 36, and then sequentially updates the values ​​of the variable parameters according to a predetermined update rule. Examples of the update rules for the learning parameters include the stochastic gradient descent method, the momentum method, the AdaGrad method, and the Adam method.

[0033] The initial values ​​of the variable parameters are set using the regression coefficient set 42 obtained by the regression calculation unit 26. For example, in the first estimator E1, the weighting coefficients are set so as to be equivalent to the simple regression calculation by the regression calculation unit 26. For example, in the second estimator E2, the weighting coefficients are set so that the magnitude of the interactions between the explanatory variables are approximately the same. A specific example of the setting method will be described in detail with reference to FIG. 8.

[0034] The display control unit 32 controls the drive operation of the display unit 16 so as to display a desired image or video. Specifically, the display control unit 32 generates display data for displaying various screens including a learning setting screen 70 (FIG. 7), and supplies the display data to the display unit 16.

[0035] The output processing unit 34 performs output processing based on the estimation result obtained by the estimation unit 28. The output target may be the estimated value 58 itself, or a product generated from the estimated value 58. Examples of the product include [1] analysis information based on the estimated value 58, or [2] a control signal for controlling a controlled object.

[0036] The storage unit 20 stores programs and data necessary for the control unit 18 to control each component. The storage unit 20 is configured as a non-transient and computer-readable recording medium. Here, the computer-readable recording medium is a storage device such as a portable medium such as a magneto-optical disk, a ROM (Read Only Memory), a CD (Compact Disc)-ROM, a flash memory, a hard disk drive (HDD) built into a computer system, or a solid state drive (SSD). In the example of FIG. 1, the storage unit 20 stores a learning dataset 40, a regression coefficient set 42, a learning parameter group 44, and estimation result information 46.

[0037] The learning data set 40 is data necessary for the learning process, and specifically, is a collection of learning data with a variable value set 50 and a correct answer value 52 as data units. The variable value set 50 is composed of a plurality of explanatory variables. The correct answer value 52 corresponds to an ideal value (or a teaching value) of the objective variable corresponding to the variable value set 50.

[0038] The regression coefficient set 42 is a collection of simple regression coefficients calculated by the regression calculation unit 26. The simple regression coefficient is calculated for each explanatory variable and for each interval. That is, the regression coefficient set 42 corresponds to [1] the slope of the linear regression equation, [2] the intercept of the linear regression equation, [3] the lower limit of the interval, and [4] the upper limit of the interval, for each explanatory variable.

[0039] The learning parameter group 44 is a collection of learning parameters determined through a learning process for the estimation model 36. Examples of the learning parameters include [1] "hyperparameters" for specifying the model structure of the estimation model 36, or [2] "variable parameters" whose optimal values ​​change depending on the population of the learning dataset 40. Examples of the variable parameters include weight coefficients between calculation units, activation function coefficients, etc.

[0040] The estimation result information 46 includes information on the estimation result by the estimation unit 28, specifically, a variable value set 54, an intermediate value set 56, and an estimated value 58. The variable value set 54 is a plurality of variable values ​​corresponding to explanatory variables used in the estimation. The intermediate value set 56 is an output value from the first estimator E1 (i.e., a collection of tentative estimated values ​​of the dependent variable). The estimated value 58 is an output value from the second estimator E2 (i.e., an estimated value of the dependent variable).

[0041] <Network structure of estimated model 36> Fig. 2 is a diagram showing an example of a network structure of the estimation model 36 in Fig. 1. In the example of Fig. 2, it is assumed that the number of explanatory variables is four, and the number of intervals of the explanatory variables is three. For convenience of explanation, only the synaptic connection related to the second explanatory variable is shown. The other explanatory variables have the same connection configuration as that of the second explanatory variable.

[0042] The estimation model 36 is configured, for example, as a hierarchical neural network consisting of five layers LY1, LY2, LY3, LY4, and LY5. The calculation rules of the estimation model 36 are determined by the values ​​of a learning parameter group 44 (FIG. 1), which is a collection of learning parameters.

[0043] The layer LY1 corresponds to the first input layer of the first estimator E1, and is composed of five arithmetic units in the example shown in the figure. The first to fourth explanatory variables are input to the first to fourth arithmetic units from the top, respectively. A fixed value (e.g., 1) is input to the fifth arithmetic unit from the top.

[0044] The layer LY2 corresponds to the first hidden layer of the first estimator E1, and in the example shown in the figure, is composed of 13 arithmetic units whose activation function is the ReLU function. The first to third arithmetic units from the top output the calculation results within the three intervals corresponding to the first explanatory variable. The fourth to sixth arithmetic units from the top output the calculation results within the three intervals corresponding to the second explanatory variable. The seventh to ninth arithmetic units from the top output the calculation results within the three intervals corresponding to the third explanatory variable. A fixed value (e.g., 1) is input to the thirteenth arithmetic unit from the top.

[0045] The layer LY3 corresponds to the first output layer of the first estimator E1 or the second input layer of the second estimator E2 (or the input layer of the first computing unit E21), and is composed of four computing units in the example of the figure. The first to fourth computing units from the top output provisional estimated values ​​(i.e., intermediate value set 56) corresponding to the first to fourth explanatory variables.

[0046] The layer LY4 corresponds to the second hidden layer of the second estimator E2 (or the output layer of the first calculator E21 and the input layer of the second calculator E22), and is composed of four calculation units in the example shown in the figure. The first to fourth calculation units from the top each output an interaction value indicating an interaction between tentative estimated values.

[0047] The layer LY5 corresponds to the second output layer of the second estimator E2 (or the output layer of the second computing unit E22), and in the example shown in the figure, is composed of one computing unit. This computing unit outputs the final estimated value.

[0048] [Operation of information processing device 10] The information processing device 10 in this embodiment is configured as described above. Next, the learning operation and estimation operation by this information processing device 10 will be described with reference to FIGS.

[0049] <Learning behavior> Fig. 3 is a first flowchart showing an example of a learning operation by the information processing device 10 in Fig. 1. Fig. 4 is a second flowchart showing an example of a learning operation by the information processing device 10 in Fig. 1.

[0050] (SP10: Acquisition step) In step SP10 of FIG. 3, the acquisition unit 24 acquires a learning data set 40 used in the learning process.

[0051] (SP12:Normalization step) In step SP12, the regression calculation section 26 normalizes each of the variable value set 50 and the correct answer value 52 included in the learning data set 40 acquired in step SP10.

[0052] FIG. 5 is a diagram showing an example of a scatter plot of explanatory variables and response variables. The horizontal axis of the scatter plot is the normalized explanatory variables {x i} (unit: dimensionless), and the vertical axis of the scatter plot shows the normalized objective variable (unit: dimensionless). Each variable is normalized within the range of [0,1]. As can be seen from Figure 5, the objective variables have a relationship that seems to be approximated by a cubic function with the explanatory variables as arguments.

[0053] (SP14: Provisional division step) 3, the regression calculation section 26 provisionally divides the interval of each explanatory variable normalized in step SP12. By this provisional division, the lower limit and upper limit of the interval are determined for each explanatory variable.

[0054] (SP16: Simple regression step) In step SP16, the regression calculation section 26 performs a simple regression calculation for each explanatory variable according to the intervals provisionally divided in step SP14, and calculates a regression coefficient (that is, the regression coefficient set 42) for each explanatory variable.

[0055] Fig. 6 is a diagram showing an example of a broken line function obtained by a simple regression calculation for the scatter diagram of Fig. 4. This broken line function is shown by a broken line PL consisting of three line elements L1, L2, and L3. Line element L1 corresponds to a line segment having an interval [0, x1], a slope of α (positive), and an intercept of y0. Line element L2 corresponds to a line segment having an interval [x1, x2], a slope of β (negative), and an intercept of y1. Line element L3 corresponds to a line segment having an interval [x2, 1], a slope of γ (positive), and an intercept of y2.

[0056] (SP18: Setting display step) 3, the display control unit 32 generates display data including information related to the settings of the learning process, and supplies the display data to the display unit 16. As a result, the learning setting screen 70 of FIG. 7 is displayed on the UI unit 22 of the information processing device 10.

[0057] Fig. 7 is a diagram showing an example of a learning setting screen 70 displayed on the UI unit 22 of Fig. 1. On the learning setting screen 70, a graph field 72, four types of information input sections 74, 76, 78, and 80, a button 82 labeled [Simple Regression], and a button 84 labeled [Optimization] are provided.

[0058] In the graph field 72, the scatter diagram of FIG. 4 is visualized and displayed. The graph field 72 is configured to include a point group 86, an approximation curve 88, and one or more division lines 90. The number or position of the point group 86 changes according to the input contents from the information input units 74, 76. The shape of the approximation curve 88 changes according to the input contents from the information input units 74, 76, 78, 80. The number of division lines 90 changes according to the input contents from the information input unit 80. The position of the division line 90 changes according to a slide operation by the user.

[0059] The information input unit 74 is composed of a user control (e.g., a pull-down menu) for selecting the type of explanatory variable. The information input unit 76 is composed of a user control (e.g., a combo box) for selecting the number of data points in the point cloud 86. The information input unit 78 is composed of a user control (e.g., a pull-down menu) for selecting the degree of the approximation curve 88. The information input unit 80 is composed of a user control (e.g., a pull-down menu) for selecting the number of interval divisions. The [Simple Regression] button 82 is a user control for accepting an operation to change the interval. The [Optimize] button 84 is a user control for accepting an operation to determine the interval.

[0060] (SP20: Decision acceptance step) 3, the control unit 18 checks whether or not a section determination operation has been received. If the determination operation has not been received (step SP20: NO), the control unit 18 proceeds to the next step SP22.

[0061] (SP22: Change acceptance step) In step SP22, the control unit 18 checks whether or not a section change operation has been accepted. If a change operation has not been accepted yet (step SP22: NO), the control unit 18 returns to step SP20, and thereafter, steps SP20 and SP22 are sequentially repeated.

[0062] For example, while viewing the graph field 72 on the learning setting screen 70 (FIG. 7), the user checks the suitability of the approximation curve 88, i.e., whether or not the approximation curve 88 of the point group 86 is accurately approximated. If the user determines that the approximation curve 88 is not suitable, the user changes the number of data, the order, the number of divisions, or the position of the division line 90, and then taps the [Simple Regression] button 82. Then, the control unit 18 accepts the operation to change the interval (step SP22: YES), and proceeds to the next step SP24.

[0063] (SP24: Repartition step) In step SP24, the regression calculation unit 26 re-divides the intervals of each explanatory variable based on the contents input to the UI unit 22 at the time when the change operation was accepted in step SP22. This re-division changes the lower and upper limits of the intervals for each explanatory variable. Thereafter, the control unit 18 returns to step SP16, and thereafter, steps SP16 to SP24 are sequentially repeated.

[0064] When the user judges that the approximation curve 88 is suitable while viewing the graph field 72 on the learning setting screen 70 (FIG. 7), the user performs a tap operation of the [Optimize] button 84. Then, the control unit 18 accepts the operation of determining the section (step SP20: YES), and then proceeds to the next step SP26.

[0065] (Step SP26: Construction step) In step SP26, the learning processing unit 30 constructs an estimation model 36 from the contents (e.g., the number of explanatory variables, the number of interval divisions) input to the UI unit 22 at the time of receiving the decision operation in step SP24. In this way, the estimation model 36 shown in FIG. 2 is constructed.

[0066] (Step SP28: Initialization step) 4, the learning processing unit 30 initializes the learning parameter group 44 for the estimation model 36 constructed in step SP26. Specifically, the learning processing unit 30 defines the learning parameters that constitute the learning parameter group 44, and assigns initial values ​​to each learning parameter.

[0067] FIG. 8 is a diagram showing an example of variable parameters corresponding to the broken line function in FIG. 6. In the example of FIG. 8, it is assumed that the activation function of the arithmetic unit is a ReLU function. In the activation function (F1) corresponding to the first interval, the threshold is θ=0, the multiplier is C=α, and the offset (or bias) is S=y0. In the activation function (F2) corresponding to the second interval, the threshold is θ=x1, the multiplier is C=β-α, and the offset is S=y0-F2(x1). In the activation function (F3) corresponding to the third interval, the threshold is θ=x2, the multiplier is C=γ-(α+β), and the offset is S=y2-F1(x2)-F2(x2).

[0068] In the example of FIG. 2, for the first estimator E1, the weighting coefficients between the i-th arithmetic unit in layer LY1 and the (3·i+j)-th arithmetic unit in layer LY2 are all set to 1. The weighting coefficient between the 5-th arithmetic unit in layer LY1 and the (3·i+j)-th arithmetic unit in layer LY2 is set to 1 by a threshold {θ i,j The weighting coefficient between the i-th arithmetic unit in layer LY2 and the (3·i+j)-th arithmetic unit in layer LY3 is set to the multiplier {C i,j The weighting coefficient between the 13th arithmetic unit in layer LY2 and the i-th arithmetic unit in layer LY3 is set to the offset {S i}.

[0069] In the example of FIG. 2, for the second estimator E2, the weighting coefficient between the i-th arithmetic unit in layer LY3 and the j-th arithmetic unit in layer LY4 is the Kronecker delta {δ i,j}.

[0070] (SP30: Learning Step) 4, the learning processing unit 30 performs a learning process on the estimation model 36 from the state initialized in step SP28. Through this learning process, each value of the learning parameter group 44 is determined.

[0071] (SP32: Result display step) In step SP32, the display control unit 32 generates display data including information related to the result of the learning process, and supplies the display data to the display unit 16. As a result, a learning result screen (not shown) is displayed on the UI unit 22 of the information processing device 10.

[0072] (SP34: Result confirmation step) In step SP34, the control unit 18 checks whether or not an operation to end learning has been received. If the operation to end learning has not been received (step SP34: NO), the control unit 18 returns to step SP20 in FIG. 3. Thereafter, the control unit 18 re-divides (SP22) and determines (SP20) the section as necessary, and then sequentially repeats steps SP26 to SP34. On the other hand, if the operation to end learning has been received (step SP34: YES), the control unit 18 proceeds to the next step SP36.

[0073] (SP36: Save step) In step SP36, the learning processing unit 30 stores the learning parameter group 44 determined in step SP30 in the storage unit 20. This allows the estimation unit 28 to perform estimation processing using the estimation model 36 by reading and setting the learning parameter group 44.

[0074] <Estimated behavior> FIG. 9 is a flowchart showing an example of the estimation operation by the information processing device 10 of FIG.

[0075] (SP40: Confirmation step) In step SP40, the acquisition unit 24 checks whether the timing to acquire data (i.e., acquisition timing) has arrived. If the acquisition timing has not arrived yet (step SP40: NO), the acquisition unit 24 remains in step SP40 until the acquisition timing arrives. After that, if the acquisition timing arrives (step SP40: YES), the acquisition unit 24 proceeds to the next step SP42.

[0076] (SP42: Acquisition step) In step SP42, the acquisition section 24 acquires the variable value set 54 used for the estimation.

[0077] (SP44: Normalization step) In step SP44, the estimation section 28 normalizes each value of the variable value set 54 acquired in step SP40 according to the same calculation rules as in step SP12 (FIG. 3).

[0078] (SP46: Estimation step) In step SP46, the estimation unit 28 estimates the dependent variable by inputting the variable value set 54 normalized in step SP44 to the estimation model 36. As a result, an intermediate value set 56 is obtained as an intermediate product, and an estimated value 58 is obtained as a deliverable.

[0079] (SP48: Output step) In step SP48, the output processing unit 34 outputs the estimated value 58 obtained in step SP46 or a generated value generated from the estimated value 58. In this manner, the estimation operation by the information processing device 10 ends.

[0080] [Summary of the embodiment] As described above, according to the information processing device 10 and the information processing program in this embodiment, the first estimation process for obtaining a set of provisional estimates of the dependent variable (i.e., the intermediate value set 56) using a simple regression model for each explanatory variable from the variable value set 54, and the second estimation process for obtaining an estimate of the dependent variable using the intermediate value set 56 are sequentially executed, thereby making it possible to explicitly separate the previous estimation that is not affected by the interaction between the explanatory variables from the later estimation that takes into account the interaction between the provisional estimates. This enables estimation that is less affected by the magnitude or tendency of the interaction, and can further improve the estimation accuracy of the dependent variable.

[0081] The simple regression model may be a regression model in which the regression equation is a broken line function formed by connecting multiple line segments (line elements L1 to L3). This makes it possible to express even a nonlinear model having a complex function shape by a set of simpler linear models, thereby improving the suitability of the regression equation.

[0082] The information processing device 10 may further include a UI unit 22 for displaying a scatter diagram (here, a graph field 72) showing the distribution of the objective variable with respect to the explanatory variables, and for receiving an input operation via the graph field 72 to divide the scatter diagram into a plurality of intervals corresponding to a plurality of line segments (line elements L1 to L3). This allows the user to easily perform an operation of dividing the intervals while visually checking the scatter diagram via the UI unit 22.

[0083] The second estimation process may include a first calculation for calculating a plurality of interaction values ​​indicating interactions between the intermediate value sets 56, and a second calculation for calculating an estimated value of the response variable by applying a contribution rate to the plurality of interaction values ​​calculated by the first calculation. By explicitly separating the calculation of the interaction value from the effect of the contribution rate, estimation that is less susceptible to the influence of interactions can be performed.

[0084] The estimation unit 28 may estimate the objective variable using an estimation model 36 in which a first estimator E1 and a second estimator E2 are connected in series. In this case, the first estimator E1 is a hierarchical neural network including a first input layer (layer LY1), a first hidden layer (layer LY2), and a first output layer (layer LY3), and the activation function of the arithmetic unit constituting the layer LY2 has a linear or broken-line functional shape. Through the estimation by the first estimator E1, an estimation operation using a broken-line function as a regression equation can be reproduced in the hierarchical neural network.

[0085] The information processing device 10 may further include a regression calculation unit 26 that uses a learning dataset 40 to calculate a regression coefficient set 42, which is a simple regression coefficient for each explanatory variable and for each interval, and a learning processing unit 30 that sets the regression coefficient set 42 as an initial value of a variable parameter that specifies a calculation rule in the first estimator E1 and performs a learning process to update the variable parameter for the estimation model 36. This makes it possible to give the first estimator E1 an optimal solution or an initial condition close to the optimal solution, thereby improving the convergence and accuracy of the learning process. In particular, the estimation model 36 has an advantage that the gradient vanishing problem or overlearning is less likely to occur because it is composed of a relatively small number of layers (five layers LY1 to LY5).

[0086] [Variations] The present invention is not limited to the above-described embodiment, and can be freely modified without departing from the spirit of the present invention. Alternatively, the configurations may be combined in any way without causing any technical inconsistency. Alternatively, the order of execution of the steps constituting the flowchart or the presence or absence of execution may be changed without causing any technical inconsistency.

[0087] In the above embodiment, the information processing device 10 is described as having both the estimation unit 28 and the learning processing unit 30 (FIG. 1), but the device configuration is not limited to this. For example, when the information processing device 10 has only the function of the estimation unit 28, it may download a learning parameter set 44 from an external device (not shown) and perform estimation processing using the learning parameter set 44. Alternatively, when the information processing device 10 has only the function of the learning processing unit 30, it may supply the learning parameter set 44 generated through the learning processing to an external device (not shown). [Explanation of symbols]

[0088] 10...information processing device, 18...control unit, 20...storage unit, 22...UI unit, 24...acquisition unit, 26...regression calculation unit, 28...estimation unit, 30...learning processing unit, 32...display control unit, 34...output processing unit, 36...estimation model, 40...learning data set, 42...regression coefficient set, 44...learning parameter group, 46...estimation result information, 54...variable value set, 56...intermediate value set, 58...estimated value, 70...learning setting screen, E1...first estimator, E2...second estimator, E21...first computing unit, E22...second computing unit, SP42...acquisition step, SP46...estimation step

Claims

1. an acquisition unit that acquires a set of variable values ​​corresponding to a plurality of explanatory variables used in the estimation; a UI unit for displaying a scatter diagram showing a distribution of a response variable with respect to the explanatory variables and for receiving an input operation via the scatter diagram to divide a range of the explanatory variables into a plurality of intervals; an estimation unit that executes a first estimation process of estimating a single objective variable from the variable value set acquired by the acquisition unit using a simple regression model for each explanatory variable, the regression equation of which is a broken line function formed by connecting a plurality of line segments corresponding to the plurality of intervals divided through the UI unit, and obtaining an intermediate value set that is a collection of provisional estimates of the objective variable, and a second estimation process of estimating the objective variable using the intermediate value set and obtaining an estimate of the objective variable; An information processing device comprising:

2. an acquisition unit that acquires a set of variable values ​​corresponding to a plurality of explanatory variables used in the estimation; an estimation unit that executes a first estimation process of estimating a single dependent variable from the variable value sets acquired by the acquisition unit using a simple regression model for each of the explanatory variables, and obtaining an intermediate value set that is a collection of provisional estimates of the dependent variable, and a second estimation process of estimating the dependent variable using the intermediate value set, and obtaining an estimate of the dependent variable; Equipped with The information processing device characterized in that the second estimation process includes a first calculation of calculating a plurality of interaction values ​​indicating interactions between the intermediate value sets, and a second calculation of applying a contribution rate to the plurality of interaction values ​​calculated by the first calculation to obtain the estimated value of the dependent variable.

3. an acquisition unit that acquires a set of variable values ​​corresponding to a plurality of explanatory variables used in the estimation; an estimation unit that executes a first estimation process of estimating a single objective variable from the variable value set acquired by the acquisition unit using a simple regression model for each explanatory variable, the regression equation of which is a broken line function formed by connecting a plurality of line segments, and obtaining an intermediate value set that is a collection of provisional estimates of the objective variable, and a second estimation process of estimating the objective variable using the intermediate value set and obtaining an estimate of the objective variable; Equipped with the estimation unit estimates the dependent variable using an estimation model in which a first estimator for executing the first estimation process and a second estimator for executing the second estimation process are connected in series; the first estimator is a hierarchical neural network including a first input layer, a first hidden layer, and a first output layer; An information processing device, characterized in that the activation function of the arithmetic units constituting the first hidden layer has a linear or polygonal functional shape.

4. a regression calculation unit that uses a learning dataset that is a collection of the explanatory variables and the target variable to determine a plurality of intervals for each of the explanatory variables, performs a simple regression calculation for each of the intervals, and calculates a regression coefficient set that is a simple regression coefficient for each of the explanatory variables and for each of the intervals; a learning processing unit that sets the regression coefficient set obtained by the regression calculation unit as an initial value of a variable parameter that specifies a calculation rule in the first estimator, and performs a learning process to update the variable parameter for the estimation model; The information processing apparatus according to claim 3 , further comprising:

5. An acquisition step of acquiring a set of variable values ​​corresponding to a plurality of explanatory variables used in the estimation; a division step of displaying a scatter plot showing a distribution of a response variable with respect to the explanatory variables through a graphical user interface and dividing a range of the explanatory variables into a plurality of intervals by accepting an input operation via the scatter plot; an estimation step of executing a first estimation process of estimating a single dependent variable from the acquired variable value set using a simple regression model for each explanatory variable, the regression equation of which is a broken line function formed by connecting a plurality of line segments corresponding to the divided plurality of intervals, and obtaining an intermediate value set which is a collection of provisional estimated values ​​of the dependent variable; and a second estimation process of estimating the dependent variable using the intermediate value set, and obtaining an estimated value of the dependent variable. An information processing program characterized by causing one or more computers to execute the above.

6. An acquisition step of acquiring a set of variable values ​​corresponding to a plurality of explanatory variables used in the estimation; an estimation step of executing a first estimation process of estimating a single objective variable from the acquired variable value set using a simple regression model for each explanatory variable, the regression equation of which is a broken line function formed by connecting a plurality of line segments, and obtaining an intermediate value set that is a collection of provisional estimates of the objective variable; and a second estimation process of estimating the objective variable using the intermediate value set and obtaining an estimate of the objective variable. on one or more computers, the second estimation process includes a first calculation for calculating a plurality of interaction values ​​indicating interactions between the intermediate value sets, and a second calculation for applying a contribution rate to the plurality of interaction values ​​calculated by the first calculation to obtain the estimated value of the dependent variable.

7. An acquisition step of acquiring a set of variable values ​​corresponding to a plurality of explanatory variables used in the estimation; an estimation step of executing a first estimation process of estimating a single objective variable from the acquired variable value set using a simple regression model for each explanatory variable, the regression equation of which is a broken line function formed by connecting a plurality of line segments, and obtaining an intermediate value set that is a collection of provisional estimates of the objective variable; and a second estimation process of estimating the objective variable using the intermediate value set and obtaining an estimate of the objective variable. on one or more computers, the estimation step estimates the dependent variable using an estimation model in which a first estimator for executing the first estimation process and a second estimator for executing the second estimation process are connected in series; the first estimator is a hierarchical neural network including a first input layer, a first hidden layer, and a first output layer; An information processing program, wherein the activation function of the arithmetic units constituting the first hidden layer has a linear or polygonal functional shape. An information processing program comprising:

8. A calculation step of determining a plurality of intervals for each of the explanatory variables using a learning data set which is a collection of the explanatory variables and the target variable, performing a simple regression calculation for each of the intervals, and calculating a regression coefficient set which is a simple regression coefficient for each of the explanatory variables and for each of the intervals; a learning step of setting the calculated regression coefficient set as an initial value of a variable parameter that specifies an operation rule in the first estimator, and performing a learning process to update the variable parameter for the estimation model; 8. The information processing program according to claim 7, further comprising:

Images